Nick Vargish wrote:
A book that covers all the above options would really help me figure
out the optimal approach to this project. I'm especially concerned
that the GIL will make a pure-Python approach risky, since this
application involves a fair abount of network data and real-time data
processing.
What risk concerns you with respect to doing networking and "real-time"
processing using Python, given that the GIL exists? Would you have the
same concerns if you didn't even know of the existence of the GIL?
(I believe you should have those concerns, but just because you're
trying to use an unfamiliar environment for something fairly intensive,
not because it's Python specifically, or the GIL.)
I do lots of "network data and real-time data processing" using Python
and have not encountered any particular difficulties, and certainly none
that can be attributed to the existence of the GIL.
Is this a case of fear arising out of ignorance? If so, why not just
write a quick test/experiment that will remove or affirm your concern?
It would be trivial to write a program that created a dozen threads,
each doing processing that would take five seconds of dedicated CPU time,
and a network thread which grabs data from an external source as it
becomes available, at whatever rate you need. You'll likely find out
that with an adequate CPU, everything will work just as you need it to,
and that the only difference between the old program and the new is that
the Python-based one consumes a lot more CPU time for the same data...
If you have specific reasons for the concern, please provide them so
we can address them. The GIL certainly doesn't get in the way of
soft real-time work in any way that's significant.
-Peter
