469,658 Members | 1,862 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,658 developers. It's quick & easy.

how to do non-blocking I/O

I am a newbie to python. I am writing a client application, I need to
read query results from the server. Mostly I'll have a file object for
the connection to the server. So I use read() or readline() to get the
response. But since these are blocking calls, I have to use a seperate
thread for the input reading. But then I have another problem, my input
thread running in the following loop

while not quit:
str = ins.readline()
.... process the response

so I can't terminate the thread when it's blocking in the readline()
until the connection is closed. So looks like I need to use non-blocking
IO such as select() or poll(). But since I need this application to be
cross-platform, and these are not well supported on Windows, I don't
know what to do.
Jul 18 '05 #1
2 2415
In article <ma************************************@python.org >,
"J. Xu" <xu@reflexsecurity.com> wrote:
I am a newbie to python. I am writing a client application, I need to
read query results from the server. Mostly I'll have a file object for
the connection to the server. So I use read() or readline() to get the
response. But since these are blocking calls, I have to use a seperate
thread for the input reading. But then I have another problem, my input
thread running in the following loop

while not quit:
str = ins.readline()
.... process the response

so I can't terminate the thread when it's blocking in the readline()
until the connection is closed. So looks like I need to use non-blocking
IO such as select() or poll(). But since I need this application to be
cross-platform, and these are not well supported on Windows, I don't
know what to do.


Far as I know, what you want to do may not be possible with
portable functions, so you may have to just implement separately
for each platform.

If you decide to use select or poll, note that they do not
account for data buffered in a file object. This loses data,
in effect, because even though it's in your process buffer,
you won't read it if select says there's no new data. You
can turn buffering off, but that forces readline() to read
1 byte at a time at an unreasonable cost in system calls.
In my opinion it makes much more sense to use recv on the
socket, and not use a file object here.

Donn Cave, do**@u.washington.edu
Jul 18 '05 #2
J. Xu wrote:
I am a newbie to python. I am writing a client application, I need to
read query results from the server. Mostly I'll have a file object for
the connection to the server. So I use read() or readline() to get the
response. But since these are blocking calls, I have to use a seperate
thread for the input reading. But then I have another problem, my input
thread running in the following loop

while not quit:
str = ins.readline()
.... process the response

so I can't terminate the thread when it's blocking in the readline()
until the connection is closed. So looks like I need to use non-blocking
IO such as select() or poll(). But since I need this application to be
cross-platform, and these are not well supported on Windows, I don't
know what to do.


select() is perfectly well supported (to a reasonable degree anyway) on
Windows, at least with actual sockets. Don't know whether it applies to
file objects attached to sockets, and never had cause to check...

If you can use the sockets themselves, with .recv() instead, then you
can safely use select() even on Windows.

-Peter
Jul 18 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

25 posts views Thread by Yves Glodt | last post: by
32 posts views Thread by Adrian Herscu | last post: by
8 posts views Thread by Bern McCarty | last post: by
399 posts views Thread by =?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.