is socket thread safe?

e2*****@gmail.com:

[code]

I certainly expect socket to be threadsafe. I use it (via urllib2/httplib)
in a multithreaded program, that runs fine with Python 2.3 and 2.4 on both
Windows XP and Linux.

--
René Pijlman

e2*****@gmail.com wrote:

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

It is safe, but watch out for this gotcha: If thread B calls
s.close() while thread A is blocked in s.read(), thread A will
never return from the read. My preferred solution is to set
socket timeout to a few seconds, and loop checking a status flag
so I know when to quit.

Steve

Steve Horsley wrote:

e2*****@gmail.com wrote:

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

It is safe, but watch out for this gotcha: If thread B calls
s.close() while thread A is blocked in s.read(), thread A will
never return from the read. My preferred solution is to set
socket timeout to a few seconds, and loop checking a status flag
so I know when to quit.

I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.

i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.

To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.
-carl

--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software

e2*****@gmail.com wrote:

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

Sockets don't have read() and write() methods. Connected
sockets have recv() and send()/sendall(). Python's socket
module has a makefile() function, but the resulting file
object wouldn't make sense in the read loop above (and
throws away control that network code typically needs).

Is it safe for one thread to receive from a socket while
another sends over the socket? Yes, that much is safe and
perfectly reasonable.
--
--Bryan

In article <F5*******************@newssvr29.news.prodigy.net> ,
Bryan Olson <fa*********@nowhere.org> wrote:

Is it safe for one thread to receive from a socket while
another sends over the socket? Yes, that much is safe and
perfectly reasonable.

I hear it works on most common platforms these days,
anyway. I have seen socket implementations where it
didn't, though - the read would end as if interrupted
by a signal.

Donn Cave, do**@u.washington.edu

Carl J. Van Arsdall wrote:

Steve Horsley wrote:

e2*****@gmail.com wrote:

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

It is safe, but watch out for this gotcha: If thread B calls s.close()
while thread A is blocked in s.read(), thread A will never return from
the read. My preferred solution is to set socket timeout to a few
seconds, and loop checking a status flag so I know when to quit.

Certainly one needs timeouts to avoid hanging should the remote
side stop. Sockets don't have a read() method, and hanging on
recv() doesn't seem to have anything to do with close().

I didn't find any definitive doc, so I tested using Python
sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
started before the close() will block/return just as if
close() were never called. The close() neither triggers recv()
to abort, nor prevents it from receiving data and detecting
shutdown.

I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.

i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.
I can infer two producer-consumer relationships from the example,
but they don't allow a condition object; the writer's consumer and
the reader's producer are on the remote end of the socket. The
socket will already handle the blocking and wake-up.
To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.

What problem are you trying to solve? Normal socket sending, receiving,
and shutdown discipline work fine. When the writer is done writing, it
should call sock.shutdown(socket.SHUT_WR). When the reader gets zero
bytes from recv(nonzero), that means the remote end has finished
writing, so the reader may call sock.shutdown(socket.SHUT_RD).
--
--Bryan

Bryan Olson wrote:

Carl J. Van Arsdall wrote:

Steve Horsley wrote:

e2*****@gmail.com wrote:

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

It is safe, but watch out for this gotcha: If thread B calls s.close()
while thread A is blocked in s.read(), thread A will never return from
the read. My preferred solution is to set socket timeout to a few
seconds, and loop checking a status flag so I know when to quit.

Certainly one needs timeouts to avoid hanging should the remote
side stop. Sockets don't have a read() method, and hanging on
recv() doesn't seem to have anything to do with close().

I didn't find any definitive doc, so I tested using Python
sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
started before the close() will block/return just as if
close() were never called. The close() neither triggers recv()
to abort, nor prevents it from receiving data and detecting
shutdown.

I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.

i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.

I can infer two producer-consumer relationships from the example,
but they don't allow a condition object; the writer's consumer and
the reader's producer are on the remote end of the socket. The
socket will already handle the blocking and wake-up.

To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.

What problem are you trying to solve? Normal socket sending, receiving,
and shutdown discipline work fine. When the writer is done writing, it
should call sock.shutdown(socket.SHUT_WR). When the reader gets zero
bytes from recv(nonzero), that means the remote end has finished
writing, so the reader may call sock.shutdown(socket.SHUT_RD).

Doh! I read the word threads and got carried away not even realizing
sockets. Well, looks like today i'll just have to remember to drink my
coffee

:-D

-c

--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software