Socket recv(1) seems to block instead of returning end of file.

[Hendrik van Rooyen]

While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs, despite having a short
time out of 0.1 set.

If however, you try to receive more than one char, (I tested with 3,
did not try 2), then when you shut the remote end down you do not
get a time out, but an empty string - the normal end of file, I suppose.

Has anybody else seen this behaviour?

The transmit side seems to give a broken pipe error, which is fine.

I am using stock standard SuSe 10, Python 2.4, out of the box.

- Hendrik

[Grant Edwards]

On 2007-08-23, Hendrik van Rooyen <ma**@microcorp.co.zawrote:

While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs,

I don't see that behavior running 2.4 on Gentoo.

despite having a short time out of 0.1 set.

What time out? A socket's recv method doesn't do timeouts.

If however, you try to receive more than one char, (I tested
with 3, did not try 2), then when you shut the remote end down
you do not get a time out, but an empty string - the normal
end of file, I suppose.

Has anybody else seen this behaviour?

No. recv(1) works fine for me (Python 2.4 under Gentoo).
Perhaps you could post a minimal example that doesn't work for
you?

------------------------------------------------------------------------
#!/usr/bin/python

#reader
import socket

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1)
print "rx:",len(data)
if not data: break
conn.close()
------------------------------------------------------------------------
------------------------------------------------------------------------
#!/usr/bin/python

# writer
import socket,random

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
print 'Connected to',((HOST,PORT))
for i in range(10):
data = "abcdefghijklmnopqrstuvwxyz"[:random.randint(1,20)]
s.send(data)
print "tx:",len(data)
conn.close()
------------------------------------------------------------------------

--
Grant Edwards grante Yow! Loni Anderson's hair
at should be LEGALIZED!!
visi.com

[Dan Stromberg - Datallegro]

On Thu, 23 Aug 2007 11:54:01 +0200, Hendrik van Rooyen wrote:

>
While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs, despite having a short
time out of 0.1 set.

If however, you try to receive more than one char, (I tested with 3,
did not try 2), then when you shut the remote end down you do not
get a time out, but an empty string - the normal end of file, I suppose.

Has anybody else seen this behaviour?

The transmit side seems to give a broken pipe error, which is fine.

I am using stock standard SuSe 10, Python 2.4, out of the box.

- Hendrik

Are you using sock.settimeout()?

I've always done timed-out sockets in python using select; IINM, the
settimeout method is a new addition.

I agree with Grant though - posting a minimal snippet of code that
replicates the problem would help us help you. In fact, it might help you
help yourself :)

[Grant Edwards]

On 2007-08-23, Dan Stromberg - Datallegro <ds********@datallegro.comwrote:

Are you using sock.settimeout()?

Hey, somebody snuck timeouts into the socket module when I wasn't
looking...

I agree with Grant though - posting a minimal snippet of code that
replicates the problem would help us help you. In fact, it might help you
help yourself :)

Yup, in my experience attempting to produce a minimally failing
example will usually reveal what's wrong.

--
Grant Edwards grante Yow! I have seen these EGG
at EXTENDERS in my Supermarket
visi.com ... I have read the
INSTRUCTIONS ...

[Grant Edwards]

On 2007-08-23, Dan Stromberg - Datallegro <ds********@datallegro.comwrote:

On Thu, 23 Aug 2007 11:54:01 +0200, Hendrik van Rooyen wrote:

>>
While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs, despite having a short
time out of 0.1 set.

If however, you try to receive more than one char, (I tested with 3,
did not try 2), then when you shut the remote end down you do not
get a time out, but an empty string - the normal end of file, I suppose.

Has anybody else seen this behaviour?

The transmit side seems to give a broken pipe error, which is fine.

I am using stock standard SuSe 10, Python 2.4, out of the box.

- Hendrik

Are you using sock.settimeout()?

FWIW, I added a call to sock.setttimeout(0.2) to my example's
receive code and added 0.1 delays in the transmit code, and it
still seems to work fine:

-------------------------------8<-------------------------------------
#!/usr/bin/python

# writer
import socket,random,time

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
print 'Connected to',((HOST,PORT))
total = 0
for i in range(10):
data = "abcdefghijklmnopqrstuvwxyz"[:random.randint(1,20)]
total += len(data)
s.send(data)
print "tx:",len(data)
time.sleep(0.1)
s.close()
print total
-------------------------------8<-------------------------------------
#!/usr/bin/python

#reader
import socket

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
conn.settimeout(0.2)
total = 0
while 1:
data = conn.recv(1)
total += len(data)
print "rx:",len(data)
if not data: break
conn.close()
print total
-------------------------------8<-------------------------------------

--
Grant Edwards grante Yow! We have DIFFERENT
at amounts of HAIR --
visi.com

[Bryan Olson]

Grant Edwards wrote:
[...]

import socket,random

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

Actually the empty string passed for the host means INADDR_ANY,
which does not really make sense for the client. On my Win-XP
system, the client fails with

socket.error: (10049, "Can't assign requested address")

For the loop-back adapter, 'localhost' or '127.0.0.1' should
work.
--
--Bryan

[Grant Edwards]

On 2007-08-24, Bryan Olson <fa*********@nowhere.orgwrote:

Grant Edwards wrote:
[...]

>import socket,random

HOST = '' # Symbolic name meaning the local host
PORT = 8765 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

Actually the empty string passed for the host means INADDR_ANY,

You're correct. I copied that code from the server example on
the Python socket module doc page, and forgot to change the host.

http://docs.python.org/lib/socket-example.html

which does not really make sense for the client. On my Win-XP
system, the client fails with

socket.error: (10049, "Can't assign requested address")

It's interesting that it works on Linux.

For the loop-back adapter, 'localhost' or '127.0.0.1' should
work.

Yup.

--
Grant Edwards grante Yow! YOU PICKED KARL
at MALDEN'S NOSE!!
visi.com

[Hendrik van Rooyen]

"Dan Stromberg - Datallegro" <dstr....tallegro.comwrote:

>
Are you using sock.settimeout()?

Yes.

>
I've always done timed-out sockets in python using select; IINM, the
settimeout method is a new addition.

I agree with Grant though - posting a minimal snippet of code that
replicates the problem would help us help you. In fact, it might help you
help yourself :)

I will try - in the interim, please see my reply to Grant

- Hendrik

[Hendrik van Rooyen]

"Grant Edwards" <gra...si.comwrote:

On 2007-08-23, Dan Stromberg - Datallegro <dstr....gro.comwrote:

Are you using sock.settimeout()?

Hey, somebody snuck timeouts into the socket module when I wasn't
looking...

I agree with Grant though - posting a minimal snippet of code that
replicates the problem would help us help you. In fact, it might help you
help yourself :)

Yup, in my experience attempting to produce a minimally failing
example will usually reveal what's wrong.

I am now extremely frustrated - I have spent most of the day mucking
around with this, and now I have managed to lose the version that
was giving me grief, and I can't remember what I have done.

At the moment everything works, and detects remote comm loss,
no matter what I do.

So I must have done something stupid, and destroyed the evidence.

Some mothers have children.
Others have bad luck.

My apologies for chasing you up like that, and thanks to all who responded.

- Hendrik

[Hendrik van Rooyen]

"Grant Edwards" <gr....comwrote:

On 2007-08-23, Hendrik van Rooyen <m....ocorp.co.zawrote:

While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs,

I don't see that behavior running 2.4 on Gentoo.

I express myself badly - when I say "close down" I don't mean
"close down as in socket.close" I mean "close down as in click on the
tkinter window, or with keyboard interrupt" - sorry for the confusion

>

despite having a short time out of 0.1 set.

What time out? A socket's recv method doesn't do timeouts.

If I set a time out, then the recv does not block,
but gives me a timed out exception.

>

If however, you try to receive more than one char, (I tested
with 3, did not try 2), then when you shut the remote end down
you do not get a time out, but an empty string - the normal
end of file, I suppose.

Has anybody else seen this behaviour?

No. recv(1) works fine for me (Python 2.4 under Gentoo).
Perhaps you could post a minimal example that doesn't work for
you?

I am now frustrated - its not as simple as I thought - my application
still only detects EOF if I do more than recv(1), but I cannot
demonstrate simply - anyway, I have modified your code to look more
like my program's code, and it follows below - if it does nothing else,
it should convince you that recv() *does* time out...

The receiver code below detects if I close the sending console window,
as well as control-c - Rats!

I will muck around with this more to see if I can get to the bottom of it,
as I can't see anything different between what is below and my program,
except that in both cases the comms are done in threads which are not
the main loop.

- Hendrik

------------------------------------------------------------------------
#!/usr/bin/python

#reader
import socket

HOST = 'Linuxbox' # Symbolic name meaning the local host
PORT = 57001 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
socket.setdefaulttimeout(0.100)

s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
data = ''
while 1:
try:
data = data + conn.recv(1)
except socket.error,msg:
if 'timed out' in msg:
print msg
continue
else:
print 'socket error is',msg
break
if not data: break
if data.endswith('\n'):
print data
data = ''
print 'heigh ho the end has come!'
conn.close()
------------------------------------------------------------------------
------------------------------------------------------------------------
#!/usr/bin/python

# writer
import socket,time,random

HOST = 'Linuxbox' # Symbolic name meaning the local host
PORT = 57001 # Arbitrary non-privileged port
socket.setdefaulttimeout(0.100)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

while True:
try:
print 'Attempting to connect to server'
s.connect((HOST, PORT))
break
except socket.error, msg:
time.sleep(5)
continue
print 'Connected - Time out is:',s.gettimeout()

data = "The quick brown fox jumps over the lazy dog 0123456789\n"
count = 0
while count < 500:
try:
s.send(data)
except socket.error,msg:
if 'timed out' in msg:
print msg
time.sleep(0.05)
continue
else:
print 'socket error is:',msg
break
print "tx:",len(data)
time.sleep(random.randint(200,800)/1000.0)
s.close()

------------------------------------------------------------------------

[Grant Edwards]

On 2007-08-24, Hendrik van Rooyen <ma**@microcorp.co.zawrote:

"Grant Edwards" <gr....comwrote:

>On 2007-08-23, Hendrik van Rooyen <m....ocorp.co.zawrote:

While doing a netstring implementation I noticed that if you
build a record up using socket's recv(1), then when you close
the remote end down, the recv(1) hangs,

I don't see that behavior running 2.4 on Gentoo.

I express myself badly - when I say "close down" I don't mean
"close down as in socket.close" I mean "close down as in click on the
tkinter window, or with keyboard interrupt" - sorry for the confusion

A FIN is a FIN is a FIN. It shouldn't matter whether the app
called close() or the c library called close() or the OS called
close().

despite having a short time out of 0.1 set.

What time out? A socket's recv method doesn't do timeouts.

If I set a time out, then the recv does not block,
but gives me a timed out exception.

My aplogies -- my question was prompted by my ignorance of the
fact that a timeout feature had been added to the socket module
since the last time I worked on it.

I am now frustrated - its not as simple as I thought - my
application still only detects EOF if I do more than recv(1),
but I cannot demonstrate simply - anyway, I have modified your
code to look more like my program's code, and it follows below
- if it does nothing else, it should convince you that recv()
*does* time out...

Yes.

The receiver code below detects if I close the sending console window,
as well as control-c - Rats!

Same here.

--
Grant Edwards grante Yow! Am I having fun yet?
at
visi.com