Strange socket problem

[huy]

Hi,

I'm using cherrypy to provide a user interface for users to start a
linux server program eg. os.system("nohu p myserver.py &"). The problem
is that if I stop cherrypy server and restart, I get the "Address
Already In Use" problem until I stop myserver.py. Can someone shed some
light on why this happens ? Why would the socket be held up in the other
process ?

Thanks

Huy

[Jeff Epler]

When using os.system(), files that are open in the parent are available
in the child, as you can see here in Linux' listing of the files open by
the child program:

[jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
3
total 5
lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd

You may be able to set the FD_CLOEXEC flag on the files that should not
be passed to children, something like this:
old = fcntl.fcntl(fd, fcntl.F_GETFD)
fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXE C)
Refer to a real Unix reference for more information on what FD_CLOEXEC
does.

You may want to replace the use of os.system() with fork + close files
+ exec. Then "myserver.p y" will no longer have the listening socket
descriptor of your cherrypy server.

Python 2.4's 'subprocess' module may work better in this respect than
os.system. It seems to include support for requesting that fds be
closed in the child (the close_fds parameter to subprocess.Pope n)

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFCsB/gJd01MZaTXX0RAr dnAJwN9iek350cq XtYDdUzzSomz6dQ IgCdFaC5
Ebd2XyIMsS+QgoL QB5jyPMU=
=9XcR
-----END PGP SIGNATURE-----

[Magnus Lycka]

huy wrote:

Hi,

I'm using cherrypy to provide a user interface for users to start a
linux server program eg. os.system("nohu p myserver.py &"). The problem
is that if I stop cherrypy server and restart, I get the "Address
Already In Use" problem until I stop myserver.py. Can someone shed some
light on why this happens ? Why would the socket be held up in the other
process ?
As I understand it, if a socket is closed by the server, the OS
will typically hold on to the socket for a while, in case the
client wants some packages resent etc. This might be a part of
your problem.

Andrew Dalke explained it much better to me when I ran into this
some time ago. Quoted below:
Magnus Lyckå wrote:

Why doesn't my socket
get released by the OS when I exit via my handle_error?

Hi Magnus,

I wrote about this at
http://www.dalkescientific.com/writi...ng_xmlrpc.html

The reason for it is described at
http://hea-www.harvard.edu/~fine/Tech/addrinuse.html

You can set the class variable "allow_reuse_ad dress = True" in
your derived ExitableSocketS erver to get the behaviour you
expect, at the expense of some problems mentioned in the
above URL.

Andrew
da***@dalkescie ntific.com

[nm674674]

Gurus,

I am still doing my baby steps in the wonderful world of python (so
far, so good).
However, I am quite familiar with sockets. There is a socket option
called
SO_REUSEADDR that your server should call to fix this problem.

[huy]

Hi Jeff,

Thanks for your help. Although I haven't confirmed this, I think you
just hit my nail on the head. I thought os.system was like a totally
separate process though, i.e nothing is shared. not the usual fork()
call within the program.

Regards,

Huy

Jeff Epler wrote:

When using os.system(), files that are open in the parent are available
in the child, as you can see here in Linux' listing of the files open by
the child program:

[jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
3
total 5
lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd

You may be able to set the FD_CLOEXEC flag on the files that should not
be passed to children, something like this:
old = fcntl.fcntl(fd, fcntl.F_GETFD)
fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXE C)
Refer to a real Unix reference for more information on what FD_CLOEXEC
does.

You may want to replace the use of os.system() with fork + close files
+ exec. Then "myserver.p y" will no longer have the listening socket
descriptor of your cherrypy server.

Python 2.4's 'subprocess' module may work better in this respect than
os.system. It seems to include support for requesting that fds be
closed in the child (the close_fds parameter to subprocess.Pope n)

Jeff

[huy]

Hi Jeff,

Thanks for your help. Although I haven't confirmed this, I think you
just hit my nail on the head. I thought os.system was like a totally
separate process though, i.e nothing is shared. not the usual fork()
call within the program.

Regards,

Huy

Jeff Epler wrote:

When using os.system(), files that are open in the parent are available
in the child, as you can see here in Linux' listing of the files open by
the child program:

[jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
3
total 5
lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd

You may be able to set the FD_CLOEXEC flag on the files that should not
be passed to children, something like this:
old = fcntl.fcntl(fd, fcntl.F_GETFD)
fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXE C)
Refer to a real Unix reference for more information on what FD_CLOEXEC
does.

You may want to replace the use of os.system() with fork + close files
+ exec. Then "myserver.p y" will no longer have the listening socket
descriptor of your cherrypy server.

Python 2.4's 'subprocess' module may work better in this respect than
os.system. It seems to include support for requesting that fds be
closed in the child (the close_fds parameter to subprocess.Pope n)

Jeff