By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,939 Members | 1,602 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,939 IT Pros & Developers. It's quick & easy.

Socket being garbage collected too early

P: n/a
I have been having trouble with the garbage collector and sockets.
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is
x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)
def do_stuff(sock):
sock_list.append(sock)
once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected. I'd like to hear the standard means for
avoiding this issue (gc appears to have only the interface to declare
something garbage, not to declare something not garbage).

Scott Robinson

Jul 18 '05 #1
Share this Question
Share on Google+
4 Replies


P: n/a
Scott Robinson wrote:
I have been having trouble with the garbage collector and sockets.
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is
x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)
def do_stuff(sock):
sock_list.append(sock)
once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected. I'd like to hear the standard means for
avoiding this issue (gc appears to have only the interface to declare
something garbage, not to declare something not garbage).

Scott Robinson

Unfortunately, assuming it's being garbage collected might turn out to
be incorrect. What evidence do you have that the socket "disappears"? Do
you get a segmentation ault, or what?

If the socket simply fails to work that would be a different case
altogether, but it seems to me that we need a bit more evodence that the
anecdotal stuff you've provided so far.

Quite apart from anything else, by the way, the code you posted appears
to use a global sock_list. A reference b y that list would in any case
stop the socket from being garbage collected (quite apart from the fact
that the socket module itself will do so as long as the socket is open).

So, could we see an error message, or some other evidence of what is
going on. For example, after the call to do_stuff(), what do you see if you

print sock_list

for example. I think your initial hypothesis is insufficient.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119
Jul 18 '05 #2

P: n/a
Scott Robinson <ds*****@bellatlantic.net> writes:
I have been having trouble with the garbage collector and sockets.
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is
x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)
def do_stuff(sock):
sock_list.append(sock)
once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected. I'd like to hear the standard means for
avoiding this issue (gc appears to have only the interface to declare
something garbage, not to declare something not garbage).


The code as shown doesn't work:
import socket
def do_stuff(sock): .. sock_list.append(sock)
.. sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
do_stuff(sock) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in do_stuff
NameError: global name 'sock_list' is not defined


If you add "sock_list = []" just before the def of do_stuff, the code
will work, and your sockets won't get garbage collected.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Jul 18 '05 #3

P: n/a
Scott Robinson <ds*****@bellatlantic.net> writes:
I have been having trouble with the garbage collector and sockets.
Are you actually getting errors or is this just theoretical?
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is
x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)
def do_stuff(sock):
sock_list.append(sock)

once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected.


Can you clarify this? What do you mean by "x.sock" disappears? Are
you getting a NameError later when trying to use "x.sock"?

x.sock is just a name binding, so it is not really involved in garbage
collection (GC applies to the objects to which names are bound).

In this case, you need to include much more in the way of code (a
fully running, but smallest possible, snippet of code would be best),
since the above can be interpreted many ways. At the least, it's very
important to include information about the namespace within which
those two code snippets run if anyone is likely to be able to give you
a good answer. Also, being very precise about the error condition you
are experiencing (including actual error messages, tracebacks, etc...)
is crucial.

Is 'x' referencing a local or global object, and does that socket code
occur within a method, a function, or what? Also, in do_stuff, where
is sock_list defined? Is it local, global?

If, as written, sock_list is a local name to do_stuff, then that
binding is going to disappear when do_stuff completes, thus, the list
to which it is bound will be destroyed, including all references to
objects that the list may contain. So at that point, when you return
from do_stuff, the only reference to the socket object will be in
x.sock. But if 'x' is also local to the function/method where the
call to do_stuff is, the name binding will be removed when the
function/method returns, at which point there will be no references to
the socket object, and yes, it will be destroyed.

But if sock_list is global, and continues to exist when do_stuff
completes, then the reference it contains to the socket will keep the
socket object alive even if you remove the x.sock binding.

-- David

Jul 18 '05 #4

P: n/a
On 16 Dec 2004 20:38:29 -0500, David Bolen <db**@fitlinxx.com> wrote:
Scott Robinson <ds*****@bellatlantic.net> writes:
I have been having trouble with the garbage collector and sockets.


Are you actually getting errors or is this just theoretical?
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is
x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)
def do_stuff(sock):
sock_list.append(sock)

once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected.


Can you clarify this? What do you mean by "x.sock" disappears? Are
you getting a NameError later when trying to use "x.sock"?

x.sock is just a name binding, so it is not really involved in garbage
collection (GC applies to the objects to which names are bound).

In this case, you need to include much more in the way of code (a
fully running, but smallest possible, snippet of code would be best),
since the above can be interpreted many ways. At the least, it's very
important to include information about the namespace within which
those two code snippets run if anyone is likely to be able to give you
a good answer. Also, being very precise about the error condition you
are experiencing (including actual error messages, tracebacks, etc...)
is crucial.

Is 'x' referencing a local or global object, and does that socket code
occur within a method, a function, or what? Also, in do_stuff, where
is sock_list defined? Is it local, global?

If, as written, sock_list is a local name to do_stuff, then that
binding is going to disappear when do_stuff completes, thus, the list
to which it is bound will be destroyed, including all references to
objects that the list may contain. So at that point, when you return
from do_stuff, the only reference to the socket object will be in
x.sock. But if 'x' is also local to the function/method where the
call to do_stuff is, the name binding will be removed when the
function/method returns, at which point there will be no references to
the socket object, and yes, it will be destroyed.

But if sock_list is global, and continues to exist when do_stuff
completes, then the reference it contains to the socket will keep the
socket object alive even if you remove the x.sock binding.

-- David

(so much for Python being executable psuedocode).

It looks like I was completely wrong. The problem I ran into was not
checking into baseHTTPserver and looking for self.close_connection=1.

I am pretty sure this happened to me before, and I rewrote the code to
avoid sockets being closed after being referenced to a live object,
but I can't reproduce it.

Anyway, here is my debugged test rig. It works. Ignore it.
Scott
[test program follows]
import socket, thread, time

class sock_holder:
pass

HOST = '127.0.0.1'
PORT = 2004

def echo_server(port):
print "started at",port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, port))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()

def wait_and_speak(a):
time.sleep(5)
try:
a.sock.send("this is message 2")
except:
print "error on message 2"
try:
data = a.sock.recv(1024)
print data
except:
print "error recieving message 2"
a.sock.close()

def other_thread(a):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
thread.start_new(echo_server,(PORT,))
time.sleep(1)
s.connect((HOST,PORT ))
s.send('Hello, port 1')
data = s.recv(1024)
print data
a.sock=s

thread.start_new(wait_and_speak,(a,))

a=sock_holder()
thread.start_new(other_thread,(a,))
time.sleep(10)
Jul 18 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.