reference counting and file objects

[John Reese]

def uselessHash(filename):
fp= open(filename)
hash= 0
for line in fp:
hash ^= hash(line.strip())
fp.close() # do I need this or is fp closed by ref count?
return hash

Consider the function above. Do I need the fp.close(), or will the
file be closed automatically when fp goes out of scope and its
reference count drops to zero?

If so, there's really no point in calling .close() in situations like
this where the function completes in a relatively short time,, and if
not, I should probably be using try-finally. That's my thinking... or
the thinking of the coworker who was arguing with me.

[Paul Rubin]

John Reese <jt*@ofb.net> writes:

Consider the function above. Do I need the fp.close(), or will the
file be closed automatically when fp goes out of scope and its
reference count drops to zero?

In CPython, fp gets closed when it leaves scope. In other
implementations you may need try-finally. There's been occasional
discussion of possible new Python control blocks to make this easier.

[Martin v. Löwis]

Paul Rubin wrote:

Consider the function above. Do I need the fp.close(), or will the
file be closed automatically when fp goes out of scope and its
reference count drops to zero?

In CPython, fp gets closed when it leaves scope.

One issue is that when the function terminates through an exception,
the file stays open as long as the traceback is available, which
exists until the next exception is raised (and thus replaces this
traceback).

If the function terminates normally (i.e. through return), it is
as you say.

Regards,
Martin

[Peter]

Martin v. Löwis wrote:

Paul Rubin wrote:

Consider the function above. Do I need the fp.close(), or will the
file be closed automatically when fp goes out of scope and its
reference count drops to zero?

In CPython, fp gets closed when it leaves scope.

One issue is that when the function terminates through an exception,
the file stays open as long as the traceback is available, which
exists until the next exception is raised (and thus replaces this
traceback).

If the function terminates normally (i.e. through return), it is
as you say.

Regards,
Martin

Does the idiom:

lines = file("myfile","r").readlines()

have any better guarantee of being closed automatically?

Peter

[Martin v. Löwis]

Peter wrote:

Does the idiom:

lines = file("myfile","r").readlines()

have any better guarantee of being closed automatically?

Yes. The file object only lives on the evaluation stack,
and that is discarded in any case when the function terminates
(whether through a return or through an exception). In
addition, the object is released as soon as readlines
returns.

Regards,
Martin

[Paul Rubin]

"Martin v. Löwis" <ma****@v.loewis.de> writes:

lines = file("myfile","r").readlines()

have any better guarantee of being closed automatically?

Yes. The file object only lives on the evaluation stack,
and that is discarded in any case when the function terminates
(whether through a return or through an exception). In
addition, the object is released as soon as readlines
returns.

It's released even if the exception is raised inside readlines?

[Martin v. Löwis]

Paul Rubin wrote:

lines = file("myfile","r").readlines()
It's released even if the exception is raised inside readlines?

I think so, but only because readlines is a builtin function.
If it wasn't, there would be a stack frame for readlines, which
would have "self" as a local variable.

As readlines is builtin, it will not make it into the traceback.
So the reference to the file would be only on the evaluation
stack, at first, but that then gets copied into the argument
tuple. The argument tuple, in turn, is released in the process
of unwinding (again, it wouldn't if readlines wasn't builtin).

I might be wrong, but I think the following trace shows that
this is what happens:

class F: .... def __del__(self):print "released"
.... len(F()) released
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: F instance has no attribute '__len__'

Compare this to a Python function:
def len2(o): raise Exception .... len2(F()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in len2
Exception raise ""

released
Traceback (most recent call last):
File "<stdin>", line 1, in ?

However, this analysis also shows that it is quite delicate
to rely on this pattern.

Regards,
Martin