473,785 Members | 2,289 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

of destructors, open files and garbage collection

Hi,

Python 2.4, Kubuntu 6.06. I'm no professional programmer (I am a ph.d.
student in biophysics) but I have a fair knowledge of Python.

I have a for loop that looks like the following :

for item in long_list:
foo(item)

def foo(item):
item.create_bla h() #<--this creates item.blah; by doing that it
opens a file and leaves it open until blah.__del__() is called

Now, what I thought is that if I call

del(item)

it will delete item and also all objects created inside item. So I
thought that item.blah.__del __() would have been called and files
closed.
Question 1:
This is not the case. I have to call del(item.blah), otherwise files
are kept open and the for loops end with a "Too many open files"
error. Why isn't __del__() called on objects belonging to a parent
object? Is it OK?

So I thought:
oh, ok, let's put del(self.blah) in item.__del__()
Question 2:
This doesn't work either. Why?

Thanks a lot,
M.

May 24 '07 #1
6 1777
On 24 May, 16:40, "massimo s." <deviceran...@g mail.comwrote:
Now, what I thought is that if I call

del(item)

it will delete item and also all objects created inside item.
Sort of, but it's a bit more subtle. You'll stop the name "item" from
referring to your item - if nothing else refers to your item, it will
be garbage collected (and __del__ will get called). But you can have
other references, and in this case, __del__ is not called until *they*
are released as well.

Here's an example:
>>class C:
.... def __del__(self):
.... print "del called"
....
>>c = C()
# Now we have one reference to the object, in c. So delete it:
....
>>del c
del called
>># Just as we want.
.... # Let's create a new C, but store a second reference to it in "a".
....
>>c = C()
a = c
# Now we can delete c, but a still refers to the object, so it isn't collected
....
>>del c
# But if we delete a, it is!
....
>>del a
del called
>>>
OK. Now in your case, it's a bit more complex. You delete item.
Suppose that causes the item to be garbage collected (there are no
other references). Then, the item will be collected. This removes the
attribute item.blah, which refers to the blah object. So the blah
object is collected - *as long as no other references exist to that
item*. Here's another example:
>>class B:
.... def __init__(self):
.... self.c = C()
.... def __del__(self):
.... print "B's delete called"
....
>>b = B()
del b
B's delete called
del called
>># But if we have a second reference to b.c, that causes the object to stay alive:
....
>>b = B()
a = b.c
del b
B's delete called
>>del a
del called
>>>
See? Even though b was collected, its c attribute is still accessible
under the name 'a', so it's kept alive.
Question 1:
This is not the case. I have to call del(item.blah), otherwise files
are kept open and the for loops end with a "Too many open files"
error. Why isn't __del__() called on objects belonging to a parent
object? Is it OK?
Did the above help to clarify?
So I thought:
oh, ok, let's put del(self.blah) in item.__del__()
Question 2:
This doesn't work either. Why?
It's not needed - it's not the item.blah reference that's keeping the
blah object alive, it's another one.

You *can* fix this by tracking down all the references and explicitly
deleting them one by one, but that's not really the best answer.
You're micromanaging stuff the garbage collector is supposed to handle
for you. Ultimately, you've got a design problem, as you're holding
onto stuff you no longer need. Whether you use del, or add an explicit
blah.close() method to close the filehandle, you've got to understand
when you're finished with a filehandle - if you know that, you can
close it at that point.

Here's a final example that may help:
>>a = []
for i in range(10):
.... a.append(C())
....
>># Lots of work, none of which uses a
....
>>a = [] # or del a
del called
del called
del called
del called
del called
del called
del called
del called
del called
del called

See how you finished with all of the C objects right after the for
loop, but they didn't get deleted until later? I suspect that's what's
happening to you. If you cleared out the list (my a = [] statement) as
soon as you're done with it, you get the resources back that much
sooner.

Hope this helps,
Paul.

May 24 '07 #2
In <11************ *********@q66g2 000hsg.googlegr oups.com>, massimo s.
wrote:
I have a for loop that looks like the following :

for item in long_list:
foo(item)

def foo(item):
item.create_bla h() #<--this creates item.blah; by doing that it
opens a file and leaves it open until blah.__del__() is called

Now, what I thought is that if I call

del(item)

it will delete item and also all objects created inside item.
It will delete the *name* `item`. It does nothing to the object that was
bound to that name. If the name was the only reference to that object, it
may be garbage collected sooner or later. Read the documentation for the
`__del__()` method for more details and why implementing such a method
increases the chance that the object *won't* be garbage collected!

Relying on the `__del__()` method isn't a good idea because there are no
really hard guaranties by the language if and when it will be called.

Ciao,
Marc 'BlackJack' Rintsch
May 24 '07 #3
It will delete the *name* `item`. It does nothing to the object that was
bound to that name. If the name was the only reference to that object, it
may be garbage collected sooner or later. Read the documentation for the
`__del__()` method for more details and why implementing such a method
increases the chance that the object *won't* be garbage collected!

Relying on the `__del__()` method isn't a good idea because there are no
really hard guaranties by the language if and when it will be called.
Ok, I gave a look at the docs and, in fact, relying on __del__ doesn't
look like a good idea.

Changing the code as to add an explicit method that closes dangling
filehandles is easy. It would be somehow nice because -since that
method would be added to a plugin API- it *forces* people writing
plugins to ensure a way to close their dangling files, and this may be
useful for a lot of future purposes. However I'd also like to track
references to my objects -this would help debugging a lot. How can I
do that?

May 24 '07 #4
Relying on the `__del__()` method isn't a good idea because there are no
really hard guaranties by the language if and when it will be called.
Ok, I read the __del__() docs and I understand using it is not a good
idea.

I can easily add a close_files() method that forces all dangling files
to be closed. It would be useful in a number of other possible
situations. However, as rightly pointed out by the exhaustive answer
of Paul Moore, tracking references of my objects would be very useful.
How can I do that?

May 24 '07 #5

"massimo s." <de**********@g mail.comwrote in message
news:11******** *************@q 66g2000hsg.goog legroups.com...
| Hi,
|
| Python 2.4, Kubuntu 6.06. I'm no professional programmer (I am a ph.d.
| student in biophysics) but I have a fair knowledge of Python.
|
| I have a for loop that looks like the following :
|
| for item in long_list:
| foo(item)
|
| def foo(item):
| item.create_bla h() #<--this creates item.blah; by doing that it
| opens a file and leaves it open until blah.__del__() is called
|
| Now, what I thought is that if I call
|
| del(item)
|
| it will delete item

No, it removes the association between the name 'item' and the object it is
currently bound to. In CPython, removing the last such reference will
cause the object to be gc'ed. In other implementations , actual deletion
may occur later. You probably should close the files directly and arrange
code so that you can do so before too many are open.

tjr

May 24 '07 #6
No, it removes the association between the name 'item' and the object it is
currently bound to. In CPython, removing the last such reference will
cause the object to be gc'ed. In other implementations , actual deletion
may occur later. You probably should close the files directly and arrange
code so that you can do so before too many are open.
Thanks a lot, I'll follow that way.

m.

May 26 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
1644
by: David Turner | last post by:
Further to the discussion from earlier titled "does python have useless destructors?", I've posted a brief summary of garbage collection, RAII, and the "using" clause (which is what PEP310's "with" clause amounts to). The post is here: http://dkturner.blogspot.com/2004/06/garbage-collection-raii-and-using.html Regards David Turner
6
2185
by: Elbert Lev | last post by:
Please correct me if I'm wrong. Python (as I understand) uses reference counting to determine when to delete the object. As soon as the object goes out of the scope it is deleted. Python does not use garbage collection (as Java does). So if the script runs a loop: for i in range(100): f = Obj(i)
6
1386
by: Mark Broadbent | last post by:
Everything I have read suggests that if I open files or database connections or the like, that I should explicitly close those said resources before making the object subject to GC. This is obviously good programming technique. However what happens if this practice is not followed. e.g. that I have a base object that creates an instance of a filestream and for arguements sake an instance of a oledb connection. Both sub objects are...
8
8551
by: LP | last post by:
Hello! I am moving from VB.NET to C#. Could someone explain to me what is the purpose of destructor in C# class if GC destroys the object on its own timing. What's the difference between destructor such as ~className and IDisposable.Dispose and finalize method. At what point destructor gets called? Doest it get invoked by GC or by the client. I thought finalize method gets called by GC right before it destroys the object, or is it a...
3
2100
by: Jimmy | last post by:
Hi when do you use destructors and what are the advantages /disadvantages? ch Jim
26
2726
by: Michi Henning | last post by:
I've been having problem with destructors in the context of having ported C# code developed under .NET to Mono. What happens is that, on a dual-CPU machine, various parts of the code crash randomly (and rarely). This always happens during process shutdown, after some thread has called System.Environment.Exit(). Clearly, some sort of race condition. Note that what follows only applies to destructors that are called when the process shuts...
9
1335
by: Peter Oliphant | last post by:
I would assume in the following that 'instance' being set to 'nullptr' should cause the instance of myClass that was created to no longer have any pointers pointing to it, and therefore be 'destroyed': ref myClass { public: myClass() {} ~myClass() {} // never called !myClass() {} // never called
2
1056
by: Paul M | last post by:
Hi folks, Given the nature of asp, when I'm constructing data bound middler tier objects, is it necessary to implement the Idisposable interface or is it good enough to rely on a Finalize call. Most of my applications will be using Forms authentication and as such the sql server connection objects will all have individual user id's i.e. no connection pooling, and because of the time between posts, I can't see any reason not rely on...
6
7531
by: mlw | last post by:
Could someone explain why there is no destructor in Java classes? There are many times you need to be called WHEN an object goes out of scope and not when it will eventally be freed.
0
9647
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10357
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9959
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8988
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7509
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6744
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5396
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5528
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4063
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.