Re: python has memory leak?

[Jean-Paul Calderone]

On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yz****@gmail.com wrote:

>Hi all,

I feel that my python script is leaking memory. And this is a test I
have:

[snip]

The test doesn't demonstrate any leaks. It does demonstrate that memory
usage can remain at or near peak memory usage even after the objects for
which that memory was allocated are no longer live in the process. This
is only a leak if peak memory goes up again each time you create any new
objects. Try repeated allocations of a large dictionary and observe how
memory usage rises and falls.

Python 2.5 does a somewhat better job of releasing memory when actual use
falls below peak, but this is a difficult thing to do perfectly.

Jean-Paul

[yzghan]

Thanks to Christian and Jean-Paul for your reply. I moved away from
the stackless version (which I don't understand what uniqueness it
comes with) and downloaded a fresh copy of Python 2.5.2 (r252:60911,
Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32. Following
J-P's suggestion, I included the block to be tested in a loop:

i=20
while i>0:
i=i-1
test(log)
log.flush()

And logged the memory usage in bytes instead of MB. The result as
included below shows the peak memory usage keeps going up in the first
7 iterations. Can someone point to some docs or books about how python
manages memory or share some experience?

Thanks,

-GH
[10:27:54] test() starting ... memory usage: 4599808L 4599808L
[10:27:56] test() ... memory usage: 107499520L 107499520L
[10:27:56] test() done. memory usage: 41754624L 107503616L
[10:27:56] test() starting ... memory usage: 41754624L 107503616L
[10:27:59] test() ... memory usage: 108216320L 108400640L
[10:27:59] test() done. memory usage: 58884096L 108400640L
[10:27:59] test() starting ... memory usage: 58884096L 108400640L
[10:28:02] test() ... memory usage: 108220416L 108433408L
[10:28:02] test() done. memory usage: 42233856L 108433408L
[10:28:02] test() starting ... memory usage: 42233856L 108433408L
[10:28:04] test() ... memory usage: 108228608L 108441600L
[10:28:05] test() done. memory usage: 75149312L 108441600L
[10:28:05] test() starting ... memory usage: 75149312L 108441600L
[10:28:07] test() ... memory usage: 108498944L 108707840L
[10:28:07] test() done. memory usage: 41992192L 108707840L
[10:28:07] test() starting ... memory usage: 41992192L 108707840L
[10:28:10] test() ... memory usage: 108122112L 108707840L
[10:28:10] test() done. memory usage: 58523648L 108707840L
[10:28:10] test() starting ... memory usage: 58523648L 108707840L
[10:28:13] test() ... memory usage: 108642304L 108855296L
[10:28:13] test() done. memory usage: 41873408L 108855296L
[10:28:13] test() starting ... memory usage: 41873408L 108855296L
[10:28:16] test() ... memory usage: 108224512L 108855296L
[10:28:16] test() done. memory usage: 58626048L 108855296L
[10:28:16] test() starting ... memory usage: 58626048L 108855296L
[10:28:18] test() ... memory usage: 108224512L 108855296L
[10:28:19] test() done. memory usage: 41975808L 108855296L
[10:28:19] test() starting ... memory usage: 41975808L 108855296L
[10:28:21] test() ... memory usage: 108224512L 108855296L
[10:28:22] test() done. memory usage: 58626048L 108855296L
[10:28:22] test() starting ... memory usage: 58626048L 108855296L
[10:28:24] test() ... memory usage: 108224512L 108855296L
[10:28:24] test() done. memory usage: 41975808L 108855296L
[10:28:24] test() starting ... memory usage: 41975808L 108855296L
[10:28:27] test() ... memory usage: 108224512L 108855296L
[10:28:27] test() done. memory usage: 58626048L 108855296L
[10:28:27] test() starting ... memory usage: 58626048L 108855296L
[10:28:30] test() ... memory usage: 108224512L 108855296L
[10:28:30] test() done. memory usage: 41975808L 108855296L
[10:28:30] test() starting ... memory usage: 41975808L 108855296L
[10:28:32] test() ... memory usage: 108224512L 108855296L
[10:28:33] test() done. memory usage: 58626048L 108855296L
[10:28:33] test() starting ... memory usage: 58626048L 108855296L
[10:28:35] test() ... memory usage: 108224512L 108855296L
[10:28:36] test() done. memory usage: 41975808L 108855296L
[10:28:36] test() starting ... memory usage: 41975808L 108855296L
[10:28:38] test() ... memory usage: 108224512L 108855296L
[10:28:38] test() done. memory usage: 58626048L 108855296L
[10:28:38] test() starting ... memory usage: 58626048L 108855296L
[10:28:41] test() ... memory usage: 108224512L 108855296L
[10:28:41] test() done. memory usage: 41975808L 108855296L
[10:28:41] test() starting ... memory usage: 41975808L 108855296L
[10:28:44] test() ... memory usage: 108224512L 108855296L
[10:28:44] test() done. memory usage: 58626048L 108855296L
[10:28:44] test() starting ... memory usage: 58626048L 108855296L
[10:28:46] test() ... memory usage: 108224512L 108855296L
[10:28:47] test() done. memory usage: 41975808L 108855296L
[10:28:47] test() starting ... memory usage: 41975808L 108855296L
[10:28:49] test() ... memory usage: 108224512L 108855296L
[10:28:50] test() done. memory usage: 58626048L 108855296L