473,231 Members | 2,013 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,231 software developers and data experts.

Memory leak when creating lots of object

Hello,

I have a program that create and pop an object off a queue, but it is
experiencing some memory leakage. I have been unable to detect where
the memory leakage occur. The strange thing is when i replace the
object creation with a plain integer/string, the leak goes away...
Here's the code I used as my test:
import Queue
import threading
import time

q = Queue.Queue(0)

class PDU:
def __init__(self, name=''):
self.name = name

class qTest(threading.Thread):
def __init__(self, name=''):
threading.Thread.__init__(self)
self.name = name
self.threadContinue = True

def run(self):
count = 0
while self.threadContinue:
pdu = q.get(True)
del pdu
count+=1
#print count
if q.qsize() == 0:
print "Total get in bulk", count
count = 0

def stop(self):
self.threadContinue = False
q.put(1)
if __name__ == "__main__":
try:
qt = qTest()
qt.start()
loopCount = 1000
while True:
for i in range(loopCount):
pdu = PDU()
## pdu = 1
q.put(pdu)
time.sleep(5)

except KeyboardInterrupt:
print "Keyboard interrupted. Program exit."
except Exception, why:
print why

if qt.isAlive():
qt.stop()

I can see the memory usage increases slowly in Task Manager under XP,
but do not know why. Anyone help?

Thank you.

Aug 14 '07 #1
3 2543
On 14 Aug, 05:57, Godzilla <godzillais...@gmail.comwrote:
Hello,

I have a program that create and pop an object off a queue, but it is
experiencing some memory leakage. I have been unable to detect where
the memory leakage occur. The strange thing is when i replace the
object creation with a plain integer/string, the leak goes away...
Here's the code I used as my test:
[...]
I can see the memory usage increases slowly in Task Manager under XP,
but do not know why. Anyone help?
I tried your code on my (Windows XP SP2, Python 2.5) system. No memory
leak here - I left it running for over 5 minutes and memory usage was
static at just under 4MB.

Do you see memory growth with precisely this code? Over what period?
How much?

Paul.

Aug 15 '07 #2

"Godzilla" <go***********@gmail.comwrote in message
news:11**********************@i38g2000prf.googlegr oups.com...
| What should I do next? Can I force garbage collection manually? If so,
| how do I do that?

Read doc for gc module. I think
import gc
gc.collect()

Aug 17 '07 #3
On 17 Aug, 04:51, Godzilla <godzillais...@gmail.comwrote:
On Aug 16, 1:13 am, Paul Moore<p.f.mo...@gmail.comwrote:
On 14 Aug, 05:57, Godzilla <godzillais...@gmail.comwrote:
Do you see memory growth with precisely this code? Over what period?
How much?

I have it running for more than 1 hour... the main application
software runs for about 50 days non stops and the memory just keep
growing...
I'm sorry. Just to be precise, how long would I need to run the code
you posted to see a memory growth?
What should I do next? Can I force garbage collection manually? If so,
how do I do that?
As Terry suggested, look at the gc module (gc.collect). But on
inspection, I don't see a memory leak that would be cured by forcing a
collection, which is why I'd like to reproduce the problem before
offering suggestions...

Paul.

Aug 18 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: Debian User | last post by:
Hi, I'm trying to discover a memory leak on a program of mine. I've taken several approaches, but the leak still resists to appear. First of all, I've tried to use the garbage collector to...
10
by: Matt Kruse | last post by:
I'm aware of the circular reference memory leak problem with IE/closures. I'm not sure exactly how to resolve it in this situation. Also, Firefox appears to grow its memory size with the same code....
7
by: Jon Davis | last post by:
OK I have a web app that I built that makes MANY calls to the DB in each request. The app wasn't tuned for scalability so this wasn't a problem, but time is too short to redesign how the database...
8
by: Adrian | last post by:
Hi I have a JS program that runs localy (under IE6 only) on a PC but it has a memory leak (probably the known MS one!) What applications are there that I could use to look at the memory usage of...
7
by: Salvador | last post by:
Hi, I am using WMI to gather information about different computers (using win2K and win 2K3), checking common classes and also WMI load balance. My application runs every 1 minute and reports...
2
by: Robert | last post by:
Hello javascript group readers, I have a question regarding how to prevent memory leaks in Internet Explorer when using closures. I already knew about the circular reference problem, and until...
0
by: nejucomo | last post by:
Hi folks, Quick Synopsis: A test script demonstrates a memory leak when I use pythonic extensions of my builtin types, but if I use the builtin types themselves there is no memory leak. ...
11
by: Alex | last post by:
Hello all, I have a main form(say "form1") .i want to display another form(say "form2") on occuring of an event (say a button click) and want to hide it after some time so that it will again...
22
by: Simon | last post by:
Hi all, I have a huge memory leak problem in what is really very simple data insert code. In my app I'm trying to use a typed dataset to insert into a database. It adds quite a few rows (say...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.