By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,814 Members | 1,111 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,814 IT Pros & Developers. It's quick & easy.

Python threads and memory usage

P: n/a
Hi,

I'm writing client-server application in Python. It's monitoring
system, where server listen and waits for TCP connections, and every
connection takes own thread. Every thread puts data from clients to
Queue and exits. Then there is one DB loader thread, which loads all
data from Queue to MySQL DB.

I observed, that every thread reserved some memory, and after exit
thread doesn't freed it. When i leaved my server working for 3 days,
then it takes 15% of 512MB memory (during that time about 15000
threads were created and stopped). When server starts it only takes
about 1% of memory.

I know that I can made client which connects once and keep this
session with server thread all time (it should resolve my problems),
but I'm curious if this is normal or it is something like memory leak.
Or maybe I'm doing something wrong :)

If You need I can paste all my code.

Regards
Jun 27 '08 #1
Share this Question
Share on Google+
2 Replies


P: n/a
On May 30, 9:16*am, Dennis Lee Bieber <wlfr...@ix.netcom.comwrote:
On Thu, 29 May 2008 12:01:30 -0700 (PDT), Mike <mszpad...@gmail.com>
declaimed the following in comp.lang.python:
I observed, that every thread reserved some memory, and after exit
thread doesn't freed it. When i leaved my server working for 3 days,
then it takes 15% of 512MB memory (during that time about 15000
threads were created and stopped). When server starts it only takes
about 1% of memory.

* * * * Do you have any outstanding references to the threads? If so, have
you performed a .join() with the thread? Until you join it, the thread
state (thread local objects/"variables") are probably being held for
access from outside the thread.
--
* * * * Wulfraed * * * *Dennis Lee Bieber * * * * * * * KD6MOG
* * * * wlfr...@ix.netcom.com * * * * * * wulfr...@bestiaria.com
* * * * * * * * HTTP://wlfraed.home.netcom.com/
* * * * (Bestiaria Support Staff: * * * * * * * web-a...@bestiaria.com)
* * * * * * * * HTTP://www.bestiaria.com/
I'm joining threads only during my program exit. I'll try to do what
You suggest.

THX
Jun 27 '08 #2

P: n/a
On May 30, 9:42*am, Mike <mszpad...@gmail.comwrote:
On May 30, 9:16*am, Dennis Lee Bieber <wlfr...@ix.netcom.comwrote:
On Thu, 29 May 2008 12:01:30 -0700 (PDT), Mike <mszpad...@gmail.com>
declaimed the following in comp.lang.python:
I observed, that every thread reserved some memory, and after exit
thread doesn't freed it. When i leaved my server working for 3 days,
then it takes 15% of 512MB memory (during that time about 15000
threads were created and stopped). When server starts it only takes
about 1% of memory.
* * * * Do you have any outstanding references to the threads? If so, have
you performed a .join() with the thread? Until you join it, the thread
state (thread local objects/"variables") are probably being held for
access from outside the thread.
--
* * * * Wulfraed * * * *Dennis Lee Bieber * * * * * * * KD6MOG
* * * * wlfr...@ix.netcom.com * * * * * * wulfr...@bestiaria.com
* * * * * * * * HTTP://wlfraed.home.netcom.com/
* * * * (Bestiaria Support Staff: * * * * * * * web-a...@bestiaria.com)
* * * * * * * * HTTP://www.bestiaria.com/

I'm joining threads only during my program exit. I'll try to do what
You suggest.

THX
It helped. Now all threads are added to thread list and every some
period of time I'm checking which threads are alive (enumerate), and
joining all which aren't. Now [memory usage is still on 1% :D:D:D

Thanks
Jun 27 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.