473,405 Members | 2,282 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,405 software developers and data experts.

Leaks in subprocess.Popen

I'm using Python 2.4.3 for Win32.
I was trying to run a few child processes simultaneously in separate
threads and get their STDOUT, but the program was leaking memory and I
found that it was because of subprocess operating in another thread.
The following code works fine, but I get a leaking handle every second.
You can see it in the task manager if you choose to see the <handle
countcolumn. Does anybody have a solution? Please help!

import subprocess, time, thread

def comm(command):
run = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = run.communicate()
print out

while 1:
thread.start_new_thread(comm, ("dir",))
time.sleep(1)

Sep 20 '06 #1
1 3939
zloster wrote:
I'm using Python 2.4.3 for Win32.
I was trying to run a few child processes simultaneously in separate
threads and get their STDOUT, but the program was leaking memory and I
found that it was because of subprocess operating in another thread.
The following code works fine, but I get a leaking handle every second.
You can see it in the task manager if you choose to see the <handle
countcolumn. Does anybody have a solution? Please help!
This bug is fixed in the 2.5 version of Python and will be fixed in the
next 2.4 maintainance release (2.4.4). See:
http://www.python.org/sf/1500293 for the bug report.

You can find the relevant changes here:
http://mail.python.org/pipermail/pyt...ne/053417.html
http://mail.python.org/pipermail/pyt...ne/053418.html

If you have Python Win32 Extensions installed you can try using that
instead of the extension in the standard library; you have to change a
single line in the subprocess module:

--- subprocess_modified.py 2006-09-20 22:04:29.734375000 +0200
+++ subprocess.py 2006-09-20 22:01:52.296875000 +0200
@@ -350,7 +350,7 @@
if mswindows:
import threading
import msvcrt
- if 0: # <-- change this to use pywin32 instead of the _subprocess
driver
+ if 1: # <-- change this to use pywin32 instead of the _subprocess
driver
import pywintypes
from win32api import GetStdHandle, STD_INPUT_HANDLE, \
STD_OUTPUT_HANDLE, STD_ERROR_HANDLE

Hope this helps,
Ziga

Sep 20 '06 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Darren Dale | last post by:
I'm a developer on the matplotlib project, and I am having trouble with the subprocess module on windows (Python 2.4.2 on winXP). No trouble to report with linux. I need to use _subprocess instead...
3
by: madpython | last post by:
playing with subprocess.Popen on Windows I stumbled into the following problem: Python 2.4.3 (#69, Mar 29 2006, 17:35:34) IDLE 1.1.3 >>> import subprocess >>>...
12
by: Eric_Dexter | last post by:
I am trying to modify a programming example and I am coming up with two problems... first is that I can't seem to pass along the arguments to the external command (I have been able to do that with...
9
by: Phoe6 | last post by:
Hi all, Consider this scenario, where in I need to use subprocess to execute a command like 'ping 127.0.0.1' which will have a continuous non- terminating output in Linux. # code # This...
12
by: bhunter | last post by:
Hi, I've used subprocess with 2.4 several times to execute a process, wait for it to finish, and then look at its output. Now I want to spawn the process separately, later check to see if it's...
7
by: skunkwerk | last post by:
Hi, i'm trying to call subprocess.popen on the 'rename' function in linux. When I run the command from the shell, like so: rename -vn 's/\.htm$/\.html/' *.htm it works fine... however when I...
25
by: Jeremy Banks | last post by:
Hi. I wondered if anyone knew the rationale behind the naming of the Popen class in the subprocess module. Popen sounds like the a suitable name for a function that created a subprocess, but the...
1
by: Mark Shewfelt | last post by:
Hello, I am attempting to use Popen() in a Windows service. I have a small Win32 .exe that I normally run through the os.popen2() function. I've written a class to work with the input and output...
5
by: thedsadude | last post by:
Hello, I'm launching a script as follows: <code> p = subprocess.Popen() p.wait() </code> If p.py writes to sys.stdout, then it is shown on the console.
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.