473,748 Members | 2,328 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Subprocess freezes when piping from stdout.

Hi All,

I'm using subprocess.Pope n to run a C++ compiler and have set stdout =
PIPE. The exact line of code that I am using is:

process = Popen(command, stdout=PIPE)
status = process.wait()

This works fine until a large amount of data is written to stdout.
When this occurs, my python program seems to freeze. It will no longer
run or respond to Ctrl-C. I am assuming that it is stuck waiting for
the process to end, but I'm not sure why this should take so long (I
have left it running for hours and it never ended)

At the moment the code is being executed on a Win32 environment.

Is anyone aware of why this might be occurring, or of any ways around
this? Does the PIPE implementation in Win32 have a maximum buffer
size?

Regards,
James McGill
Sep 8 '08 #1
1 2691
James McGill wrote:
Is anyone aware of why this might be occurring, or of any ways around
this? Does the PIPE implementation in Win32 have a maximum buffer
size?
pipes always have a limited buffer size, on all platforms. you're
supposed to read data from them as it arrives (e.g. by explicitly
reading from the "stdout" attribute, or by calling "communicat e").

if you cannot do that, bind the channel to a file, not a pipe.

</F>

Sep 8 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
3399
by: Marc Carter | last post by:
I am trying to rewrite a PERL automation which started a "monitoring" application on many machines, via RSH, and then multiplexed their collective outputs to stdout. In production there are lots of these subprocesses but here is a simplified example what I have so far (python n00b alert!) - SNIP --------- import subprocess,select,sys speakers=
2
4085
by: Stewart Midwinter | last post by:
this has me puzzled; I've created a small test app to show the problem I'm having. I want to use subprocess to execute system commands from inside a Tkinter app running under Cygwin. When I open a python interpreter and run my subprocess command, all is well. But when I run the same command from inside a Tkinter app, I'm getting errors.
3
5506
by: Darren Dale | last post by:
I'm a developer on the matplotlib project, and I am having trouble with the subprocess module on windows (Python 2.4.2 on winXP). No trouble to report with linux. I need to use _subprocess instead of pywin32, but my trouble exists with either option: import subprocess process = subprocess.Popen(, stderr=subprocess.STDOUT, stdout=subprocess.PIPE) stat = process.wait() print process.stdout.read()
5
14328
by: Grant Edwards | last post by:
I'm trying to use the py-gnuplot module on windows, and have been unable to get it to work reliably under Win2K and WinXP. By default, it uses popen(gnuplotcmd,'w'), but in some situations that consistently gets an "invalid operand" IOError when write() is called on the pipe. So I switched to subprocess. It works fine when executed "normally" (e.g. "python progname.py"), but when bundled by py2exe, it always does this:
2
1851
by: smitty1e | last post by:
The first print statement does what you'd expect. The second print statement has rather a lot of rat in it. The goal here is to write a function that will return the man page for some command (mktemp used as a short example here) as text to client code, where the groff markup will be chopped to extract all of the command options. Those options will eventually be used within an emacs mode, all things going swimmingly. I don't know what's...
9
6495
by: Phoe6 | last post by:
Hi all, Consider this scenario, where in I need to use subprocess to execute a command like 'ping 127.0.0.1' which will have a continuous non- terminating output in Linux. # code # This hangs at this point. How should I handle these kind of commands (ping 127.0.0.1) with
1
2450
by: NeoGregorian | last post by:
Hello, I am writing a wrapper to a basic Input/Output programs (where you type a one line command at a time and then get 0 or more lines of output before you can input the next command). I'm sorry if this problem description is a bit long, but I wanted to make the problem clear. Example run of the original program: C:\home\>start Starting up program
4
17531
by: 7stud | last post by:
Hi, What is the difference between: 1) getting the returncode directly from the subprocess object 2) calling poll() on the subprocess object? Here is an example:
12
4532
by: bhunter | last post by:
Hi, I've used subprocess with 2.4 several times to execute a process, wait for it to finish, and then look at its output. Now I want to spawn the process separately, later check to see if it's finished, and if it is look at its output. I may want to send a signal at some point to kill the process. This seems straightforward, but it doesn't seem to be working. Here's my test case:
0
8983
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8822
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9528
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9359
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8235
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6792
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4592
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
2774
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2206
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.