473,799 Members | 3,114 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Why does producer delay halt shell pipe?

I have 2 python scripts: examples of a producer and a filter,
respectively:

#! /usr/bin/env python
import sys, time
if __name__ == "__main__":
while True:
sys.stdout.writ e("hello.\r\n ")
time.sleep(0.00 0001)

#! /usr/bin/env python
import sys
if __name__ == "__main__":
line = sys.stdin.readl ine()
while line:
sys.stdout.writ e(line.upper())
line = sys.stdin.readl ine()

I wish to use these programs in Bash, like so:

$ ./producer.py | ./filter.py

However, the producer's time delay makes this not work. If I remove
or reduce the delay, it works. In reality the producer has an
unavoidable one-second delay. I do NOT want to use popen or its
cousins because I want flexibility from the command line; I have many
filters. Is there any way to write the filter to make this work?

thanks,

!!Dean
Dec 11 '07 #1
2 1451
On Dec 11, 1:05 pm, dwhall <dwhall...@gmai l.comwrote:
filters. Is there any way to write the filter to make this work?

thanks,

!!Dean
turn off python buffering & it should work.
export PYTHONUNBUFFERE D=t

n'joy
-N
Dec 11 '07 #2
Thanks, N, it works like a charm.

!!Dean

On Dec 11, 12:49 pm, Nanjundi <nanju...@gmail .comwrote:
turn off python buffering & it should work.
export PYTHONUNBUFFERE D=t
Dec 12 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
2373
by: Evan Simpson | last post by:
WEBoggle needs a new game board every three minutes. Boards take an unpredictable (much less than 3min, but non-trivial) amount of time to generate. The system is driven by web requests, and I don't want the request that happens to trigger the need for the new board to have to pay the time cost of generating it. I set up a producer thread that does nothing but generate boards and put them into a length-two Queue (blocking). At the rate...
0
1393
by: Tom Brown | last post by:
I need to chain together three linux commands and get the final output. I read the documentation for Popen in the subprocess module for replacing the shell pipe line. I followed the example and keep getting a 0 where I should be getting a 1. I am trying to do this: grep "Sep 22" /var/log/auth.log | grep "Illegal user" | wc -l which returns a 1 when I run this manually. This is what I did with python:
3
3525
by: smith4894 | last post by:
Hello, I have an application that essentially consists of two threads doing their things. One thread is a producer, and pushes bytes (of a struct) into a pipe, and another is a consumer that simply checks the pipe at regular intervals reads out the bytes. If the consumer extracts bytes and sees that it's not enough to recast to the struct, it caches it and waits for more bytes in the pipe. My question is: Do I still need to maintain...
2
6023
by: ShawnD | last post by:
I'm having some issues when trying to read input off of a pipe using a python script. I'm trying to process packet data from tcpdump in real-time, so it's a filter that needs to read data while the process being piped from is still reading. On both my Linux and OSX system, it seems that there's a large delay reading from stdin if the process being read from is still running, even if it's producing output to be read. It will wait until it...
4
2627
by: jelle | last post by:
Hi, I use python quite a bit to couple different programs together. Doing so has been a _lot_ easier since subprocess came around, but would really like to be able to use the succinct shell syntax; >, <, | That really shouldn't be too hard to wrap in a class, but so far I didn't succeed to do so this well, since I'm facing some trouble with operator precedence that I do not know how to overcome.
2
3981
by: Jeremy Moles | last post by:
I'm not sure if this is really the right place to ask this question, but since the implementation is in Python, I figured I'd give it a shot. I want to "wrap" a shell process using popen inside of python program rather than creating a new shell process for each line I process in the app. For example, the code might look like: std = stdin, stdout, stderr = os.popen3("bash")
3
3831
by: George Sakkis | last post by:
I'm trying to figure out why Popen captures the stderr of a specific command when it runs through the shell but not without it. IOW: cmd = if 1: # this captures both stdout and stderr as expected pipe = Popen(' '.join(cmd), shell=True, stderr=PIPE, stdout=PIPE) else: # this captures only stdout pipe = Popen(cmd, shell=False, stderr=PIPE, stdout=PIPE) # this prints the empty string if not run through the shell
1
3560
by: raocheng | last post by:
Please see the following code. Suppose I have many shell commands to be executed. And I don't want to fork a sub shell for each command(eg: status,output = commands.getstatusoutput(cmd)) because it is too expensive. I want to use only one sub shell to execute all these commands and want to get each command's output. How can I accomplish this task ? Thanks in advance. =========================================== #!/usr/bin/env python
7
6240
by: Samuel A. Falvo II | last post by:
I have a shell script script.sh that launches a Java process in the background using the &-operator, like so: #!/bin/bash java ... arguments here ... & In my Python code, I want to invoke this shell script using the Subprocess module. Here is my code: def resultFromRunning_(command):
0
9685
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10247
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10214
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10023
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9067
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5459
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5583
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4135
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2935
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.