473,216 Members | 2,217 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,216 software developers and data experts.

How to depress the output of an external module ?

Hi,

I'm writing a program which imports an external module writing in C and
calls a function provided by the module to do my job. But the method
produces
a lot of output to the stdout, and this consumes most of the running time.

My question is, is there a way to depress the output produced by the
function and hence make my program run faster? It's too complicated for me
to modify the source code and recompile the external module.

Any hints will be greatly appreciated.

Regards,

xiaojf

Dec 26 '06 #1
14 1743
On Tue, 26 Dec 2006 15:49:10 +0800, fd********@gmail.com wrote:
Hi,

I'm writing a program which imports an external module writing in C and
calls a function provided by the module to do my job. But the method
produces
a lot of output to the stdout, and this consumes most of the running time.

My question is, is there a way to depress the output produced by the
function and hence make my program run faster? It's too complicated for me
to modify the source code and recompile the external module.
Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

--
Steven.

Dec 26 '06 #2
On Tuesday, 26.12.06 at 21:28, Steven D'Aprano wrote:
>
# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result
There's no need for savestdout. There's a backup copy in sys.__stdout__
-Luis
Dec 26 '06 #3
Steven D'Aprano wrote:
Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result
Thanks!

I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.
Regards,

xiaojf
Dec 26 '06 #4

fd********@gmail.com wrote:
Steven D'Aprano wrote:
Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result
Thanks!

I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.
Perhaps try redirecting sys.stderr instead of sys.stdout.

André

Regards,

xiaojf
Dec 26 '06 #5
fd********@gmail.com wrote:
>
I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.
After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Still trying...

Regards,

xiaojf
Dec 26 '06 #6
Hi Luis,

Luis Armendariz wrote:
There's no need for savestdout. There's a backup copy in sys.__stdout__
Depending on the code that ran before the call to the
function run_without_stdout, sys.stdout may not be
the same as sys.__stdout__ . Of course, you also have
to be careful regarding threads which also use
sys.stdout, perhaps implicitly via print statements.

Stefan
Dec 26 '06 #7
fd********@gmail.com <fd********@gmail.comwrote
fd********@gmail.com wrote:
>>
I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified
version of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in
python don't produce output, but function() produces output to the
standard output just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.
After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the
function's execution.

Still trying...
On Linux systems you may try os.open('/dev/stdout', os.O_WRONLY'). This
will connect to lowest available file descriptor to standard output. If
you're lucky and no files have been opened after closing standard
output, sys.stdout will point to standard output again.

Bye
Sebastian 'lunar' Wiesner

--
Freedom is always the freedom of dissenters.
(Rosa Luxemburg)
Dec 26 '06 #8
On Tue, 26 Dec 2006 03:21:57 -0800, Luis Armendariz wrote:
On Tuesday, 26.12.06 at 21:28, Steven D'Aprano wrote:
>>
# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

There's no need for savestdout. There's a backup copy in sys.__stdout__
You shouldn't assume that sys.stdout is standard output when the function
is called. It may already have been reassigned to something else. My code
will restore sys.stdout to whatever state it was in before the function
ran.

--
Steven.

Dec 27 '06 #9
fd********@gmail.com wrote:
After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.
Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)

It's poor design and extremely inconsiderate for a library to print to
stdout except as an overridable default. (Not to mention it gives
ammo to people who want dynamically scoped globals.) A long term
solution might be to haggle the author to stop being such a jerk.

Carl Banks

Dec 27 '06 #10
fd********@gmail.com wrote:
Hi,

I'm writing a program which uses an external module written in C
and calls a function provided by the module to do my job. The
function produces a lot of output to the stdout.

Is there a way to suppress the output produced by the function and
hence make my program run faster?
It's too complicated for me to modify the source code and recompile
the external module.
This would be the best method, you could define printf and fprintf
macros that would eliminate the output code altogether.
Any hints will be greatly appreciated.
Well, it will depend on your OS, but the trick is to essentially
replace the C stdout channel with a file which has been opened to
write to "/dev/null" or "NUL.txt" (unix and Windows respectively).
You'll need to first copy the channel to another so you can use
it again after the function is done (a system call). Next do the
raw open (which should get the available channel), and the C stdout
stuff is successfully redirected. Once done w/ your function,
close your new stdout and copy the channel back.

--Scott David Daniels
sc***********@acm.org
Dec 27 '06 #11

Carl Banks wrote:
fd********@gmail.com wrote:
After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)
Also, right after closing file descriptor 1, you might want to set it
to something so as to eliminate an annoying warning message when Python
tries to close it upon termination but finds it already closed. This
opens /dev/null and puts it in file descriptor 1 if not there already
(the fdz != 1 test might be unnecessary; I don't know if all flavors of
Unix always use the lowest available file descriptor).

fdz = os.open("/dev/null",os.O_WRONLY)
if fdz != 1:
os.dup2(fdz,1)
os.close(fdz)
Carl Banks

Dec 27 '06 #12
Carl Banks wrote:
Carl Banks wrote:
>fd********@gmail.com wrote:
>>After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.
Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)

Also, right after closing file descriptor 1, you might want to set it
to something so as to eliminate an annoying warning message when Python
tries to close it upon termination but finds it already closed. This
opens /dev/null and puts it in file descriptor 1 if not there already
(the fdz != 1 test might be unnecessary; I don't know if all flavors of
Unix always use the lowest available file descriptor).

fdz = os.open("/dev/null",os.O_WRONLY)
Is it possible that I redirect low level standard output to
a file-like object created by cStringIO.StringIO() instead
of "/dev/null" ?
if fdz != 1:
os.dup2(fdz,1)
os.close(fdz)
Carl Banks

Dec 27 '06 #13

Scott David Daniels wrote:
fd********@gmail.com wrote:
Hi,

I'm writing a program which uses an external module written in C
and calls a function provided by the module to do my job. The
function produces a lot of output to the stdout.

Is there a way to suppress the output produced by the function and
hence make my program run faster?
It's too complicated for me to modify the source code and recompile
the external module.
This would be the best method, you could define printf and fprintf
macros that would eliminate the output code altogether.
Any hints will be greatly appreciated.
Well, it will depend on your OS, but the trick is to essentially
replace the C stdout channel with a file which has been opened to
write to "/dev/null" or "NUL.txt" (unix and Windows respectively).
You'll need to first copy the channel to another so you can use
it again after the function is done (a system call). Next do the
raw open (which should get the available channel), and the C stdout
stuff is successfully redirected. Once done w/ your function,
close your new stdout and copy the channel back.
In Windows the null device is, strictly speaking, "nul" or "nul:", not
"nul.txt", but the latter appears to work too.

Dec 27 '06 #14
MRAB wrote:
Scott David Daniels wrote:
>fd********@gmail.com wrote:
In Windows the null device is, strictly speaking, "nul" or "nul:", not
"nul.txt", but the latter appears to work too.
Although I find the windows design and reasoning to be a mistake, I
believe the use of file names NUL, PRN, and CON (at least) are diverted
to the corresponding device (data sink, printer, console) regardless of
directory and/or extension. The idea, I believe, was that too many
programs take a name from the user and slap the extension they intend to
use on the end, and even put a directory on the front. So, the OS
ignores those parts and switches to the chosen device. I use .txt for
two reasons: to remind me to avoid those names, and to indicate I am
doing straight text output.

--Scott David Daniels
sc***********@acm.org
Dec 30 '06 #15

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Fernando Rodriguez | last post by:
Hi, How can I save the output of an external program, called with os.system()?
2
by: PWR | last post by:
Platform: Windows Python: 2.3 Python Skill: Reading, Learning, but still newbie. Mission: My mission is writing a script that is triggered from my mail server. The script will execute and...
4
by: Avi Kak | last post by:
Is there a Python function in any of the standard-distribution modules that does what the backticks do in Perl? I want to run an external command and I'd like its output to be captured directly...
3
by: Yuan HOng | last post by:
In my program I have to call an external program and parse its output. For that I use the os.popen2 function, and then read the output stream. But the complexity is that the external program...
2
by: James Colannino | last post by:
Hey everyone. First off, I'm new to the list. I had had a little bit of experience with Perl before discovering Python. The more Python I learn, the more I love it :) I just have a quick...
1
by: S. van Beek | last post by:
Dear reader, In case an application is ussing an external model library you have to address this external model library in your application. This addressing takes place in a module of the...
5
by: amit.uttam | last post by:
Hey everyone, I've recently jumped big time into python and I'm working on a software program for testing automation. I had a question about proper logging of output. What I would like is: 1....
6
by: drhilbert | last post by:
Dear all, in these days, we have found a problem we can't solve even after long long googling, so we are here asking your precious help. In our program instead of using cout to print messages...
0
by: ravikim1 | last post by:
Hi , use Sort::External; $output = '1.txt'; open(FILE, $output); my $sortex = Sort::External->new( mem_threshold => 1024**2 * 1024 ); while (<FILE>) { $sortex->feed($_); # tried to...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.