How to depress the output of an external module ?

[fdu.xiaojf]

Hi,

I'm writing a program which imports an external module writing in C and
calls a function provided by the module to do my job. But the method
produces
a lot of output to the stdout, and this consumes most of the running time.

My question is, is there a way to depress the output produced by the
function and hence make my program run faster? It's too complicated for me
to modify the source code and recompile the external module.

Any hints will be greatly appreciated.

Regards,

xiaojf

[Steven D'Aprano]

On Tue, 26 Dec 2006 15:49:10 +0800, fd********@gmail.com wrote:

Hi,

I'm writing a program which imports an external module writing in C and
calls a function provided by the module to do my job. But the method
produces
a lot of output to the stdout, and this consumes most of the running time.

My question is, is there a way to depress the output produced by the
function and hence make my program run faster? It's too complicated for me
to modify the source code and recompile the external module.

Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

--
Steven.

[Luis Armendariz]

On Tuesday, 26.12.06 at 21:28, Steven D'Aprano wrote:

>
# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

There's no need for savestdout. There's a backup copy in sys.__stdout__
-Luis

[fdu.xiaojf]

Steven D'Aprano wrote:

Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

Thanks!

I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.
Regards,

xiaojf

[=?iso-8859-1?B?QW5kcuk=?=]

fd********@gmail.com wrote:

Steven D'Aprano wrote:

Try something like this:

# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

Thanks!

I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.

Perhaps try redirecting sys.stderr instead of sys.stdout.

André

Regards,

xiaojf

[fdu.xiaojf]

fd********@gmail.com wrote:

>
I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified version
of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in python
don't produce output, but function() produces output to the standard output
just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.

After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Still trying...

Regards,

xiaojf

[Stefan Schwarzer]

Hi Luis,

Luis Armendariz wrote:

There's no need for savestdout. There's a backup copy in sys.__stdout__

Depending on the code that ran before the call to the
function run_without_stdout, sys.stdout may not be
the same as sys.__stdout__ . Of course, you also have
to be careful regarding threads which also use
sys.stdout, perhaps implicitly via print statements.

Stefan

[Sebastian 'lunar' Wiesner]

fd********@gmail.com <fd********@gmail.comwrote

fd********@gmail.com wrote:

>>
I have tried your method, but I found it didn't work as expected.

The output produced by the external function couldn't be depressed,
but the "print " statement i wrote in python is depressed. It seems
make cStringIO.StringIO() as a temporary replacement of sys.stdout
has no effect on the external function.

Here is an example to make myself clear(actually it's modified
version of Steven's code):

def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
print "something"
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
print "some other thing"
return result

When run_without_stdout() is called, the "print" statements wrote in
python don't produce output, but function() produces output to the
standard output just as before:(

I have tried to replace sys.stdout globally with cStringIO.StringIO()
in my program(I mean, make "sys.stdout = cStringIO.StringIO()" as a
globall statement), but it worked just as previous version did.

After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the
function's execution.

Still trying...

On Linux systems you may try os.open('/dev/stdout', os.O_WRONLY'). This
will connect to lowest available file descriptor to standard output. If
you're lucky and no files have been opened after closing standard
output, sys.stdout will point to standard output again.

Bye
Sebastian 'lunar' Wiesner

--
Freedom is always the freedom of dissenters.
(Rosa Luxemburg)

[Steven D'Aprano]

On Tue, 26 Dec 2006 03:21:57 -0800, Luis Armendariz wrote:

On Tuesday, 26.12.06 at 21:28, Steven D'Aprano wrote:

>>
# WARNING: untested
def run_without_stdout(*args, **kwargs):
function = args[0]
args = args[1:]
savestdout = sys.stdout
sys.stdout = cStringIO.StringIO()
result = None
try:
result = function(*args, **kwargs)
finally:
# don't forget to restore stdout, or you
# really will regret it...
sys.stdout = savestdout
return result

There's no need for savestdout. There's a backup copy in sys.__stdout__

You shouldn't assume that sys.stdout is standard output when the function
is called. It may already have been reassigned to something else. My code
will restore sys.stdout to whatever state it was in before the function
ran.

--
Steven.

[Carl Banks]

fd********@gmail.com wrote:

After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)

It's poor design and extremely inconsiderate for a library to print to
stdout except as an overridable default. (Not to mention it gives
ammo to people who want dynamically scoped globals.) A long term
solution might be to haggle the author to stop being such a jerk.

Carl Banks

[Scott David Daniels]

fd********@gmail.com wrote:

Hi,

I'm writing a program which uses an external module written in C
and calls a function provided by the module to do my job. The
function produces a lot of output to the stdout.

Is there a way to suppress the output produced by the function and
hence make my program run faster?

It's too complicated for me to modify the source code and recompile
the external module.

This would be the best method, you could define printf and fprintf
macros that would eliminate the output code altogether.

Any hints will be greatly appreciated.

Well, it will depend on your OS, but the trick is to essentially
replace the C stdout channel with a file which has been opened to
write to "/dev/null" or "NUL.txt" (unix and Windows respectively).
You'll need to first copy the channel to another so you can use
it again after the function is done (a system call). Next do the
raw open (which should get the available channel), and the C stdout
stuff is successfully redirected. Once done w/ your function,
close your new stdout and copy the channel back.

--Scott David Daniels
sc***********@acm.org

[Carl Banks]

Carl Banks wrote:

fd********@gmail.com wrote:

After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)

Also, right after closing file descriptor 1, you might want to set it
to something so as to eliminate an annoying warning message when Python
tries to close it upon termination but finds it already closed. This
opens /dev/null and puts it in file descriptor 1 if not there already
(the fdz != 1 test might be unnecessary; I don't know if all flavors of
Unix always use the lowest available file descriptor).

fdz = os.open("/dev/null",os.O_WRONLY)
if fdz != 1:
os.dup2(fdz,1)
os.close(fdz)
Carl Banks

[fdu.xiaojf]

Carl Banks wrote:

Carl Banks wrote:

>fd********@gmail.com wrote:

>>After some trials I found that put "os.close(1)" before calling the
function will depress the output. In fact, "os.close(1)" closed
standard output, but I don't know how to open it again after the function's
execution.

Try this:

fd = os.dup(1)
os.close(1)
sys.stdout = os.fdopen(fd)

Also, right after closing file descriptor 1, you might want to set it
to something so as to eliminate an annoying warning message when Python
tries to close it upon termination but finds it already closed. This
opens /dev/null and puts it in file descriptor 1 if not there already
(the fdz != 1 test might be unnecessary; I don't know if all flavors of
Unix always use the lowest available file descriptor).

fdz = os.open("/dev/null",os.O_WRONLY)

Is it possible that I redirect low level standard output to
a file-like object created by cStringIO.StringIO() instead
of "/dev/null" ?

if fdz != 1:
os.dup2(fdz,1)
os.close(fdz)
Carl Banks

[MRAB]

Scott David Daniels wrote:

fd********@gmail.com wrote:

Hi,

I'm writing a program which uses an external module written in C
and calls a function provided by the module to do my job. The
function produces a lot of output to the stdout.

Is there a way to suppress the output produced by the function and
hence make my program run faster?

It's too complicated for me to modify the source code and recompile
the external module.

This would be the best method, you could define printf and fprintf
macros that would eliminate the output code altogether.

Any hints will be greatly appreciated.

Well, it will depend on your OS, but the trick is to essentially
replace the C stdout channel with a file which has been opened to
write to "/dev/null" or "NUL.txt" (unix and Windows respectively).
You'll need to first copy the channel to another so you can use
it again after the function is done (a system call). Next do the
raw open (which should get the available channel), and the C stdout
stuff is successfully redirected. Once done w/ your function,
close your new stdout and copy the channel back.

In Windows the null device is, strictly speaking, "nul" or "nul:", not
"nul.txt", but the latter appears to work too.

[Scott David Daniels]

MRAB wrote:

Scott David Daniels wrote:

>fd********@gmail.com wrote:

In Windows the null device is, strictly speaking, "nul" or "nul:", not
"nul.txt", but the latter appears to work too.

Although I find the windows design and reasoning to be a mistake, I
believe the use of file names NUL, PRN, and CON (at least) are diverted
to the corresponding device (data sink, printer, console) regardless of
directory and/or extension. The idea, I believe, was that too many
programs take a name from the user and slap the extension they intend to
use on the end, and even put a directory on the front. So, the OS
ignores those parts and switches to the chosen device. I use .txt for
two reasons: to remind me to avoid those names, and to indicate I am
doing straight text output.

--Scott David Daniels
sc***********@acm.org