473,326 Members | 2,655 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Problem flushing stderr in embedded python

Hi,

My application has python embedded into it. I noticed that when I run any
python code the output is buffered and doesn't get flushed until my
application exits. To fix this I simply flush sys.stdout and sys.stderr
every once in while by using the following code:

//Get handle to python stdout file and flush it
PyObject *pyStdout = PySys_GetObject("stdout");
if(pyStdout && PyFile_Check(pyStdout)) {
PyObject *result = PyObject_CallMethod(pyStdout,"flush",NULL);
Py_XDECREF(result);
}

//Get handle to python stderr file and flush it
PyObject *pyStderr = PySys_GetObject("stderr");
if(pyStderr && PyFile_Check(pyStderr)) {
PyObject *result = PyObject_CallMethod(pyStderr,"flush",NULL);
Py_XDECREF(result);
}

This works for stdout but not for stderr. In order to flush stderr I have to
do:

PyRun_SimpleString("sys.stderr.flush()");

Does anybody know why my first method doesn't work with stderr? I checked
and PyObject_CallMethod is being called for both objects and no errors are
occuring. The solution I have is acceptable, but I would still like to know
why my first method didn't work.

BTW, I'm running python 2.3 on Windows XP

Thanks,

Farshid
Jul 18 '05 #1
6 4410
"Farshid Lashkari" <la********@SPAMworldviz.com> wrote in message news:<VoRjd.33$iY3.7@trnddc01>...
Hi,

My application has python embedded into it. I noticed that when I run any
python code the output is buffered and doesn't get flushed until my
application exits. To fix this I simply flush sys.stdout and sys.stderr
every once in while by using the following code:

//Get handle to python stdout file and flush it
PyObject *pyStdout = PySys_GetObject("stdout");
if(pyStdout && PyFile_Check(pyStdout)) {
PyObject *result = PyObject_CallMethod(pyStdout,"flush",NULL);
Py_XDECREF(result);
}

//Get handle to python stderr file and flush it
PyObject *pyStderr = PySys_GetObject("stderr");
if(pyStderr && PyFile_Check(pyStderr)) {
PyObject *result = PyObject_CallMethod(pyStderr,"flush",NULL);
Py_XDECREF(result);
}

This works for stdout but not for stderr. In order to flush stderr I have to
do:

PyRun_SimpleString("sys.stderr.flush()");

Does anybody know why my first method doesn't work with stderr? I checked
and PyObject_CallMethod is being called for both objects and no errors are
occuring. The solution I have is acceptable, but I would still like to know
why my first method didn't work.

BTW, I'm running python 2.3 on Windows XP

Thanks,

Farshid

I'm sorry to notice that you haven't received any responses back to
your questions even though you have found a workaround. I've noticed
this trend quite a lot in the python community and having learned
python first and now C(in order to understand python better) I'm
frankly disspapointed. I believe python and C are a wonderful marriage
but I suppose because many ppl come to python as their intro language
they info on extending and embedding is limited. I myself haven't
started using the python api but will soon, I hope I for one will be
someone who will be able to provide an answer next time.

Good luck.

BTW: from a C perspective. I don't understand your problem though.
stdout and stderr are the normal output device for your system -
usally your screen. I don't see why you should have to call one via
python sys module.
Perhaps because one doesn't normally flush(stderr) explicity but
flush(stdout).

interested to know as well.
Jul 18 '05 #2
I don't understand it from a C perspective either. I poked around the source
code for python and found where it is flushing the file. It is simply
calling fflush on the FILE handle. Also, I'm somewhat bewildered as to why
running the string "sys.stderr.flush()" is different than directly calling
the flush method of the stderr object. Oh well, I guess some things are
better left unknown.

"caroundw5h" <ca********@yahoo.com> wrote in message
news:aa**************************@posting.google.c om...
"Farshid Lashkari" <la********@SPAMworldviz.com> wrote in message

news:<VoRjd.33$iY3.7@trnddc01>...
Hi,

My application has python embedded into it. I noticed that when I run any python code the output is buffered and doesn't get flushed until my
application exits. To fix this I simply flush sys.stdout and sys.stderr
every once in while by using the following code:

//Get handle to python stdout file and flush it
PyObject *pyStdout = PySys_GetObject("stdout");
if(pyStdout && PyFile_Check(pyStdout)) {
PyObject *result = PyObject_CallMethod(pyStdout,"flush",NULL);
Py_XDECREF(result);
}

//Get handle to python stderr file and flush it
PyObject *pyStderr = PySys_GetObject("stderr");
if(pyStderr && PyFile_Check(pyStderr)) {
PyObject *result = PyObject_CallMethod(pyStderr,"flush",NULL);
Py_XDECREF(result);
}

This works for stdout but not for stderr. In order to flush stderr I have to do:

PyRun_SimpleString("sys.stderr.flush()");

Does anybody know why my first method doesn't work with stderr? I checked and PyObject_CallMethod is being called for both objects and no errors are occuring. The solution I have is acceptable, but I would still like to know why my first method didn't work.

BTW, I'm running python 2.3 on Windows XP

Thanks,

Farshid

I'm sorry to notice that you haven't received any responses back to
your questions even though you have found a workaround. I've noticed
this trend quite a lot in the python community and having learned
python first and now C(in order to understand python better) I'm
frankly disspapointed. I believe python and C are a wonderful marriage
but I suppose because many ppl come to python as their intro language
they info on extending and embedding is limited. I myself haven't
started using the python api but will soon, I hope I for one will be
someone who will be able to provide an answer next time.

Good luck.

BTW: from a C perspective. I don't understand your problem though.
stdout and stderr are the normal output device for your system -
usally your screen. I don't see why you should have to call one via
python sys module.
Perhaps because one doesn't normally flush(stderr) explicity but
flush(stdout).

interested to know as well.

Jul 18 '05 #3
Quoth "Farshid Lashkari" <la********@SPAMworldviz.com>:
| I don't understand it from a C perspective either. I poked around the source
| code for python and found where it is flushing the file. It is simply
| calling fflush on the FILE handle. Also, I'm somewhat bewildered as to why
| running the string "sys.stderr.flush()" is different than directly calling
| the flush method of the stderr object. Oh well, I guess some things are
| better left unknown.

It could be different if the stderr object - i.e., PySys_GetObject("stderr") -
is actually not sys.stderr. I don't understand why, but am not familiar
with PySys_*, so I offer that as a hypothesis you might be able to test.
By the way, since as you observed it's only calling fflush(), I think you
could leave Python out of it altogether and do this with just
fflush(stdout);
fflush(stderr);

Donn Cave, do**@u.washington.edu
Jul 18 '05 #4
Here's the code for PySys_GetObject. It's getting the object from the sys
dictionary, so it should be the correct object.

PyObject *
PySys_GetObject(char *name)
{
PyThreadState *tstate = PyThreadState_Get();
PyObject *sd = tstate->interp->sysdict;
if (sd == NULL)
return NULL;
return PyDict_GetItemString(sd, name);
}

I've tried calling fflush on stderr, but that doesn't work either.

I've encountered another weird problem also. If my python code causes an
error and I don't flush stderr, then I get a runtime error when my app
exits. It also prints out:

Exception exceptions.NameError: "global name 'y' is not defined" in 'garbage
collection' ignored
Fatal Python error: unexpected exception during garbage collection.

Does this give anybody a clue as to what's happening?

"Donn Cave" <do**@drizzle.com> wrote in message
news:1100065836.697971@yasure...
Quoth "Farshid Lashkari" <la********@SPAMworldviz.com>:
| I don't understand it from a C perspective either. I poked around the source | code for python and found where it is flushing the file. It is simply
| calling fflush on the FILE handle. Also, I'm somewhat bewildered as to why | running the string "sys.stderr.flush()" is different than directly calling | the flush method of the stderr object. Oh well, I guess some things are
| better left unknown.

It could be different if the stderr object - i.e., PySys_GetObject("stderr") - is actually not sys.stderr. I don't understand why, but am not familiar
with PySys_*, so I offer that as a hypothesis you might be able to test.
By the way, since as you observed it's only calling fflush(), I think you
could leave Python out of it altogether and do this with just
fflush(stdout);
fflush(stderr);

Donn Cave, do**@u.washington.edu

Jul 18 '05 #5
Quoth "Farshid Lashkari" <la********@SPAMworldviz.com>:
....
| I've tried calling fflush on stderr, but that doesn't work either.

OK, this would seem to mean that your sys.stderr has been replaced
with something else. I guess I don't know for sure what you can
count on in Windows, but I would have to expect that fflush on stderr
really does work - that is, it flushes stderr's buffer.

| I've encountered another weird problem also. If my python code causes an
| error and I don't flush stderr, then I get a runtime error when my app
| exits. It also prints out:
|
| Exception exceptions.NameError: "global name 'y' is not defined" in 'garbage
| collection' ignored
| Fatal Python error: unexpected exception during garbage collection.
|
| Does this give anybody a clue as to what's happening?

Doesn't do anything for me, but you may find out what that is when you
locate the code for the faux stderr you're (not) flushing. I would be
more worried about this 'y' thing in principle, though, because it sounds
like a problem in the C (or whatever) to Python interface that could have
other unpleasant consequences.

Donn Cave, do**@drizzle.com
Jul 18 '05 #6
Well I've somewhat solved my problem. First off, the 'y' problem was just an
error I purposely created in order to trigger an exception. My application
uses a bunch of calls to PyObject_Call to run python code. After each of
these calls I added:

if(PyErr_Occurred()) PyErr_Print();

This will force the errors to be printed to stderr, and now calling the
flush method on the stderr object works. I still don't understand why
running the string "sys.stderr.flush()" is different. Maybe it forces any
current errors to be printed and then flushes them too. My app doesn't
replace the python stderr object either.

Anyways, thanks to everybody for your help and suggestions.

-Farshid

"Donn Cave" <do**@drizzle.com> wrote in message
news:1100156058.567069@yasure...
Quoth "Farshid Lashkari" <la********@SPAMworldviz.com>:
...
| I've tried calling fflush on stderr, but that doesn't work either.

OK, this would seem to mean that your sys.stderr has been replaced
with something else. I guess I don't know for sure what you can
count on in Windows, but I would have to expect that fflush on stderr
really does work - that is, it flushes stderr's buffer.

| I've encountered another weird problem also. If my python code causes an
| error and I don't flush stderr, then I get a runtime error when my app
| exits. It also prints out:
|
| Exception exceptions.NameError: "global name 'y' is not defined" in 'garbage | collection' ignored
| Fatal Python error: unexpected exception during garbage collection.
|
| Does this give anybody a clue as to what's happening?

Doesn't do anything for me, but you may find out what that is when you
locate the code for the faux stderr you're (not) flushing. I would be
more worried about this 'y' thing in principle, though, because it sounds
like a problem in the C (or whatever) to Python interface that could have
other unpleasant consequences.

Donn Cave, do**@drizzle.com

Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Vincent Touquet | last post by:
Hi, In a project where I have embedded Python in a C++ application, I have the need to replace what Python considers to be stdin, stdout and/or stderr. In sysmodule.c in the Python sources, I...
3
by: gf gf | last post by:
Is there any way to make Python's print() flush automatically, similar to...mmm...that other language's $|=1 ? If not, how can I flush it manually? sys.stdout.flush() didn't seem to work. ...
9
by: Bryan Olson | last post by:
Here's a module to show stderr output from console-less Python apps, and stay out of the way otherwise. I plan to make a ASPN recipe of it, but I thought I'd run it by this group first. To use...
2
by: Xah Lee | last post by:
Python Doc Problem Example: os.system Xah Lee, 2005-09 today i'm trying to use Python to call shell commands. e.g. in Perl something like output=qx(ls) in Python i quickly located the...
0
by: Stewart Midwinter | last post by:
I have a Tkinter app running on cygwin. It includes a Test menu item that does nothing more than fetch a directory listing and display it in a Toplevel window (I'd use a tkMessageBox showinfo...
10
by: A.M | last post by:
Hi, I am having difficulty with shell scripting in Python. I use the following command to run a DOS command and put the return value in a Python variable:
0
by: Robin Becker | last post by:
We have a client using a fully embedded python in a large DTP app. It used to be Mac OS 9/X only, but the MAC 9 support has gone away and we now have support for the PC with the embedding being...
37
by: Vince C. | last post by:
Hi all. I've installed Bloodshed Dev-C++ on a Windows 2000 SP4 machine. I'm using MinGW 3.4.2. I'd like to temporarily disable standard functions to write to stderr, i.e. for instance...
1
by: grbgooglefan | last post by:
I am in a perculiar situation. I want to use PyRun_SimpleString for creating Python functions in embedded Python in C++. But there could be cases when Python function code compilation could fail &...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.