Can someone explain this unexpected raw_input behavior?

[Mike Kent]

It's often useful for debugging to print something to stderr, and to
route the error output to a file using '2>filename' on the command
line.

However, when I try that with a python script, all prompt output from
raw_input goes to stderr. Consider the following test program:

=== Start test.py ===
import sys

def main():
print "Hello world"
raw_input("Press ENTER")
print "Goodbye world"

if __name__ == "__main__":
main()
=== End test.py ===

If I run it with the command line 'python2.5 test.py', I get the
following output:

Hello world
Press ENTER <=== This appeared,the program paused, I press ENTER,
and the program continued
Goodbye world

However, if I run it with the command line 'python2.5 test.py 2>/dev/
null' (I'm routing stderr output to /dev/null), I instead get:

Hello world
<=== No output appeared, the program paused, I
press ENTER, and the program continued
Goodbye world

This indicates to me that the prompt output of raw_input is being sent
to stderr. I did check the source code for raw_input, and it appears
to be sending it to stdout as expected.

I get this behavior on multiple OS platforms, with multiple versions
of Python. I am building python on these platforms myself, but to my
knowledge, I am not doing anything special which could account for
this behavior.

Any suggestions or pointers on how to get the expected behavior out of
raw_input?

[Gabriel Genellina]

En Wed, 23 Jan 2008 18:27:56 -0200, Mike Kent <mr******@cox.netescribió:

It's often useful for debugging to print something to stderr, and to
route the error output to a file using '2>filename' on the command
line.

However, when I try that with a python script, all prompt output from
raw_input goes to stderr. Consider the following test program:
[...]
This indicates to me that the prompt output of raw_input is being sent
to stderr. I did check the source code for raw_input, and it appears
to be sending it to stdout as expected.

Surely you've seen that in bltinmodule.c, builtin_raw_input calls
PyOS_Readline(PyFile_AsFile(fin), PyFile_AsFile(fout), prompt);
where fin and fout are sys.stdin and sys.stdout respectively.
That function is defined in Parser/myreadline.c, and eventually calls
PyOS_StdioReadline with the same arguments. But PyOS_StdioReadline doesn't
use its sys_stdout parameter to output the prompt; instead, it always uses
stderr, no matter what arguments it receives.
Looking at the svn history, PyOS_StdioReadline always has used stderr; got
its current signature in rev 29400 (2002).

Perhaps that behavior is based on the reverse of your use case, which may
be more common. A script writes most of its output to stdout, and you
capture that using >filename. If raw_input prompted the user using stdout,
that would not be visible in this case, so using stderr is more useful.

--
Gabriel Genellina

[Mike Kent]

Gabriel, thank you for clarifying the source of this behavior. Still,
I'm surprised it would be hard-coded into Python. Consider an
interactive program, that asks the user several questions, and
displays paragraphs of information based on those questions. The
paragraphs are output using print, and the questions are asked via
raw_input. You want to do some simple debugging of the program by
printing some debugging statements via 'print >>sys.stderr', and you
don't want the debug output mixed in with the normal output on the
screen, so you try to route the debugging output to a file by adding
'2>filename' to the end of the command line.

Unfortunately, you will no longer see any of the questions being
printed via raw_input. The rest of the output will be fine, but the
questions disappear. Your program just stops, without asking
anything... you have to guess what should be there.

I'm surprised that Python hard-codes this behavior without giving the
programmer any way to change it. It leaves me with two options: to
either always use the logging module for debugging messages (which is
not odious at all, it's just that the code in question predates the
logging module, which is why debugging was done as it is), or change
the program so that raw_input is never used with a prompt parameter;
the prompt must always be printed separately.

<shrug At least I now know I'm not crazy... regarding this, anyway.

[Ben Finney]

Mike Kent <mr******@cox.netwrites:

Consider an interactive program, that asks the user several
questions, and displays paragraphs of information based on those
questions. The paragraphs are output using print, and the questions
are asked via raw_input.

Okay so far.

You want to do some simple debugging of the program by printing some
debugging statements via 'print >>sys.stderr', and you don't want
the debug output mixed in with the normal output on the screen, so
you try to route the debugging output to a file by adding
'2>filename' to the end of the command line.

This issue isn't specific to Python. "Program stops to ask questions
from the user" is not compatible with "Can safely redirect output of
the program to a file".

Either one, or both, of those requirements will have to be
compromised, or dropped completely.

--
\ "Holy hole in a donut, Batman!" -- Robin |
`\ |
_o__) |
Ben Finney

[Gabriel Genellina]

En Thu, 24 Jan 2008 01:00:53 -0200, Mike Kent <mr******@cox.netescribió:

Gabriel, thank you for clarifying the source of this behavior. Still,
I'm surprised it would be hard-coded into Python. Consider an
interactive program, that asks the user several questions, and
displays paragraphs of information based on those questions. The
paragraphs are output using print, and the questions are asked via
raw_input. You want to do some simple debugging of the program by
printing some debugging statements via 'print >>sys.stderr', and you
don't want the debug output mixed in with the normal output on the
screen, so you try to route the debugging output to a file by adding
'2>filename' to the end of the command line.

Unfortunately, you will no longer see any of the questions being
printed via raw_input. The rest of the output will be fine, but the
questions disappear. Your program just stops, without asking
anything... you have to guess what should be there.

You have one console, two streams to output data (stdout and stderr), and
three data sources (program output, user prompt, and debugging messages).
Someone has to give.
I'm now convinced that the current behavior is rather reasonable...

I'm surprised that Python hard-codes this behavior without giving the
programmer any way to change it. It leaves me with two options: to
either always use the logging module for debugging messages (which is
not odious at all, it's just that the code in question predates the
logging module, which is why debugging was done as it is), or change
the program so that raw_input is never used with a prompt parameter;
the prompt must always be printed separately.

Perhaps raw_input could have a use_stderr=True parameter; with False would
display the prompt on stdout.

--
Gabriel Genellina

[Ben Finney]

Mike Kent <mr******@cox.netwrites:

A bug issue has been opened in the Python Trac system for this.

Wouldn't it be better to report it in the official Python bug tracker
<URL:http://bugs.python.org/>, which is Roundup, not Trac?

--
\ "The right to use [strong cryptography] is the right to speak |
`\ Navajo." -- Eben Moglen |
_o__) |
Ben Finney

[Terry Reedy]

"Ben Finney" <bi****************@benfinney.id.auwrote in message
news:87***************@benfinney.id.au...
| Mike Kent <mr******@cox.netwrites:
|
| A bug issue has been opened in the Python Trac system for this.
|
| Wouldn't it be better to report it in the official Python bug tracker
| <URL:http://bugs.python.org/>, which is Roundup, not Trac?

Has been by Skip: http://bugs.python.org/issue1927

[Mike Kent]

On Jan 24, 5:13 pm, "Terry Reedy" <[e-mail]:

"Ben Finney" <[e-mail] in message

news:87***************@benfinney.id.au...
| Mike Kent <[e-mail]:
|
| A bug issue has been opened in the Python Trac system for this.
|
| Wouldn't it be better to report it in the official Python bug tracker
| <URL:http://bugs.python.org/>, which is Roundup, not Trac?

Has been by Skip: http://bugs.python.org/issue1927

My mistake in using the word 'Trac'.