sys.stdin.readline()

[Mike Maxwell]

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

for sL in sys.stdin.readline(): print sL

....
abc
a
b
c

(I typed in 'abc', and the loop printed out 'a\nb\nc\n')

I.e. how can I make readline() wait for the newline before returning a
value? 'readline()' seems to be acting exactly like 'read()' here.

('readlines()' works fine in this context, except that it waits for
eof; I'd really rather iterate over lines in stdin as they come in)

[Grant Edwards]

On 2004-08-31, Mike Maxwell <ma*****@ldc.upenn.edu> wrote:

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

for sL in sys.stdin.readline(): print sL

...
abc
a
b
c

(I typed in 'abc', and the loop printed out 'a\nb\nc\n')

I.e. how can I make readline() wait for the newline before returning a
value?

It is.
'readline()' seems to be acting exactly like 'read()' here.

Sort of.

What you want is:

import sys
while True:
s = sys.stdin.readline()
if not s:
break
print s

--
Grant Edwards grante Yow! .. I see TOILET
at SEATS...
visi.com

[Hallvard B Furuseth]

Mike Maxwell wrote:

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

for sL in sys.stdin.readline(): print sL

It does return a full line. *One* line. Then your loop iterates
over the characters in that line.

Try `for sL in sys.stdin.xreadlines(): print sL'.
Or in newer Pythons, simply `for sL in sys.stdin: print sL'.

--
Hallvard

[Steven Bethard]

Hallvard B Furuseth <h.b.furuseth <at> usit.uio.no> writes:

Or in newer Pythons, simply `for sL in sys.stdin: print sL'.

This doesn't work for me in the interactive shell:

Python 2.4a2 (#55, Aug 5 2004, 11:42:43) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import sys
for l in sys.stdin:

.... print repr(l)
....
A
B
C
D
^Z
'A\n'
'B\n'
'C\n'
'D\n'

Note that I don't get output until I hit ^Z (this is, obviously, a Windows
box). I tried starting Python with the -u option too, and I still get this
result. Is there any way to make 'for l in sys.stdin' work the way you
suggest it does at an interactive prompt?

Steve

[Hallvard B Furuseth]

Steven Bethard wrote:

Hallvard B Furuseth <h.b.furuseth <at> usit.uio.no> writes:

Or in newer Pythons, simply `for sL in sys.stdin: print sL'.
This doesn't work for me in the interactive shell:
(...)

Whoops - you just found a mysterious bug in a program I'm writing.
Thanks:-)
Note that I don't get output until I hit ^Z (this is, obviously, a Windows
box). I tried starting Python with the -u option too, and I still get this
result. Is there any way to make 'for l in sys.stdin' work the way you
suggest it does at an interactive prompt?

Can't see any in the manual, now that I've looked more closely.
Info node "(python-lib)File Objects" says file.next() uses a read-
ahead buffer. It doesn't say one can disable that.

--
Hallvard

[Mike Maxwell]

Hallvard B Furuseth wrote:

Mike Maxwell wrote:

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

>for sL in sys.stdin.readline(): print sL

It does return a full line. *One* line. Then your loop iterates
over the characters in that line.

LoL, thanks!
Try `for sL in sys.stdin.xreadlines(): print sL'.
Or in newer Pythons, simply `for sL in sys.stdin: print sL'.

I think I saw that, but when I tried it, I got:

/lib/python2.3/pydoc.py:250: DeprecationWarning:
xreadlines is deprecated; use 'for line in file'.

--which got me off on this dead end.

What I was originally trying to do, is to implement a one-liner that
would act something like 'sed', but applying to Unicode characters. An
example would have a command that looked something like
"s/u'\u0D0A'//"
i.e. delete all instances of the Unicode char U+0D0A (which you can't do
with 'sed', at least not the version that I'm using).

The guy down the hall does these kinds of things with perl one-liners,
but I have more dignity than to use perl... Unfortunately, it's looking
more and more complex to do one-liners like this in Python. Am I
overlooking s.t.?

Mike Maxwell

[Alex Martelli]

Mike Maxwell <ma*****@ldc.upenn.edu> wrote:
...

The guy down the hall does these kinds of things with perl one-liners,
but I have more dignity than to use perl... Unfortunately, it's looking
more and more complex to do one-liners like this in Python. Am I
overlooking s.t.?

No, I think you're correctly observing that Python isn't oriented to
one-liners -- not at all. Most interesting things in Python require
more than one line. You could write a "oneliners shell" that takes some
defined separator and turns it into a newline, of course, e.g.:

bangoneliner.py:
#!/usr/bin/env python
import sys
oneliner = sys.argv.pop(1)
exec '\n'.join(oneliner.split('!'))

now, sometying like

bangoneliner.py 'for x in xrange(7):! if x%2:! print x'

should work (note that inserting the spaces after the bangs to simulate
proper indentation IS a silly fuss, but you hafta...:-).
Alex

[Mike Maxwell]

Alex Martelli wrote:

Mike Maxwell <ma*****@ldc.upenn.edu> wrote:
No, I think you're correctly observing that Python isn't oriented to
one-liners -- not at all. Most interesting things in Python require
more than one line.
<rant>
I don't care whether it's "interesting", I just want to get some work
done. And since most of the text processing tools in Unixes that I
would otherwise use (grep, sed, tr) don't support Unicode, and are
inconsistent in their regular expression notation to boot, it would be
nice if I could write regex operations in a single, consistent
programming language. Python is a single, consistent programming
language, but as you say, it doesn't lend itself to one-liners.
</rant>
You could write a "oneliners shell" that takes some
defined separator and turns it into a newline, of course, e.g.:

bangoneliner.py:
#!/usr/bin/env python
import sys
oneliner = sys.argv.pop(1)
exec '\n'.join(oneliner.split('!'))

now, sometying like

bangoneliner.py 'for x in xrange(7):! if x%2:! print x'

should work
Hmm, I may give that a try...thanks!
note that inserting the spaces after the bangs to simulate
proper indentation IS a silly fuss, but you hafta...:-).

Well, I guess I could translate some other char (one that's easier to
count than spaces) into indents, too.

Mike Maxwell

[Alex Martelli]

Mike Maxwell <ma*****@ldc.upenn.edu> wrote:

Alex Martelli wrote:

Mike Maxwell <ma*****@ldc.upenn.edu> wrote:
No, I think you're correctly observing that Python isn't oriented to
one-liners -- not at all. Most interesting things in Python require
more than one line.
<rant>
I don't care whether it's "interesting", I just want to get some work
done.

Something that lets you get work done IS thereby interesting. Most
interesting things in Python require more than one line. So, I don't
see the basis for your rant.
And since most of the text processing tools in Unixes that I
would otherwise use (grep, sed, tr) don't support Unicode, and are
inconsistent in their regular expression notation to boot, it would be
nice if I could write regex operations in a single, consistent
programming language. Python is a single, consistent programming
language, but as you say, it doesn't lend itself to one-liners.

No, but a supporting script similar to the one I suggest below can
easily be adapter to offer more sensible functionality than any oneliner
might sensibly support -- for example (for the kinds of tasks you imply)
by including and automatically using such modules as re and fileinput.

now, sometying like

bangoneliner.py 'for x in xrange(7):! if x%2:! print x'

should work

Hmm, I may give that a try...thanks!

You're welcome.

note that inserting the spaces after the bangs to simulate
proper indentation IS a silly fuss, but you hafta...:-).

Well, I guess I could translate some other char (one that's easier to
count than spaces) into indents, too.

Sure, or you could use (e.g.) '!3' to translate into 'newline then three
spaces', or use block start/endmarkers and translate them into
indents/dedents, etc, etc.

Personally, given your now-restated problem, that you need 'better'
versions of grep, sed and tr, I would take another tack -- I would
reimplement _those_ in Python with its re sublanguage and Unicode
support. Using them should be easier and tighter than putting newliners
together, I think.
Alex

[Peter Hansen]

Mike Maxwell wrote:

Alex Martelli wrote:

Mike Maxwell <ma*****@ldc.upenn.edu> wrote:
No, I think you're correctly observing that Python isn't oriented to
one-liners -- not at all. Most interesting things in Python require
more than one line.

<rant>
I don't care whether it's "interesting", I just want to get some work
done. And since most of the text processing tools in Unixes that I
would otherwise use (grep, sed, tr) don't support Unicode, and are
inconsistent in their regular expression notation to boot, it would be
nice if I could write regex operations in a single, consistent
programming language. Python is a single, consistent programming
language, but as you say, it doesn't lend itself to one-liners.
</rant>

Most of us who do this conclude that our potential one-liner
is actually likely to be re-used, and we write the Python
three-liner equivalent in a file. Later on, we typically find
that it has grown to seventeen lines and we don't particularly
mind the slight extra inconvenience of having to put the
initial code in a file instead of trying to retype it each
time we need it.

-Peter

[Mike Maxwell]

I don't want to drag this out any longer. Thanks to all the posters.
Several mentioned that one-liners tend to both get re-used and to grow
to multiple lines, and that that implied that maybe the notion of a
one-liner was faulty. This may be true, and I may find myself doing
exactly what they suggest--building Unicode-aware versions of grep, sed,
and tr (hopefully not awk :-)). Time will tell.

Thank you also to Simon B. (si***@brunningonline.net) for pointing me to
his one-liner implementation, which is much better than I {w|c}ould have
done.

Mike Maxwell

[Hallvard B Furuseth]

I just realized...

Mike Maxwell wrote:

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

for sL in sys.stdin.readline(): print sL

for sL in iter(sys.stdin.readline, ''): print sL,

--
Hallvard

[Mike Maxwell]

Hallvard B Furuseth wrote:

Mike Maxwell wrote:

When I invoke readline() in a for loop, why does it return a series of
one-char strings, rather than the full line?

>for sL in sys.stdin.readline(): print sL

for sL in iter(sys.stdin.readline, ''): print sL,

You just forced me to look at the documentation for iter(), which I had
previously avoided doing (on the excuse that it was confusing). Thanks
for the push!

Mike Maxwell