reading from sys.stdin

[7stud]

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

But, I can break out of the for loop when I do this:

---------
import sys

lst = []
for line in open("aaa.txt"):
lst.append(line)
break

print lst
----------

Why doesn't break work in the first example?

[Diez B. Roggisch]

7stud schrieb:

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

Works for me. But only after the stdin is closed with a C-d.

I presume this is an OS thing. The first lines aren't communicated to
the process until either the file is closed - C-d - or the buffer the OS
puts before the stream is filled. You can switch to unbuffered behviour
somehow, google for it. Termios should be in your query.

Either way, it's not python behaving differently.

Diez

[Michael Bentley]

On Apr 12, 2007, at 3:20 AM, 7stud wrote:

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

But, I can break out of the for loop when I do this:

---------
import sys

lst = []
for line in open("aaa.txt"):
lst.append(line)
break

print lst
----------

Why doesn't break work in the first example?

My guess is because you've not entered an EOF. Try entering ^D (or
if you're on Windows I think it's ^Z) while using your first example
and see what it does.

[Gabriel Genellina]

En Thu, 12 Apr 2007 05:20:58 -0300, 7stud <bb**********@yahoo.com>
escribió:

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

Python 2.5.1c1 (r251c1:54692, Apr 5 2007, 09:19:18) [MSC v.1310 32 bit
(Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
pyimport sys
pylst = []
pyfor line in sys.stdin:
.... lst.append(line)
.... break
....
hola
que
tal
como
estan
^Z
pyprint lst
['hola\n']
-----------

I typed many lines, but lst contains only one item, as expected. Same as
your regular file example: the file contains many lines, but only the
first goes into the list.

--
Gabriel Genellina

[Maric Michaud]

Le jeudi 12 avril 2007 10:34, Diez B. Roggisch a écrit*:

I presume this is an OS thing. The first lines aren't communicated to
the process until either the file is closed - C-d - or the buffer the OS
puts before the stream is filled. You can switch to unbuffered behviour
somehow, google for it. Termios should be in your query.

I don't know if this a python or OS thing, but I know that iterating over a
file is not like applying successive call to readline method. You should try
to use readline instead.

The following work exactly the same on windows and Linux (python2.4) :

>>f=open('txt')
l=f.readline()
while(l) :

... print l,
... print "rest : " + f.read()
... l=f.readline()
...
foo
rest : bar
baz

works as expected, while :

>>f=open('txt')
for l in f :

... print l,
... print "rest : " + f.read()
...
foo
rest :
bar
rest :
baz
rest :

doesn't, it seems that file iteratiion itself use a buffer. In python 2.5, you
just can't do this :

Python 2.5 (release25-maint, Dec 9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>f=open('txt')
for l in f :

... print l,
... print "rest : " + f.read()
...
foo
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
ValueError: Mixing iteration and read methods would lose data

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: [phone]
Mobile: [phone]

[Steve Holden]

Maric Michaud wrote:

Le jeudi 12 avril 2007 10:34, Diez B. Roggisch a écrit :

>I presume this is an OS thing. The first lines aren't communicated to
the process until either the file is closed - C-d - or the buffer the OS
puts before the stream is filled. You can switch to unbuffered behviour
somehow, google for it. Termios should be in your query.

I don't know if this a python or OS thing, but I know that iterating over a
file is not like applying successive call to readline method. You should try
to use readline instead.

That, of course, is because files are iterators.

The following work exactly the same on windows and Linux (python2.4) :

>>>f=open('txt')
l=f.readline()
while(l) :

... print l,
... print "rest : " + f.read()
... l=f.readline()
...
foo
rest : bar
baz

works as expected, while :

>>>f=open('txt')
for l in f :

... print l,
... print "rest : " + f.read()
...
foo
rest :
bar
rest :
baz
rest :

doesn't, it seems that file iteratiion itself use a buffer. In python 2.5, you
just can't do this :

Python 2.5 (release25-maint, Dec 9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>>f=open('txt')
for l in f :

... print l,
... print "rest : " + f.read()
...
foo
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
ValueError: Mixing iteration and read methods would lose data

But you *can* do this (untested):

for l in f:
print l,
print "rest:", f.next()

Of course there are always edge cases. In this particular instance I
suspect that the handling of files with odd numbers of lines might be
slightly different, but I can't work up enough enthusiasm to actually
run this code. The error method pretty much explains what the problem
is: you have to iterate over files or read them, you shouldn't try and
do both on the same file.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

[Matimus]

On Apr 12, 1:20 am, "7stud" <[e-mail]:

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

But, I can break out of the for loop when I do this:

---------
import sys

lst = []
for line in open("aaa.txt"):
lst.append(line)
break

print lst
----------

Why doesn't break work in the first example?

1. Why are you breaking in the for loop? That will only give you the
first line.

2. Your issue is caused by buffered IO. Because of the buffered IO,
your code is equivalent to doing this:

lst = []
s = sys.stdin.read() #All* of the reading up to EOF is done upfront.

for line in s:
lst.append(line)
break # I'm sure you don't really want the break here though :)

One way to get around the buffering is to do this:

# Open a file or the input stream
if filename:
f = open(filename,"r")
else:
f = sys.stdin
 
# Then you check to see if your file is interactive
if f.isatty():
# This is unbuffered, and will iterate the same as f
f = iter(raw_input(),"")
 
# Now do your stuff
lst = []
for line in f:
lst.append(line)
 
print lst
 

* well, not ALL, it will read in chunks. But, I think they are 4096
Byte chunks by default.

[draghuram]

On Apr 12, 4:20 am, "7stud" <[e-mail]:

I can't break out of the for loop in this example:

------
import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst
-----------

You may want to look at a related issue:

http://www.python.org/sf/1633941

Thanks,
Raghu.

[draghuram]

On Apr 12, 10:25 am, "Matimus" <[e-mail]:

* well, not ALL, it will read in chunks. But, I think they are 4096
Byte chunks by default.

If you are referring to the read ahead buffer size, it is 8192 bytes.

Raghu.

[Maric Michaud]

Le jeudi 12 avril 2007 16:25, Matimus a écrit*:

# Then you check to see if your file is interactive
if f.isatty():
* * # This is unbuffered, and will iterate the same as f
* * f = iter(raw_input(),"")

This should be f = iter(raw_input,"") and this will end in a EOFError andstop
on blank line.
So you need a wrapper like :

>>def stdin_iterator() :

... while(True) :
... try : yield raw_input()
... except EOFError : return
...

>>f = stdin_iterator()

Do you really need to iterate on the file this way instead of using the
straightforward readline method ?

>>import sys
l=sys.stdin.readline()

>>while(l) :

... print l,
... l=sys.stdin.readline()


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: [phone]
Mobile: [phone]

[Matimus]

On Apr 12, 8:20 am, Maric Michaud <[e-mail]:

Le jeudi 12 avril 2007 16:25, Matimus a écrit :

# Then you check to see if your file is interactive
if f.isatty():
# This is unbuffered, and will iterate the same as f
f = iter(raw_input(),"")

This should be f = iter(raw_input,"") and this will end in a EOFError and stop
on blank line.
So you need a wrapper like :

>def stdin_iterator() :

... while(True) :
... try : yield raw_input()
... except EOFError : return
...

>f = stdin_iterator()

Do you really need to iterate on the file this way instead of using the
straightforward readline method ?

>import sys
l=sys.stdin.readline()
while(l) :

... print l,
... l=sys.stdin.readline()

--
_____________

Maric Michaud
_____________

Aristote -www.aristote.info
3 place des tapis
69004 Lyon
Tel: [phone]
Mobile: [phone]

You are correct, I looked back to a program I wrote a while ago where
I ran into this issue. Here is the actual solution I used:

if f.isatty():
f = iter(f.readline,"")

[7stud]

Hi,

Thanks for the responses. My book, Beginning Python: From Novice to
Professional(p. 266) says that "sys.stdin is iterable, just like other
files", so I thought I would test it out. However, I don't see how
it is acting similar to a file in my example.

I assume all input is buffered by default, so I'm not sure how it
explains things to say that input from sys.stdin is buffered.

I typed many lines, but lst contains only one item, as expected. Same as
your regular file example: the file contains many lines, but only the
first goes into the list.

Interesting example--not as I expected! But there is a difference in
the two examples isn't there? When you iterate over a file, the whole
file isn't put into an internal buffer first, is it?

I don't know if this a python or OS thing, but I know that iterating over a
file is not like applying successive call to readline method. You should try
to use readline instead.

I've wondered what the differences were. Thanks for bringing that
up. I searched around on google, and PEP 234 has this to say about
that:
------------------------------------
- Files implement a tp_iter slot that is equivalent to
iter(f.readline, ""). This means that we can write

for line in file:
...

as a shorthand for

for line in iter(file.readline, ""):
...

which is equivalent to, but faster than

while 1:
line = file.readline()
if not line:
break
...
....
Because the file iterator uses an internal buffer, mixing this
with other file operations (e.g. file.readline()) doesn't work
right. Also, the following code:

for line in file:
if line == "\n":
break
for line in file:
print line,

doesn't work as you might expect, because the iterator created by
the second for-loop doesn't take the buffer read-ahead by the
first for-loop into account. A correct way to write this is:

it = iter(file)
for line in it:
if line == "\n":
break
for line in
it:
print line,
--------------------------------------------

You may want to look at a related issue:
http://www.python.org/sf/1633941

Bad link.

>This should be f = iter(raw_input,"") and this will end in a EOFError
and stop on blank line. So you need a wrapper

Why a wrapper? This example seems to work without error:

lst = []

f = iter(raw_input, "")
for line in f:
lst.append(line)

print lst

[Michael Hoffman]

7stud wrote:

I assume all input is buffered by default, so I'm not sure how it
explains things to say that input from sys.stdin is buffered.

The difference with sys.stdin is that it has indeterminate length until
you signal EOF. I believe you'd get the same problem reading from, say,
a named pipe.

>I typed many lines, but lst contains only one item, as expected. Same as
your regular file example: the file contains many lines, but only the
first goes into the list.

Interesting example--not as I expected! But there is a difference in
the two examples isn't there? When you iterate over a file, the whole
file isn't put into an internal buffer first, is it?

It is if the file is smaller than the buffer size.

>This should be f = iter(raw_input,"") and this will end in a EOFError
and stop on blank line. So you need a wrapper

Why a wrapper?

Because without a wrapper you'll get EOFError, while the file iterator
would ordinarily give you StopIteration.
--
Michael Hoffman

[7stud]

On Apr 13, 3:13 am, Michael Hoffman <[e-mail]:

7stud wrote:

I assume all input is buffered by default, so I'm not sure how it
explains things to say that input from sys.stdin is buffered.

The difference with sys.stdin is that it has indeterminate length until
you signal EOF. I believe you'd get the same problem reading from, say,
a named pipe.

Couldn't you say the same thing about a file you are iterating over?

I typed many lines, but lst contains only one item, as expected. Same as
your regular file example: the file contains many lines, but only the
first goes into the list.

Interesting example--not as I expected! But there is a difference in
the two examples isn't there? When you iterate over a file, the whole
file isn't put into an internal buffer first, is it?

It is if the file is smaller than the buffer size.

How is that relevant?

This should be f = iter(raw_input,"") and this will end in a EOFError
and stop on blank line. So you need a wrapper

Why a wrapper?

Because without a wrapper you'll get EOFError, while the file iterator
would ordinarily give you StopIteration.

Did you run my example? Did you get an error? I don't get an error.

[7stud]

On Apr 13, 3:36 am, "7stud" <[e-mail]:

>

It is if the file is smaller than the buffer size.

How is that relevant?

If I put 100 lines of text in a file with each line having 50
characters, and I run this code:

import sys

lst = []
for line in open("aaa.txt"):
print "an iteration"
lst.append(line)
break

print lst
The output is:

$ python test1.py

an iteration
['helleo haljdfladj ahdflasdjf ds hdljfalsdjfdsljfds \n']

It seems clear to me that the whole file wasn't first read into a
buffer before the code started processing the data.

[Marc 'BlackJack' Rintsch]

In <11**********************@p77g2000hsh.googlegroups .com>, 7stud wrote:

On Apr 13, 3:36 am, "7stud" <[e-mail]:

>>

It is if the file is smaller than the buffer size.

How is that relevant?

If I put 100 lines of text in a file with each line having 50
characters, and I run this code:

import sys

lst = []
for line in open("aaa.txt"):
print "an iteration"
lst.append(line)
break

print lst
The output is:

$ python test1.py

an iteration
['helleo haljdfladj ahdflasdjf ds hdljfalsdjfdsljfds \n']

It seems clear to me that the whole file wasn't first read into a
buffer before the code started processing the data.

How is that clear to you? Why do you think the file was not loaded
completely into a buffer and only the first line from this buffer ends up
in `lst`?

Let's try this:

def main():
test_file = open('test.txt', 'w')
for i in xrange(100):
test_file.write('%d %s\n' % (i, 'x' * 50))
test_file.close()

test_file = open('test.txt', 'r')
for line in test_file:
print line
break
print '>%s<' % test_file.read(75)
test_file.close()

Output:
0 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

><

You see the read after the loop doesn't read anything because the file
content is in the buffer of the iterator. If the test file has more
lines, I tested it with 1000, the output looks like this:

0 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

>xx

151 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
152 xxxxxxxxxxxxx<

Ciao,
Marc 'BlackJack' Rintsch

[Michael Bentley]

On Apr 13, 2007, at 4:47 AM, 7stud wrote:

On Apr 13, 3:36 am, "7stud" <[e-mail]:

>>

>>It is if the file is smaller than the buffer size.

How is that relevant?

If I put 100 lines of text in a file with each line having 50
characters, and I run this code:

import sys

lst = []
for line in open("aaa.txt"):
print "an iteration"
lst.append(line)
break

print lst
The output is:

$ python test1.py

an iteration
['helleo haljdfladj ahdflasdjf ds hdljfalsdjfdsljfds \n']

It seems clear to me that the whole file wasn't first read into a
buffer before the code started processing the data.

The break statement causes it to bail after the first iteration, so
that doesn't really prove your point. For example:

lst = []
for line in ['we', 'all live', 'in a yellow', 'submarine']:
print "an iteration"
lst.append(line)
break

print lst

The output is:

an iteration
['we']

[Michael Hoffman]

7stud wrote:

On Apr 13, 3:13 am, Michael Hoffman <[e-mail]:

>7stud wrote:

>>I assume all input is buffered by default, so I'm not sure how it
explains things to say that input from sys.stdin is buffered.

The difference with sys.stdin is that it has indeterminate length until
you signal EOF. I believe you'd get the same problem reading from, say,
a named pipe.

Couldn't you say the same thing about a file you are iterating over?

Only if the file has indeterminate length. Regular files have a length.

>>>This should be f = iter(raw_input,"") and this will end in a EOFError
and stop on blank line. So you need a wrapper
Why a wrapper?

Because without a wrapper you'll get EOFError, while the file iterator
would ordinarily give you StopIteration.

Did you run my example? Did you get an error? I don't get an error.

Yes I did. I did get an error.

>>lst = []
f = iter(raw_input, "")
for line in f:

.... lst.append(line)
....
abc
def
<Ctrl-D>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
EOFError
--
Michael Hoffman

[7stud]

On Apr 13, 6:20 am, Michael Hoffman <[e-mail]:

7stud wrote:

On Apr 13, 3:13 am, Michael Hoffman <[e-mail]:

7stud wrote:
I assume all input is buffered by default, so I'm not sure how it
explains things to say that input from sys.stdin is buffered.
The difference with sys.stdin is that it has indeterminate length until
you signal EOF. I believe you'd get the same problem reading from, say,
a named pipe.

Couldn't you say the same thing about a file you are iterating over?

Only if the file has indeterminate length. Regular files have a length.

>>This should be f = iter(raw_input,"") and this will end in a EOFError
and stop on blank line. So you need a wrapper
Why a wrapper?
Because without a wrapper you'll get EOFError, while the file iterator
would ordinarily give you StopIteration.

Did you run my example? Did you get an error? I don't get an error.

Yes I did. I did get an error.

>>lst = []
>>f = iter(raw_input, "")
>>for line in f:

... lst.append(line)
...
abc
def
<Ctrl-D>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
EOFError
--
Michael Hoffman

But if you hit return on a blank line, there is no error. In other
words, will stop on a blank line and not return EOFError.

Anyway, it seems everyone is saying that when you iterate over a file,
the whole file is first read into memory. Therefore iterating over
sys.stdin is consistent: you have to type Ctrl+D to signal EOF before
the iteration can start. Is that about right?

[Marc 'BlackJack' Rintsch]

In <11**********************@n59g2000hsh.googlegroups .com>, 7stud wrote:

Anyway, it seems everyone is saying that when you iterate over a file, the
whole file is first read into memory. Therefore iterating over sys.stdin
is consistent: you have to type Ctrl+D to signal EOF before the iteration
can start. Is that about right?

Not the whole file is read but a block of it. If you type in enough to
fill the buffer the iteration can start without an EOF on `sys.stdin`.

See the second example in <pa****************************@gmx.netwhich
demonstrates that not the whole file is read at once by the iterator.

Ciao,
Marc 'BlackJack' Rintsch

[Steve Holden]

7stud wrote:

On Apr 13, 6:20 am, Michael Hoffman <[e-mail]:

[...]

>
But if you hit return on a blank line, there is no error. In other
words, will stop on a blank line and not return EOFError.

Anyway, it seems everyone is saying that when you iterate over a file,
the whole file is first read into memory. Therefore iterating over
sys.stdin is consistent: you have to type Ctrl+D to signal EOF before
the iteration can start. Is that about right?

No. The file content is usually buffered, but the buffering doesn't
necessarily include the whole content of the file.

If you are iterating over the file the correct way to access the next
line is to call the file's .next() method, as I indicated before.

If you are reading lines the appropriate way is to use readline().

And, as you have already seen an error message telling you, mixing the
two types is unlikely to give you usable results.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

[7stud]

On Apr 14, 7:43 am, Steve Holden <[e-mail]:

7stud wrote:

On Apr 13, 6:20 am, Michael Hoffman <[e-mail]:

[...]

But if you hit return on a blank line, there is no error. In other
words, will stop on a blank line and not return EOFError.

Anyway, it seems everyone is saying that when you iterate over a file,
the whole file is first read into memory. Therefore iterating over
sys.stdin is consistent: you have to type Ctrl+D to signal EOF before
the iteration can start. Is that about right?

No. The file content is usually buffered, but the buffering doesn't
necessarily include the whole content of the file.

If you are iterating over the file the correct way to access the next
line is to call the file's .next() method, as I indicated before.

If you are reading lines the appropriate way is to use readline().

And, as you have already seen an error message telling you, mixing the
two types is unlikely to give you usable results.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

Does iterating over stdin work the same way? If one were to type in
enough lines to fill the buffer would iteration begin before entering
EOF with Ctrl+D?

[Steve Holden]

7stud wrote:

On Apr 14, 7:43 am, Steve Holden <[e-mail]:

>7stud wrote:

>>On Apr 13, 6:20 am, Michael Hoffman <[e-mail]:

[...]

>>But if you hit return on a blank line, there is no error. In other
words, will stop on a blank line and not return EOFError.
Anyway, it seems everyone is saying that when you iterate over a file,
the whole file is first read into memory. Therefore iterating over
sys.stdin is consistent: you have to type Ctrl+D to signal EOF before
the iteration can start. Is that about right?

No. The file content is usually buffered, but the buffering doesn't
necessarily include the whole content of the file.

If you are iterating over the file the correct way to access the next
line is to call the file's .next() method, as I indicated before.

If you are reading lines the appropriate way is to use readline().

And, as you have already seen an error message telling you, mixing the
two types is unlikely to give you usable results.

Does iterating over stdin work the same way? If one were to type in
enough lines to fill the buffer would iteration begin before entering
EOF with Ctrl+D?

Why don't you try it and tell me? That's what the interactive
interpreter is for.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

[7stud]

On Apr 15, 10:59 am, Steve Holden <[e-mail]:

7stud wrote:

On Apr 14, 7:43 am, Steve Holden <[e-mail]:

7stud wrote:
On Apr 13, 6:20 am, Michael Hoffman <[e-mail]:
[...]

>But if you hit return on a blank line, there is no error. In other
words, will stop on a blank line and not return EOFError.
Anyway, it seems everyone is saying that when you iterate over a file,
the whole file is first read into memory. Therefore iterating over
sys.stdin is consistent: you have to type Ctrl+D to signal EOF before
the iteration can start. Is that about right?
No. The file content is usually buffered, but the buffering doesn't
necessarily include the whole content of the file.

If you are iterating over the file the correct way to access the next
line is to call the file's .next() method, as I indicated before.

If you are reading lines the appropriate way is to use readline().

And, as you have already seen an error message telling you, mixing the
two types is unlikely to give you usable results.

Does iterating over stdin work the same way? If one were to type in
enough lines to fill the buffer would iteration begin before entering
EOF with Ctrl+D?

Why don't you try it and tell me? That's what the interactive
interpreter is for.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

I just typed in 700 lines of text, and the iteration hasn't begun
yet. Should I keep going?

[Gabriel Genellina]

En Sun, 15 Apr 2007 16:51:12 -0300, 7stud <bb**********@yahoo.com>
escribió:

I just typed in 700 lines of text, and the iteration hasn't begun
yet. Should I keep going?

How much text each line? Python uses an 8K buffer.

--
Gabriel Genellina

[Diez B. Roggisch]

>Steve Holden [phone] [phone]

>Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

I just typed in 700 lines of text, and the iteration hasn't begun
yet. Should I keep going?

Well, I'm not sure how much of a programmer you are if you don't get the
idea of writing a _program_ to do the typing....

-------- a.py -----------
import sys

counter = 0
try:
while True:
s = str(counter) + '\n'
counter += len(s)
sys.stdout.write(s)
except IOError:
sys.stderr.write(s)
------- b.py ------------

import sys

lst = []
for line in sys.stdin:
lst.append(line)
break

print lst

-----------------

Besides that, a little bit of logic thinking should make clear that
having to wait till EOF won't work too much for piping programs that
produce continously output with possibly terabytes of data. Even under
modern memory constraints...
Diez