while c = f.read(1)

[Greg McIntyre]

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.

This is what I would ideally like:

f = open("blah.txt" , "r")
while c = f.read(1):
# ... work on c

But I get a syntax error.

while c = f.read(1):
^
SyntaxError: invalid syntax

And read() doesn't work that way anyway because it returns '' on EOF
and '' != False. If I try:

f = open("blah.txt" , "r")
while (c = f.read(1)) != '':
# ... work on c

I get a syntax error also. :(

Is this related to Python's expression vs. statement syntactic
separation? How can I be write this code more nicely?

Thanks

[Bengt Richter]

On 18 Aug 2005 22:21:53 -0700, "Greg McIntyre" <gr**@puyo.cjb. net> wrote:

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.

This is what I would ideally like:

f = open("blah.txt" , "r")
while c = f.read(1):
# ... work on c
How about (untested):

for c in iter((lambda f=open('blah.tx t', 'r'): f.read(1)), ''):
# ... work on c

("if c=='': break" functionality courtesy of iter(f, sentinel) form above)

Of course, reading characters one by one is not very efficient, so if the file
is reasonably sized, you might just want to read the whole thing and iterate
through it, something like

for c in open('blah.txt' ).read():
# ... work on c
But I get a syntax error.

while c = f.read(1):
^
SyntaxError: invalid syntax

And read() doesn't work that way anyway because it returns '' on EOF
and '' != False. If I try:

f = open("blah.txt" , "r")
while (c = f.read(1)) != '':
# ... work on c

I get a syntax error also. :(

Is this related to Python's expression vs. statement syntactic
separation? How can I be write this code more nicely?

Yes, it is related as you suspect. I'll leave it to you to make
a chunk-buffering one-liner for huge files that iterates by characters,
if one-liners turn you on. Otherwise it is easy to write a generator that will do it.
Byt the time I post this, someone will probably have done it ;-)
Regards,
Bengt Richter

[Robert Kern]

Greg McIntyre wrote:

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.

That's not spaghetti. Not even close.

In any case, is there a reason you are reading one character at a time
instead of reading the contents of the file into memory and iterating
over the resulting string?

f = open('blah.txt' , 'r')
text = f.read()
f.close()

for c in f:
# ...

If you must read one character at a time,

def reader(fileobj, blocksize=1):
"""Return an iterator that reads blocks of a given size from a
file object until EOF.
"""
# Note that iter() can take a function to call repeatedly until it
# receives a given sentinel value, here ''.
return iter(lambda: fileobj.read(bl ocksize), '')

f = open('blah.txt' , 'r')
try:
for c in reader(f):
# ...
finally:
f.close()

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Donn Cave]

Quoth "Greg McIntyre" <gr**@puyo.cjb. net>:
| I have a Python snippet:
|
| f = open("blah.txt" , "r")
| while True:
| c = f.read(1)
| if c == '': break # EOF
| # ... work on c
|
| Is some way to make this code more compact and simple? It's a bit
| spaghetti.

Actually I'd make it a little less compact -- put the "break"
on its own line -- but in any case this is fine. It's a natural
and ordinary way to express this in Python.

....
| But I get a syntax error.
|
| while c = f.read(1):
| ^
| SyntaxError: invalid syntax
|
| And read() doesn't work that way anyway because it returns '' on EOF
| and '' != False. If I try:

This is the part I really wanted to respond to. Python managed
without a False for years (and of course without a True), and if
the introduction of this superfluous boolean type really has led
to much of this kind of confusion, then it was a bad idea for sure.

The condition that we're looking at here, and this is often the
way to look at conditional expressions in Python, is basically
something vs. nothing. In this and most IO reads, the return
value will be something, until at end of file it's nothing.
Any type of nothing -- '', {}, [], 0, None - will test "false",
and everything else is "true". Of course True is true too, and
False is false, but as far as I know they're never really needed.

You are no doubt wondering when I'm going to get to the part where
you can exploit this to save you those 3 lines of code. Sorry,
it won't help with that.

| Is this related to Python's expression vs. statement syntactic
| separation? How can I be write this code more nicely?

Yes, exactly. Don't worry, it's nice as can be. If this is
the worst problem in your code, you're far better off than most
of us.

Donn Cave, do**@drizzle.co m

[John Machin]

Greg McIntyre wrote:

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
That could read like this
if not c: break # EOF
# see below for comments on what is true/false
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.
Not at all, IMHO. This is a simple forward-branching exit from a loop in
explicable circumstances (EOF). It is a common-enough idiom that doesn't
detract from readability & understandabili ty. Spaghetti is like a GOTO
that jumps backwards into the middle of a loop for no discernable reason.

This is what I would ideally like:

f = open("blah.txt" , "r")
while c = f.read(1):
# ... work on c

But I get a syntax error.

while c = f.read(1):
^
SyntaxError: invalid syntax

And read() doesn't work that way anyway because it returns '' on EOF
and '' != False. >
You have a bit of a misunderstandin g here that needs correcting:

In "if <blah>" and "while <blah>", <blah> is NOT restricted to being in
(True, False). See section 5.10 of the Python Reference Manual:

"""
In the context of Boolean operations, and also when expressions are used
by control flow statements, the following values are interpreted as
false: None, numeric zero of all types, empty sequences (strings, tuples
and lists), and empty mappings (dictionaries). All other values are
interpreted as true.
"""

.... AND it's about time that list is updated to include False explicitly
-- save nitpicking arguments about whether False is covered by
"numeric zero of all types" :-)
If I try:

f = open("blah.txt" , "r")
while (c = f.read(1)) != '':
# ... work on c

I get a syntax error also. :(

Is this related to Python's expression vs. statement syntactic
separation? How can I be write this code more nicely?

Thanks

How about
for c in f.read():
?
Note that this reads the whole file into memory (changing \r\n to \n on
Windows) ... performance-wise for large files you've spent some memory
but clawed back the rather large CPU time spent doing f.read(1) once per
character. The "more nicely" factor improves outasight, IMHO.

Mild curiosity: what are you doing processing one character at a time
that can't be done with a built-in function, a standard module, or a
3rd-party module?

[John Machin]

Bengt Richter wrote:

On 18 Aug 2005 22:21:53 -0700, "Greg McIntyre" <gr**@puyo.cjb. net> wrote:

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.

This is what I would ideally like:

f = open("blah.txt" , "r")
while c = f.read(1):
# ... work on c

How about (untested):

for c in iter((lambda f=open('blah.tx t', 'r'): f.read(1)), ''):
# ... work on c

:-)
Bengt, did you read on to the bit where the OP wanted to do it "more
nicely"? YMMV, but I think you've strayed into "pas devant les enfants"
territory.
(-:

Cheers,
John

On 18 Aug 2005 22:21:53 -0700
Greg McIntyre wrote:

I have a Python snippet:

f = open("blah.txt" , "r")
while True:
c = f.read(1)
if c == '': break # EOF
# ... work on c

Is some way to make this code more compact and simple? It's a bit
spaghetti.

import itertools
f = open("blah.txt" , "r")
for c in itertools.chain (*f):
print c
# ...

The "f" is iterable itself, yielding a new line from the file every time.
Lines are iterable as well, so the itertools.chain iterates through each
line and yields a character.

--
jk

[Paul Rubin]

<en**********@o spaz.ru> writes:

import itertools
f = open("blah.txt" , "r")
for c in itertools.chain (*f):
print c
# ...

The "f" is iterable itself, yielding a new line from the file every time.
Lines are iterable as well, so the itertools.chain iterates through each
line and yields a character.

But that can burn an unlimited amount of memory if there are long
stretches of the file with no newlines. There's no real good way
around ugly code.

[Robert Kern]

en**********@os paz.ru wrote:

import itertools
f = open("blah.txt" , "r")
for c in itertools.chain (*f):
print c
# ...

The "f" is iterable itself, yielding a new line from the file every time.
Lines are iterable as well, so the itertools.chain iterates through each
line and yields a character.

As far as I can tell, that code is just going to read the whole file in
when Python does the *arg expansion. What's the point?

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

On 19 Aug 2005 03:43:31 -0700
Paul Rubin wrote:

<en**********@o spaz.ru> writes:

import itertools
f = open("blah.txt" , "r")
for c in itertools.chain (*f):

But that can burn an unlimited amount of memory if there are long
stretches of the file with no newlines. There's no real good way
around ugly code.

I agree. Moreover, in fact, it is the same as just

for c in f.read():
# ...

--
jk