Graceful detection of EOF

[MickeyBob]

How does one detect the EOF gracefully? Assuming I have a pickle file
containing an unknown number of objects, how can I read (i.e.,
pickle.load()) until the EOF is encountered without generating an EOF
exception?

Thanks for any assistance.
MickeyBob

[Josiah Carlson]

lines = iter("abc")
first = lines.next()
print first a for line in lines:

... print line
...
b
c

Would hurt less feeling I presume.

Unless it was empty, then you'd get the dreaded StopIteration!

IMO, unconditionally breaking out of a for loop is the nicer way of
handling things in this case, no exceptions to catch.

- Josiah

[Mel Wilson]

In article <ma************ *************** ***********@pyt hon.org>,
Egbert Bouwman <eg*********@hc cnet.nl> wrote:

A file is too large to fit into memory.
The first line must receive a special treatment, because
it contains information about how to handle the rest of the file.

Of course it is not difficult to test if you are reading the first line
or another one, but it hurts my feelings to do a test which by definition
succeeds at the first record, and never afterwards.
Any suggestions ?

f = file("lines.txt ", "rt")
first_line_proc essing (f.readline())
for line in f:
line_processing (line)

ought to work.

Regards. Mel.

[Alex Martelli]

Peter Otten <__*******@web. de> wrote:
...

Looking at it from another angle, the initial for-loop ist just a peculiar
way to deal with an empty iterable. So the best (i. e. clear, robust and
general) approach is probably

items = iter(...)
try:
first = items.next()
except StopIteration:
# deal with empty iterator, e. g.:
raise ValueError("nee d at least one item")
else:
# process remaining data

I think it can't be optimal, as coded, because it's more nested than it
needs to be (and "flat is better than nested"): since the exception
handler doesn't fall through, I would omit the try statement's else
clause and outdent the "process remaining data" part. The else clause
would be needed if the except clause could fall through, though.
Alex

[Steven Bethard]

Josiah Carlson <jcarlson <at> uci.edu> writes:

IMO, unconditionally breaking out of a for loop is the nicer way of
handling things in this case, no exceptions to catch.

There's still a NameError to catch if you haven't initialized line:

for line in []: .... break
.... line

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'line' is not defined

I don't much like the break out of a for loop, because it feels like a misuse
of a construct designed for iteration... But take your pick: StopIteration or
NameError. =)

Steve

[Peter Otten]

Steven Bethard wrote:

There's still a NameError to catch if you haven't initialized line:

for line in []: ... break
... line

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'line' is not defined

No, you would put code specific to the first line into the loop before the
break.
I don't much like the break out of a for loop, because it feels like a
misuse

I can understand that.

Peter

[Peter Otten]

Alex Martelli wrote:

Peter Otten <__*******@web. de> wrote:
...

Looking at it from another angle, the initial for-loop ist just a
peculiar way to deal with an empty iterable. So the best (i. e. clear,
robust and general) approach is probably

items = iter(...)
try:
first = items.next()
except StopIteration:
# deal with empty iterator, e. g.:
raise ValueError("nee d at least one item")
else:
# process remaining data

I think it can't be optimal, as coded, because it's more nested than it
needs to be (and "flat is better than nested"): since the exception
handler doesn't fall through, I would omit the try statement's else
clause and outdent the "process remaining data" part. The else clause
would be needed if the except clause could fall through, though.

I relied more on the two letters 'e. g.' than I should have as there are two
different aspects I wanted to convey:

1. Don't let the StopIteration propagate:

items = iter(...)
try:
first = items.next()
except StopIteration:
raise MeaningfulExcep tion("clear indication of what caused the error")

2. General structure when handling the first item specially:

items = iter(...)
try:
first = items.next()
except StopIteration:
# handle error
else:
# a. code relying on 'first'
# b. code independent of 'first' or relying on the error handler
# defining a proper default.

where both (a) and (b) are optional.

As we have now two variants, I have to drop the claim to generality.
Regarding the Zen incantation, "flat is better than nested", I tend measure
nesting as max(indent level) rather than avg(), i. e. following my (perhaps
odd) notion the else clause would affect nesting only if it contained an
additional if, for, etc. Therefore I have no qualms to sometimes use else
where it doesn't affect control flow:

def whosAfraidOf(co lor):
if color == red:
return peopleAfraidOfR ed
else:
# if it ain't red it must be yellow - nobody's afraid of blue
return peopleAfraidOfY ellow

as opposed to

def whosAfraidOf(co lor):
if color == red:
return peopleAfraidOfR ed
return peopleAfraidOfA nyOtherColor

That said, usually my programs have bigger problems than the above subtlety.

Peter

[Egbert Bouwman]

On Fri, Oct 08, 2004 at 11:59:32AM +0200, Alex Martelli wrote:

option 3, a bit cutesy:

for first_line in thefile: break
for line in thefile: ...

(again, in 2.2 you'll need some foo=iter(thefil e)).
This technique depends in the file being positioned at line 2,
after the break.

However, In the Nutshell book, page 191, you write: Interrupting such a loop prematurely (e.g. with break)
leaves the file's current position with an arbitrary value.

So the information about the current position is useless.

Do I discover a contradiction ?
egbert
--
Egbert Bouwman - Keizersgracht 197 II - 1016 DS Amsterdam - 020 6257991
=============== =============== =============== =============== ============

[Alex Martelli]

Egbert Bouwman <eg*********@hc cnet.nl> wrote:

On Fri, Oct 08, 2004 at 11:59:32AM +0200, Alex Martelli wrote:

option 3, a bit cutesy:

for first_line in thefile: break
for line in thefile: ...

(again, in 2.2 you'll need some foo=iter(thefil e)).
This technique depends in the file being positioned at line 2,
after the break.

Not exactly, if by "being positioned" you mean what's normally meant for
file objects (what will thefile.tell() respond, what next five bytes
will thefile.read(5) read, and so on). All it depends on is the
_iterator_ on the file being "positioned " in the sense in which
iterators are positioned (what item will come if you call next on the
iterator).

In 2.3 a file is-an iterator; in 2.2 you need to explicitly get an
iterator as indicated in the parenthesis you've also quoted.

However, In the Nutshell book, page 191, you write:

Interrupting such a loop prematurely (e.g. with break)
leaves the file's current position with an arbitrary value.

So the information about the current position is useless.

Do I discover a contradiction ?

Nope -- the file's current position is (e.g.) what tell will respond if
you call it, and that IS arbitrary. In 2.2 (which is what the Nutshell
covers) you need to explicitly get an iterator to do anything else; in
2.3 you can rely on the fact that a file is its own iterator to make
your code simpler. But the iteration state is not connected with the
file's current position.
Alex

[Egbert Bouwman]

On Sun, Oct 10, 2004 at 12:41:37AM +0200, Alex Martelli wrote:

....
But the iteration state is not connected with the
file's current position.

That is very useful information.Tha nks.
egbert
--
Egbert Bouwman - Keizersgracht 197 II - 1016 DS Amsterdam - 020 6257991
=============== =============== =============== =============== ============

[Alex Martelli]

Steven Bethard <st************ @gmail.com> wrote:
...

I don't much like the break out of a for loop, because it feels like a misuse
of a construct designed for iteration... But take your pick: StopIteration or
NameError. =)

Jacopini and Bohm have much to answer for...;-)

Alex