csv iterator question

[davidj411]

When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

example of csv.reader method:
reader = csv.reader(open(csvfilename))
here are my results:

>>reader = csv.reader(open('export.csv'))
for line in reader:

.... print line
<lots of data on page...>

but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?

>>for line in reader:

.... print line
....

do i need to seek() the beginning of this file object ? any help is
greatly appreciated here.

[Mike Driscoll]

On May 23, 3:36*pm, davidj411 <[e-mail]:

When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

example of csv.reader method:
reader = csv.reader(open(csvfilename))

here are my results:>>reader = csv.reader(open('export.csv'))

>for line in reader:

... *print line

<lots of data on page...>

but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?

>for line in reader:

... *print line
...

do i need to seek() the beginning of this file object ? any help is
greatly appreciated here.

I think using seek() is what you need as it is currently starting on
the last line that it read, so to speak. You should have just tried
it...experimenting is part of the fun of Python, after all.

Mike

[Marc 'BlackJack' Rintsch]

On Fri, 23 May 2008 13:36:55 -0700, davidj411 wrote:

example of open method on file object:
fhandle=open(filename).readelines()

Here the name `fhandle` is somewhat misleading. You don't bind the file
object to that name, but the result of the call of the `readlines()`
method. Which is a list of lines. If you would have bound the file
object, you'd observe similar behavior to your CSV reader example.

example of csv.reader method:
reader = csv.reader(open(csvfilename))

Because here you are binding the reader object. Throw in a `list()` to
read all CSV records into a list:

records = list(csv.reader(open(csv_filename)))

In both cases you don't close the file explicitly which might cause
problems. That CPython closes it for you almost immediately is an
implementation detail. Other implementations might let those open files
stay open for much longer.

Ciao,
Marc 'BlackJack' Rintsch

[Ethan Furman]

davidj411 wrote:

>When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

>>fhandle = open('c:/temp/08-02024.csv').readlines()
type(fhandle)

<type 'list'>

>>len(fhandle)

381

fhandle is a list containing the contents of the file, not a file handle
-- that's why you can easily re-use it.

>example of csv.reader method:
reader = csv.reader(open(csvfilename))

Try this instead:

>>>reader = csv.reader(open('c:/temp/08-02024.csv'))
contents = [line for line in reader]
type(contents)

<type 'list'>

>>len(contents)

381

You still get a list that you can easily use, that has gone through the
csv routines.

Hope this helps.
--
Ethan

[alex23]

On May 24, 6:36 am, davidj411 <[e-mail]:

but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?

No, the iterator is finished. Iterators are generally use-once:

"The intention of the protocol is that once an iterator's next()
method raises StopIteration, it will continue to do so on subsequent
calls. Implementations that do not obey this property are deemed
broken." [http://docs.python.org/lib/typeiter.html]

As well as reading the iterator into a list, you can also create a
copy of the iterator:

>>import itertools
reader1, reader2 = itertools.tee(csv.reader(open('export.csv')))

Two copies is the default, but you can optionally specify as many as
you like:

>>m, n, o = itertools.tee(iterator, 3)

- alex23