473,289 Members | 2,091 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,289 software developers and data experts.

csv iterator question

When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

example of csv.reader method:
reader = csv.reader(open(csvfilename))
here are my results:
>>reader = csv.reader(open('export.csv'))
for line in reader:
.... print line
<lots of data on page...>

but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?
>>for line in reader:
.... print line
....

do i need to seek() the beginning of this file object ? any help is
greatly appreciated here.
Jun 27 '08 #1
4 1276
On May 23, 3:36*pm, davidj411 <davidj...@gmail.comwrote:
When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

example of csv.reader method:
reader = csv.reader(open(csvfilename))

here are my results:>>reader = csv.reader(open('export.csv'))
>for line in reader:

... *print line

<lots of data on page...>

but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?
>for line in reader:

... *print line
...

do i need to seek() the beginning of this file object ? any help is
greatly appreciated here.
I think using seek() is what you need as it is currently starting on
the last line that it read, so to speak. You should have just tried
it...experimenting is part of the fun of Python, after all.

Mike
Jun 27 '08 #2
On Fri, 23 May 2008 13:36:55 -0700, davidj411 wrote:
example of open method on file object:
fhandle=open(filename).readelines()
Here the name `fhandle` is somewhat misleading. You don't bind the file
object to that name, but the result of the call of the `readlines()`
method. Which is a list of lines. If you would have bound the file
object, you'd observe similar behavior to your CSV reader example.
example of csv.reader method:
reader = csv.reader(open(csvfilename))
Because here you are binding the reader object. Throw in a `list()` to
read all CSV records into a list:

records = list(csv.reader(open(csv_filename)))

In both cases you don't close the file explicitly which might cause
problems. That CPython closes it for you almost immediately is an
implementation detail. Other implementations might let those open files
stay open for much longer.

Ciao,
Marc 'BlackJack' Rintsch
Jun 27 '08 #3
davidj411 wrote:
>When you save an open file to a variable, you can re-use that variable
for membership checking.
it does not seem to be that way with the csv.reader function, even
when set to a variable name.

what is the best way to store the open CSV file in memory or do i need
to open the file each time?

example of open method on file object:
fhandle=open(filename).readelines()

>>fhandle = open('c:/temp/08-02024.csv').readlines()
type(fhandle)
<type 'list'>
>>len(fhandle)
381

fhandle is a list containing the contents of the file, not a file handle
-- that's why you can easily re-use it.
>example of csv.reader method:
reader = csv.reader(open(csvfilename))

Try this instead:
>>>reader = csv.reader(open('c:/temp/08-02024.csv'))
contents = [line for line in reader]
type(contents)
<type 'list'>
>>len(contents)
381

You still get a list that you can easily use, that has gone through the
csv routines.

Hope this helps.
--
Ethan

Jun 27 '08 #4
On May 24, 6:36 am, davidj411 <davidj...@gmail.comwrote:
but when i try to iterate through it again, it appears to print
nothing (no error either). the file is exhausted?
No, the iterator is finished. Iterators are generally use-once:

"The intention of the protocol is that once an iterator's next()
method raises StopIteration, it will continue to do so on subsequent
calls. Implementations that do not obey this property are deemed
broken." [http://docs.python.org/lib/typeiter.html]

As well as reading the iterator into a list, you can also create a
copy of the iterator:
>>import itertools
reader1, reader2 = itertools.tee(csv.reader(open('export.csv')))
Two copies is the default, but you can optionally specify as many as
you like:
>>m, n, o = itertools.tee(iterator, 3)
- alex23


Jun 27 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

38
by: Grant Edwards | last post by:
In an interview at http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=273 Alan Kay said something I really liked, and I think it applies equally well to Python as well as the languages...
26
by: Michael Klatt | last post by:
I am trying to write an iterator for a std::set that allows the iterator target to be modified. Here is some relvant code: template <class Set> // Set is an instance of std::set<> class...
5
by: music4 | last post by:
Greetings, I want to STL map class. But I don't know how to use iterator. Let me state my question by following small sample: map<int, int> mymap; // insert some <key,value> in mymap ...
11
by: Mateusz Loskot | last post by:
Hi, I have a simple question about naming convention, recommeded names or so in following case. I have a class which is implemented with one of STL container (aggregated) and my aim is to...
8
by: Mateusz Łoskot | last post by:
Hi, I know iterator categories as presented by many authors: Stroustrup, Josuttis and Koenig&Moo: Input <---| |<--- Forward <--- Bidirectional <--- Random Output <---|
5
by: Mark Stijnman | last post by:
I have a question about forward iterators and what one should do or not do with them. I'm planning on writing a container that, when all boils down to it, stores a sequence of strings. I want...
3
by: wolverine | last post by:
Hi I am accessing a map from inside threads. There is a chance that an element is inserted into the map, from inside any thread. Since i don't know about thread safety of stl implementation i am...
3
by: Alex__655321 | last post by:
Hello All I hope I'm correct posting an STL question here - if not feel free to direct me to somewhere more appropriate. I'm writing some code using an std::set which I believe is the best...
16
by: Juha Nieminen | last post by:
I'm actually not sure about this one: Does the standard guarantee that if there's at least one element in the data container, then "--container.end()" will work and give an iterator to the last...
2
by: Terry Reedy | last post by:
Luis Zarrabeitia wrote: Interesting observation. Iterators are intended for 'iterate through once and discard' usages. To zip a long sequence with several short sequences, either use...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.