get quite impressive compression. However, uncompressed, they're
large enough to thrash my memory to swap and in general do bad
performance-related things. I'm trying to figure out how to
produce a file-like iterator out of the contents of such an item.
132987864>>z = zipfile.zipFile ("test.zip")
info = z.getinfo("data .txt")
info.compress _size
1344250972>>info.file_siz e
20>>len(z.namelis t())
I need to be able to access multiple files within it, but I can
iterate over each one, only seeing small slices of the file.
Using the read() method triggers the volumnous read. Thus what I
have to do currently:
1344250972>>content = z.read("data.tx t") # ouch!
len(content )
What I'm trying to figure out how to do is something like the>>for row in content.splitli nes(): process(row) # pain!
mythical:
to more efficiently handle the huge stream of data.>>for row in z.file_iter("da ta.txt"): process(row) # aah
Am I missing something obvious? It seems like iterating over zip
contents would be a common thing to do (especially when compared
to reading the whole contents...I mean, they're zipped because
they're big! :)
Thanks for any pointers,
-tkc