get quite impressive compression. However, uncompressed, they're
large enough to thrash my memory to swap and in general do bad
performance-related things. I'm trying to figure out how to
produce a file-like iterator out of the contents of such an item.
132987864>>z = zipfile.zipFile("test.zip")
info = z.getinfo("data.txt")
info.compress_size
1344250972>>info.file_size
20>>len(z.namelist())
I need to be able to access multiple files within it, but I can
iterate over each one, only seeing small slices of the file.
Using the read() method triggers the volumnous read. Thus what I
have to do currently:
1344250972>>content = z.read("data.txt") # ouch!
len(content)
What I'm trying to figure out how to do is something like the>>for row in content.splitlines(): process(row) # pain!
mythical:
to more efficiently handle the huge stream of data.>>for row in z.file_iter("data.txt"): process(row) # aah
Am I missing something obvious? It seems like iterating over zip
contents would be a common thing to do (especially when compared
to reading the whole contents...I mean, they're zipped because
they're big! :)
Thanks for any pointers,
-tkc