472,783 Members | 954 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,783 software developers and data experts.

zipfile library - problem..

I'm working to create a backup function for a software; in particular,
from a directory with 12300 files (Jpg, medium size = 250KB / total size
= 2.90GB), i have to create a zip file.

I've decided to use the standard "zipfile" library to do that.
This is the code:

zipName = path.join(config.get('server.xbakPath'), 'backup.zip')
fileExport = zipfile.ZipFile(zipName,'w',zipfile.ZIP_DEFLATED)

#Insert images
listaImg = listdir( config.get('server.iPath') )
for file in listaImg:
if not path.isdir(path.join(config.get('server.iPath'), file)):
fileExport.write(path.join(config.get('server.iPat h'), file), \

#close ZIP

Unfortunately, after few minutes, i see this error message:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\cherrypy\_cphttptools.py", line
256, in run
File "C:\Python24\Lib\site-packages\cherrypy\_cphttptools.py", line
452, in main
body = func(*(virtualPathList + cherrypy.request.paramList),
File "C:\pagine\utilities.py", line 162, in exportdati
File "C:\Python24\lib\zipfile.py", line 503, in close
OverflowError: long int too large to convert to int

Reading the error I've known that the error is generated calling
"fileExport.close()", that calls the zipfile module,
and into that module there is an exception.

For information: when code stops, the zip file has size = 3.177.950.237
byte, but this size doesn't exceed ZIP limits:

the limits of ZIP files according to the folks who make info-zip:
http://www.info-zip.org/pub/in fozip/FAQ.html#limits

statistic limit
number of files 65,536
uncompressed size of a single file 4 GB
compressed size of a single file 4 GB
total size of archive 256 TB
maximum path/filename length 64 KB

Have you any idea?
Oh, the O.S. is WinXP Pro

Thank you!.
Aug 11 '05 #1
2 1503
Renzo ha scritto:
<cut all>
I've seen that the error is a known bug:

"zipfile still has 2GB boundary bug"

so I've used the module TarFile instead ZipFile, and now the script
works perfectly.

Thanks to Deelan and Manlio Perillo (i.c.l.p.)

Aug 11 '05 #2
Renzo wrote:
Renzo ha scritto:
<cut all>
I've seen that the error is a known bug:

"zipfile still has 2GB boundary bug"

so I've used the module TarFile instead ZipFile, and now the script
works perfectly.

Thanks; good to know. Looks like people are working on this
Python library bug. Next after that, we'll need to fix the zip
format. The 4GB limit on contained files and 256TB limit on
archives both looked astronomical at the time they were defined.
Sorry to sound like an old guy, but wow, how times change.

Incidentally, since your files are jpegs, they're already
compressed, and thus TarFile should be just as good as ZipFile
Aug 11 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: LC | last post by:
Hi, I'm having a problem using the zipfile module in Windows 2000 sp4. When I use it to zip a small file it works fine, but large file doesnt. Here's the error msg i get......
by: Gerson Kurz | last post by:
AAAAAAAARG I hate the way python handles unicode. Here is a nice problem for y'all to enjoy: say you have a variable thats unicode directory = u"c:\temp" Its unicode not because you want it...
by: Bennie | last post by:
Hi, I have a problem with ZipFile. It works okay untily I come across a file that is greater then 1Gb. Then it exit with the error: OverflowError: long int too large to convert to int How...
by: Hari Sekhon | last post by:
I do import zipfile zip=zipfile.ZipFile('d:\somepath\cdimage.zip') zip.namelist() then either of the two: A) file('someimage.iso','w').write(zip.read('someimage.iso'))
by: Martin Maney | last post by:
To quote from zipfile.py (2.4 library): # Search the last END_BLOCK bytes of the file for the record signature. # The comment is appended to the ZIP file and has a 16 bit length. # So the...
by: xamdam | last post by:
Hi fellas, I am experiencing problems reading a 2GB zipfile consisting of multiple zipped files. I found a thread http://mail.python.org/pipermail/python-dev/2005-April/053027.html that mentions...
by: towers | last post by:
Hi I'm probably doing something stupid but I've run into a problem whereby I'm trying to add a csv file to a zip archive - see example code below. The csv just has several rows with carriage...
by: Neil Crighton | last post by:
I'm using the zipfile library to read a zip file in Windows, and it seems to be adding too many newlines to extracted files. I've found that for extracted text-encoded files, removing all instances...
by: John Machin | last post by:
On Jun 4, 8:06 pm, jwesonga <crazylun...@gmail.comwrote: Nothing is ever as it seems. Let's try to work backwards from the error message ... and we don't need your magnificent script, just the...
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 2 August 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
by: erikbower65 | last post by:
Here's a concise step-by-step guide for manually installing IntelliJ IDEA: 1. Download: Visit the official JetBrains website and download the IntelliJ IDEA Community or Ultimate edition based on...
by: kcodez | last post by:
As a H5 game development enthusiast, I recently wrote a very interesting little game - Toy Claw ((http://claw.kjeek.com/))。Here I will summarize and share the development experience here, and hope it...
by: Rina0 | last post by:
I am looking for a Python code to find the longest common subsequence of two strings. I found this blog post that describes the length of longest common subsequence problem and provides a solution in...
by: DJRhino | last post by:
Private Sub CboDrawingID_BeforeUpdate(Cancel As Integer) If = 310029923 Or 310030138 Or 310030152 Or 310030346 Or 310030348 Or _ 310030356 Or 310030359 Or 310030362 Or...
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
by: lllomh | last post by:
How does React native implement an English player?
by: Mushico | last post by:
How to calculate date of retirement from date of birth
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.