473,573 Members | 2,940 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

zipfile module

LC
Hi,

I'm having a problem using the zipfile module in Windows 2000 sp4. When I
use it to zip a small file it works fine, but large file doesnt. Here's the
error msg i get...
---------------------------------------------------------------------
dynamite4_db_ar chive.py", line 45, in ?
file.write(name , os.path.basenam e(name), zipfile.ZIP_DEF LATED)
File "C:\Python22\li b\zipfile.py", line 426, in write
zinfo.file_size ))
OverflowError: long int too large to convert to int
---------------------------------------------------------------------
I've read a couple of problems with this module with long int, and it says
it's a know problem in Python earlier than 2.3
So I upgraded to 2.3 and i'm still getting the error msg. I'm not proficient
at python so I'm not sure what to do at this point. The file I'm currently
trying to zip is about 2.5GB.

Here's my code that's causing the error...
----------------------------------------------------------------------
currentDirFiles = os.listdir(mydi r) #list log file in dir
os.mkdir(todays _dir)

for item in currentDirFiles : #find the file with the date from 7 days ago

file_to_zip = mydir + item
destination = todays_dir + str('/') + item + str('_') + todays_date +
str('.zip')

print "\n" + file_to_zip + "\n" + destination

file = zipfile.ZipFile (destination, "w")

for name in glob.glob(file_ to_zip):
file.write(name , os.path.basenam e(name), zipfile.ZIP_DEF LATED)

file.close()
---------------------------------------------------------------------

Any help would be appreciated. TIA

LC
Jul 18 '05 #1
1 2913

lco> dynamite4_db_ar chive.py", line 45, in ?
lco> file.write(name , os.path.basenam e(name), zipfile.ZIP_DEF LATED)
lco> File "C:\Python22\li b\zipfile.py", line 426, in write
lco> zinfo.file_size ))
lco> OverflowError: long int too large to convert to int

...

lco> for name in glob.glob(file_ to_zip):
lco> file.write(name , os.path.basenam e(name), zipfile.ZIP_DEF LATED)

How about you write the file in chunks? I'm not familiar with the zipfile
module, but it looks like writestr will do the job for you. My guess is
something like

...
file = zipfile.ZipFile (destination, "w", zipfile.ZIP_DEF LATE)
for name in glob.glob(file_ to_zip):
...
f = open(name)
data = f.read(8192)
while data:
file.writestr(o s.path.basename (name), data)
data = f.read(8192)
f.close()
file.close()

might do the trick. (The zipfile module's interface to writing data looks a
bit odd to me.)

Skip

Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
4147
by: stewart.midwinter | last post by:
I've been experimenting with the python zipfile module, and have run into a snag. I'm able to create a new zipfile with the module's ZipFile class and add files to it. After closing the file, I'm able to view the contents of it with WinZip. I can also confirm that it is a python zip file by using the is_zipfile() method. All good so far....
5
2444
by: Waguy | last post by:
Hi all, I am new to python and want to create a process to unzip large numbers of zip files I get from a SOAP application. The files all have a ZIP extention and can be unzipped using WinZip. However when I try opening the files using zlib or zipfile modules I get the following error: Traceback (most recent call last):
11
7574
by: Hari Sekhon | last post by:
I do import zipfile zip=zipfile.ZipFile('d:\somepath\cdimage.zip') zip.namelist() then either of the two: A) file('someimage.iso','w').write(zip.read('someimage.iso'))
1
1757
by: Ritesh Raj Sarraf | last post by:
Hi, I've got a problem here. def compress_the_file(zip_file_name, files_to_compress, sSourceDir): """ Condenses all the files into one single file for easy transfer """ try:
5
2116
by: OriginalBrownster | last post by:
This will probably sound like a very dumb question. I am trying to zip some files within a directory. I want to zip all the files within a directory called "temp" and have the zip archive saved in a directory with temp called ziptemp I was trying to read up on how to use the zipfile module python provides, but I cannot seem to find...
8
3934
by: =?utf-8?B?5Lq66KiA6JC95pel5piv5aSp5rav77yM5pyb5p6B | last post by:
I made a C/S network program, the client receive the zip file from the server, and read the data into a variable. how could I process the zipfile directly without saving it into file. In the document of the zipfile module, I note that it mentions the file-like object? what does it mean? class ZipFile( file]]) Open a ZIP file, where file...
3
4206
by: towers | last post by:
Hi I'm probably doing something stupid but I've run into a problem whereby I'm trying to add a csv file to a zip archive - see example code below. The csv just has several rows with carriage return line feeds (CRLF). However after adding it to an archive and then decompressing the line endings have been converted to just line feeds...
3
1475
by: Larry Bates | last post by:
I'm trying to learn about subclassing new style classes and the first project I went to do needs to subclass zipfile to add some methods. Why does this: import zipfile class walkZip(zipfile): pass
5
5141
by: Neil Crighton | last post by:
I'm using the zipfile library to read a zip file in Windows, and it seems to be adding too many newlines to extracted files. I've found that for extracted text-encoded files, removing all instances of '\r' in the extracted file seems to fix the problem, but I can't find an easy solution for binary files. The code I'm using is something...
0
7684
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8006
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7768
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6400
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5579
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5288
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3729
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2191
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
1035
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.