473,320 Members | 1,949 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

compressed file ended before the logical end-of-stream was detected

Hi,

The program downloads the files from the internet and compresses them
to a single zip archive using compress_the_file().

Upon running syncer() which calls the decompress_the_file(), the first
iteration succeeds. But upon second iteration, I get an IOError
exception with the message:
"compressed file ended before the logical end-of-stream was detected"

Any ideas why this happens ? I don't think it has anything to do with
bad packets from the internet.
Here's the code.

There are two functions:

1) compress_the_file() - This function takes files as an argument to it
and put all of them into a zip archive

def compress_the_file(zip_file_name, files_to_compress, sSourceDir):
'''Condenses all the files into one single file for easy
transfer'''

try:
import zipfile
except ImportError:
sys.stderr.write("Aieeee! module not found.\n")

try:
os.chdir(sSourceDir)
except:
#TODO: Handle this exception
pass

try:
filename = zipfile.ZipFile(zip_file_name, "a")
except IOError:
#INFO By design zipfile throws an IOError exception when you
open
# in "append" mode and the file is not present.
filename = zipfile.ZipFile(zip_file_name, "w")
except:
#TODO Handle the exception
sys.stderr.write("\nAieee! Some error exception in creating zip
file %s\n" % (zip_file_name))
sys.exit(1)

filename.write(files_to_compress, files_to_compress,
zipfile.ZIP_DEFLATED)
filename.close()

My actual program iterates over a loop and sends files to this funtion
as it downloads from the internet.

2) decompress_the_file() - This function takes archive file as an
argument and decompresses them.

The above function decompress_the_file() is called by function syncer()

syncer(): This function tells decompress_the_file() what type file it
is and at which path to decompress.

def syncer(install_file_path, target_path, type=None):
'''Syncer does the work of syncing the downloaded files.
It syncs "install_file_path" which could be a valid file path
or a zip archive to "target_path'''

if type == 1:
try:
import zipfile
except ImportError:
sys.stderr.write("Aieeee! Module zipfile not found.\n")
sys.exit(1)

file = zipfile.ZipFile(install_file_path, "r")
for filename in file.namelist():
try:
import file_magic
except ImportError:
sys.stderr.write("Aieeee! Module file_magic not
found.\n")
sys.exit(1)
data = open(filename, "wb")
data.write(file.read(filename))
data.close()
#data = file.read(filename)
if file_magic.file(filename) == "application/x-bzip2":
decompress_the_file(os.path.abspath(filename),
target_path, filename, 1)
elif file_magic.file(filename) == "PGP armored data":
try:
shutil.copy(filename, target_path)
sys.stdout.write("%s file synced.\n" % (filename))
except shutil.Error:
sys.stderr.write("%s is already present.\n" %
(filename))

Jul 5 '06 #1
1 6379
In <11*********************@m79g2000cwm.googlegroups. com>, Ritesh Raj
Sarraf wrote:

I have some remarks on exception handling.
1) compress_the_file() - This function takes files as an argument to it
and put all of them into a zip archive

def compress_the_file(zip_file_name, files_to_compress, sSourceDir):
'''Condenses all the files into one single file for easy
transfer'''

try:
import zipfile
except ImportError:
sys.stderr.write("Aieeee! module not found.\n")
`zipfile` is part of the standard library. So it should be there. If it
isn't then you write the message and the program will run into a name
error exception because `zipfile` isn't imported then but you use it later
on.
try:
os.chdir(sSourceDir)
except:
#TODO: Handle this exception
pass
Don't do that! This catches every exception and ignores it. Write just a
comment instead that says something like `TODO: add proper exception
handling`.
try:
filename = zipfile.ZipFile(zip_file_name, "a")
except IOError:
#INFO By design zipfile throws an IOError exception when you
open
# in "append" mode and the file is not present.
filename = zipfile.ZipFile(zip_file_name, "w")
except:
#TODO Handle the exception
sys.stderr.write("\nAieee! Some error exception in creating zip
file %s\n" % (zip_file_name))
sys.exit(1)

filename.write(files_to_compress, files_to_compress,
zipfile.ZIP_DEFLATED)
filename.close()
Poor choice of names IMHO. I expect `filename` to be a file name, i.e. a
string and `files_to_compress` is plural so one may expect a list or
another iterable.

This `import` wrapped into `try`/`except ImportError` idiom sprinkled all
over the source is a bit odd. Just do your imports and wrap the main
programs function into `try`/`except` if you want to catch the
`ImportError`\s. If you stop the program anyway this leads to much less
cluttered source code.

And it maybe would make sense to print the actual catched `IOError`\s too
because it may be a valueable information for the user *why* it failed.
For example because he has no rights or the file system is full.

Ciao,
Marc 'BlackJack' Rintsch
Jul 5 '06 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Mark Wall | last post by:
If I have a UNIX compressed text file 'python.txt.Z' with contents: "Python " How do I read this data without uncompressing the file? Thank you, Mark
8
by: Dennis Hotson | last post by:
Hi, I'm trying to write a function that adds a file-like-object to a compressed tarfile... eg ".tar.gz" or ".tar.bz2" I've had a look at the tarfile module but the append mode doesn't support...
6
by: ad | last post by:
I have a compressed file in client. How could we send this compressed file (.zip) to server by web service?
8
by: robert | last post by:
Hello, I want to put (incrementally) changed/new files from a big file tree "directly,compressed and password-only-encrypted" to a remote backup server incrementally via FTP,SFTP or DAV.... At...
2
by: pmlane2001 | last post by:
I have a PHP SOAP XML file size problem that I was wondering if anyone has seen before. I have an XML file that when I put it through my PHP script with 270 lines (13,082 KB) it works fine. If I...
1
by: prince.matt | last post by:
Hi, I am using GZipStream to compress a simple text file approx 24MB in size. I am finding that the destination "compressed" file is several MB larger than the source file (31MB). The code is...
1
by: sedwick | last post by:
I'm using .NET 2.0.50727 in VS 2005 Pro, ENU Service Pack 1 (KB926601). I've been experimenting with the System.IO.Compression classes GZipStream and DeflateStream, and I found it interesting to...
4
by: PRR | last post by:
i have a xml file ..around 200kb (max) .. i compressed it .. using GZipStream class. Now i wanna encrypt it.... Is it right? or the other way round... First encrypt and then compress.... Also i wanna...
1
by: =?Utf-8?B?S2V5QmVl?= | last post by:
- YOU CAN PROTECT ZIPPED COMPRESSED FOLDER WITH A PASSWORD (PW). - YOU MUST PROVIDE THE PW BEFORE THE FILE CAN BE EXTRACTED. - HOW CAN YOU EXTRACT THE FILE IF YOU FORGOT THE PW ?
10
by: Mark | last post by:
I used pickle and found the file was saved in text format. I wonder whether anyone is familiar with a good compact off-the-shelf module available that will save in compressed format... or maybe an...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.