curl extension downloads 1 byte only from zip files

I have set up a system to download datafeeds in pain text or zipped.
The download part of this system uses the curl extension to download
the files.

All was well when I tested it with various datafeeds in zipped and
unzipped formats until I came to download zipped files from the
affiliate program we are starting with.

I have verified that the datafeeds download properly in Internet
Explorer but interestingly they also fail in Firefox.

Plain text files will download correctly from the same server with the
curl script.

Using curl to download a file results in a 1 byte file. Downloading
with Firefox produces a 4kb corrupted zip file. I have tried multiple
files all with the same result.

A php script appears to be being used to create the zips on the fly?
could this be the cause of the problem?

Any thoughts on this one are much appricated

-Ben

re: curl extension downloads 1 byte only from zip files

On 12 Jul 2005 12:57:35 -0700, "benji" <benpjohnson@gmail.com> wrote:
[color=blue]
>I have set up a system to download datafeeds in pain text or zipped.
>The download part of this system uses the curl extension to download
>the files.
>
>All was well when I tested it with various datafeeds in zipped and
>unzipped formats until I came to download zipped files from the
>affiliate program we are starting with.
>
>I have verified that the datafeeds download properly in Internet
>Explorer but interestingly they also fail in Firefox.
>
>Plain text files will download correctly from the same server with the
>curl script.
>
>Using curl to download a file results in a 1 byte file. Downloading
>with Firefox produces a 4kb corrupted zip file. I have tried multiple
>files all with the same result.
>
>A php script appears to be being used to create the zips on the fly?
>could this be the cause of the problem?[/color]

Minimal example code reproducing the problem?

Example URL?

First place to look would be the Content-type header.

--
Andy Hassall / <andy@andyh.co.uk> / <http://www.andyh.co.uk>
<http://www.andyhsoftware.co.uk/space> Space: disk usage analysis tool

re: curl extension downloads 1 byte only from zip files

Thanks for the feedback it has helped my understanding of the problem.

Here is a cut down version of the code I am using. Sorry about omitting
the url I don't think the company would be impressed if I posted it in
a public forum.

I have also posted the responce headers below. I think that the
Content-Disposition: attachment; part is the key. I am guessing that
curl is downloading the result of the url (nothing) rather than the
file the headers are pointing to unfortunality I still cant seem to
find any relevent information that explains how curl handles this
situation.

-Ben

$ch = curl_init('url');

$fp = fopen("downloaded.zip", "w");

curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($ch, CURLOPT_FILE, $fp);

$downloaded = curl_exec($ch);

curl_close($ch);

fclose($fp);

HTTP/1.1 200 OK
Date: Tue, 12 Jul 2005 23:42:34 GMT
Server: Apache/2.0.46 (Red Hat)
Accept-Ranges: bytes
X-Powered-By: PHP/4.3.2
Set-Cookie: PHPSESSID=ddb9c654715d488d3f6036127b1d624d; path=/
Cache-Control: must-revalidate
Content-Length:
Content-Disposition: attachment; filename=awcsv.zip
Connection: close
Content-Type: application/zip

re: curl extension downloads 1 byte only from zip files

> Content-Length:


The problem might be in empty Content-Length header field.
You may signal this error to the company providing the data
this way.


Hilarion

re: curl extension downloads 1 byte only from zip files

Thank you for your help with this problem I will add it to my support
ticket and see if they do anything about it.

Interestingly I have knocked up a solution using fsockopen which works
with the data, although I would still prefer to use curl as I am using
it for other parts of my application.

Best Regards

Ben

re: curl extension downloads 1 byte only from zip files

> Interestingly I have knocked up a solution using fsockopen which works[color=blue]
> with the data, although I would still prefer to use curl as I am using
> it for other parts of my application.[/color]


You may try using curl_setopt with CURLOPT_WRITEFUNCTION and writing
your own function handling data retrieval.


Hilarion

re: curl extension downloads 1 byte only from zip files

Thanks for all your help Hilarion,

I will take a look at CURLOPT_WRITEFUNCTION