Requesting Compressed Pages?

[Keith Bowes]

I have a script for requesting HTTP resources and I want it to use HTTP
compression (to reduce bandwidth), if possible. What's the best way to
do this? I've tried using zlib functions but they always return FALSE.
And I've tried to use the zlib wrapper, which works but makes
non-compressed resources take longer.

TIA.

[Manuel Lemos]

Hello,

On 12/21/2003 11:03 AM, Keith Bowes wrote:

I have a script for requesting HTTP resources and I want it to use HTTP
compression (to reduce bandwidth), if possible. What's the best way to
do this? I've tried using zlib functions but they always return FALSE.
And I've tried to use the zlib wrapper, which works but makes
non-compressed resources take longer.

Do you want to serve compressed pages via PHP or request compressed
pages to be retrieved from a remote server?

--

Regards,
Manuel Lemos

Free ready to use OOP components written in PHP
http://www.phpclasses.org/

[Keith Bowes]

Manuel Lemos wrote:

Do you want to serve compressed pages via PHP or request compressed
pages to be retrieved from a remote server?

Request compressed pages. I can find all sorts of documentation and
code examples for the former, but not any about requesting pages. To be
more accurate, I don't know how to decompress the pages. I can request
them with no problem, but the information isn't of use to me in
compressed form.

[Daniel Tryba]

Keith Bowes <do****@spam.me> wrote:

Request compressed pages. I can find all sorts of documentation and
code examples for the former, but not any about requesting pages. To be
more accurate, I don't know how to decompress the pages. I can request
them with no problem, but the information isn't of use to me in
compressed form.

How do you request the page? HTTP/1.0 or HTTP/1.1? In the later case you
might get a chunked response (tranfser-encoding). See 3.6.1 Chunked
Transfer Coding of rfc 2616.

--

Daniel Tryba

[Keith Bowes]

Daniel Tryba wrote:

How do you request the page? HTTP/1.0 or HTTP/1.1? In the later case you
might get a chunked response (tranfser-encoding). See 3.6.1 Chunked
Transfer Coding of rfc 2616.

Yeah, I use HTTP/1.1. I've read the HTTP/1.1 spec, but Chunked Encoding
didn't make sense. I'll try to read it again.

[CountScubula]

Ok, I might be missing something so please forgive me.

If you are able to have succcess with the zlib library, then you 90%
done, you just need to look at the return header from the server.

your request should look like this (edit as needed)

$hHOST = "www.somedomain.com"; // the domain name for the page
$hURI = "/page_to_get.html"; // the page you want
$hREF = "http://mysite.com/was"; // in case you need the referer
$hAGENT = "My fuzzy web bot v1.0"; // the name of the user agent
(browser), IE or Netscape, etc..
$header = "GET $hURI HTTP/1.1" .
. "Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, "
. "application/vnd.ms-powerpoint, application/vnd.ms-excel,
application/msword, "
. .application/x-shockwave-flash, */*\n"
. "Host: $hHOST\n"
. "Referer: $hREF \n"
. "User-Agent: $hAGENT\n"
. "Accept-Encoding: gzip\n"
. "Accept-Language: en-us\n"
. "Connection: Close\n"
. "\n";

now send the $header string to the server after you connect to it,
then look for this line from the server

"Content-Encoding: gzip"

if you find gzip in the encoding header that the server sends back,
then it is sending back encoded data, otherwise it did not encode it.

I hope this help.

Mike Bradley
http://gzen.myhq.info -- free online php tools

[Daniel Tryba]

CountScubula <sc*****@hotmail.com> wrote:

Ok, I might be missing something so please forgive me. [snip] now send the $header string to the server after you connect to it,
then look for this line from the server

"Content-Encoding: gzip"

if you find gzip in the encoding header that the server sends back,
then it is sending back encoded data, otherwise it did not encode it.

You are missing something indeed. The OP doesn't have problems
requesting compressed files, but _decompressing_ them.

Now if he only posted some sample code with a url to test, we might be
able to help him point to the flaws in his script....

--

Daniel Tryba

[CountScubula]

Daniel Tryba <ne****************@canopus.nl> wrote in message news:<bs**********@news.tue.nl>...

CountScubula <sc*****@hotmail.com> wrote:

Ok, I might be missing something so please forgive me.

[snip]

now send the $header string to the server after you connect to it,
then look for this line from the server

"Content-Encoding: gzip"

if you find gzip in the encoding header that the server sends back,
then it is sending back encoded data, otherwise it did not encode it.

You are missing something indeed. The OP doesn't have problems
requesting compressed files, but _decompressing_ them.

Now if he only posted some sample code with a url to test, we might be
able to help him point to the flaws in his script....

Sounds good to me, :)
I have a small web page that reqest data from server:
http://gzen.myhq.info/serverchk.php

they should try using that link to see if my script can do it?
..... just a thought
Mike Bradley
http://gzen.myhq.info -- free online php tools

[CountScubula]

BTW,

With chunked,

after the header (+ 1 blank line)
you will recieve a number, that number is how many bytes follow
read that many bytes in your buffer, that is the chunk.

then you will recieve another number, that is the next chunk size,
read that many bytes, then repeat

when you recieve 0 as the number, you are done.
Mike Bradley
http://gzen.myhq.info -- free online php tools