By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
437,967 Members | 1,687 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 437,967 IT Pros & Developers. It's quick & easy.

limiting download size with curl

P: n/a
Hi!

I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end. Since the serfver doesn't support HTTP range to
use it in curl like curl_setopt($ch, CURLOPT_RANGE, "0-5000"); I
started looking around and found that using the custom write function
I might be able to do the same.

As I understand it, closing the handle and thus terminating the
transfer from within the callback function is not possible, so I'm a
bit stuck..

How could I achieve this? Could someone paste me a fewliner example?

Thanks in advance
Andrew
Feb 26 '08 #1
Share this Question
Share on Google+
4 Replies


P: n/a
sh******@gmail.com wrote:
I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end.
Have you though using a HTTP HEAD request instead of a HTTP GET request?

Cheers,
--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-

No es culpable el asesino, sino el asesinado.
Feb 26 '08 #2

P: n/a
On Feb 26, 6:06*pm, Iván Sánchez Ortega <ivansanchez-...@rroba-
escomposlinux.-.punto.-.orgwrote:
shika...@gmail.com wrote:
I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end.

Have you though using a HTTP HEAD request instead of a HTTP GET request?

Cheers,
As I understand it, the HTTP HEAD request would get me a response that
doesn't have the body of the file in question. The target of my file
downloads are about 20% into body of the file, so this approach
wouldn't work.

Thank you for the suggestion though!
Feb 26 '08 #3

P: n/a
sh******@gmail.com wrote:
As I understand it, closing the handle and thus terminating the transfer
from within the callback function is not possible, so I'm a bit stuck..

How could I achieve this?
TCP Sockets. GET requests are not *that* difficult to implement.
--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 27 days, 23:43.]

Bottled Water
http://tobyinkster.co.uk/blog/2008/02/18/bottled-water/
Feb 26 '08 #4

P: n/a
On Feb 26, 6:26*pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
wrote:
shika...@gmail.com wrote:
As I understand it, closing the handle and thus terminating the transfer
from within the callback function is not possible, so I'm a bit stuck..
How could I achieve this?

TCP Sockets. GET requests are not *that* difficult to implement.
Could you please show me an example of this? I'm not sure what I'm
supposed to be googling for at this point.
Feb 26 '08 #5

This discussion thread is closed

Replies have been disabled for this discussion.