Hi!
I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end. Since the serfver doesn't support HTTP range to
use it in curl like curl_setopt($ch , CURLOPT_RANGE, "0-5000"); I
started looking around and found that using the custom write function
I might be able to do the same.
As I understand it, closing the handle and thus terminating the
transfer from within the callback function is not possible, so I'm a
bit stuck..
How could I achieve this? Could someone paste me a fewliner example?
Thanks in advance
Andrew
4  2834  sh******@gmail. com wrote:
I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end.
Have you though using a HTTP HEAD request instead of a HTTP GET request?
Cheers,
--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-
No es culpable el asesino, sino el asesinado.
On Feb 26, 6:06*pm, Iván Sánchez Ortega <ivansanchez-...@rroba-
escomposlinux.-.punto.-.orgwrote:
shika...@gmail. com wrote:
I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end.
Have you though using a HTTP HEAD request instead of a HTTP GET request?
Cheers,
As I understand it, the HTTP HEAD request would get me a response that
doesn't have the body of the file in question. The target of my file
downloads are about 20% into body of the file, so this approach
wouldn't work.
Thank you for the suggestion though! sh******@gmail. com wrote:
As I understand it, closing the handle and thus terminating the transfer
from within the callback function is not possible, so I'm a bit stuck..
How could I achieve this?
TCP Sockets. GET requests are not *that* difficult to implement.
--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 27 days, 23:43.]
Bottled Water http://tobyinkster.co.uk/blog/2008/02/18/bottled-water/
On Feb 26, 6:26*pm, Toby A Inkster <usenet200...@t obyinkster.co.u k>
wrote:
shika...@gmail. com wrote:
As I understand it, closing the handle and thus terminating the transfer
from within the callback function is not possible, so I'm a bit stuck..
How could I achieve this?
TCP Sockets. GET requests are not *that* difficult to implement.
Could you please show me an example of this? I'm not sure what I'm
supposed to be googling for at this point. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Chuck Anderson |
last post by:
I am writing a Php script to run on my home PC (Windows) that downloads
an Apache access log file and inserts new entries into a database.. The
only way I can access these log files is through a cPanel URL that
delivers a gzipped copy of the log file with all entires for the current
month (up to the current date and time).
I want to run this script once a day (at night) and would like to
download the gzipped file by resuming from where...
|
by: gehegeradeaus |
last post by:
I used curl a few times now, and it always worked fine... but now I
have a little question:
If I post to a certain url , and that page returns a file for download.
(it doesn't display anything)
How can accept the download, and read the contents...?
My code :
|
by: a.r.austin |
last post by:
Hello,
I am trying to download a few files one after another from a remote
server. Problem is that I don't know how to, or if I am able at all,
set a time out for download. I don't want to time out whole script,
just a part if file won't download in 2mins then skip to the next one.
Previously, I had a Javascript implementation with AJAX, this time I
thought of doing it in PHP since PHP has far better array functions, I
don't have to do...
|
by: xerc |
last post by:
I am trying to create a generic function I can call to download all files from a single remote FTP directory -- using CURL. I want to multi-thread it, but need to get the single thread functionality working first before I tackle that. Anyway, in my function I can list all the files, but the function I have, no matter how I try, will only return one file -- the last file. My for() loop seems pretty straightforward, so not sure why only the...
|
by: Dimitrios Apostolou |
last post by:
Hello list,
I want to limit the download speed when using urllib2. In particular,
having several parallel downloads, I want to make sure that their total
speed doesn't exceed a maximum value.
I can't find a simple way to achieve this. After researching a can try
some things but I'm stuck on the details:
1) Can I overload some method in _socket.py to achieve this, and perhaps
| | |
by: Floris |
last post by:
I'm using libcurl to download some content from another webpage. For
reasons previously unknown to me all calls are very slow to the point of
usual timeout. Now i found curl_getinfo() and retrieved some info on the
subject:
Array
(
=https://undisclosed-website
=text/html; charset=UTF-8
=200
|
by: Mikhail Kovalev |
last post by:
Suppose I have something like this
$resource = curl_init();
curl_setopt($resource, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($resource, CURLOPT_URL, '......');
curl_exec($resource);
$lastUrl = curl_getinfo($resource, CURLINFO_EFFECTIVE_URL);
curl_close($resource);
|
by: nestle |
last post by:
I have DSL with a download speed of 32MB/s and an upload speed of 8MB/s(according to my ISP), and I am using a router. My upload speed is always between 8MB/s and 9MB/s(which is above the max upload speed), ALWAYS. However, my download speed doesn't go over 25MB/s. And when my brother turns on the internet from his computer and takes up half the download/upload speeds (routers automatically split the speeds in two when two computers are using...
|
by: jeddiki |
last post by:
I am using curl to down load a zip file and I am
then trying to unzip it, but I don't think that I am
using the functions properly because I get warnings
This is my code:
<?php
if (!extension_loaded('zip')) {
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| | |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
