473,326 Members | 2,023 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

What am I doing wrong? curl

I want to store result of curl in a variable, which means to store a
webpage in a variable in order to parse it later and get our some useful
information. But unfortunately I'm not too experienced in C and I'm
making some stupid mistake. Could someone help me solve it?
#include <string.h>
#include <curl/curl.h>
size_t write_data(const char *buffer, size_t size, size_t nmemb, char
*userp)
{
char *string = userp;
size_t len;
len = size * nmemb;
strncat(string, buffer,len); // I get segfault here. Why????
return len;
}

int main(int argc, char **argv)
{
char *tablica="";
CURL *curl;
CURLcode success;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
sample url.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
success = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}

Thanks in advance for your help.
Nov 15 '05 #1
6 10291
On Mon, 05 Sep 2005 21:18:29 +0200, Shutdownrunner
<"shutdownrunner[NOSPAM]"@o2.pl> wrote:
I want to store result of curl in a variable, which means to store a
webpage in a variable in order to parse it later and get our some useful
information. But unfortunately I'm not too experienced in C and I'm
making some stupid mistake. Could someone help me solve it?
#include <string.h>
#include <curl/curl.h>
And we know what is in this header because ...


size_t write_data(const char *buffer, size_t size, size_t nmemb, char
*userp)
{
char *string = userp;
size_t len;
len = size * nmemb;
strncat(string, buffer,len); // I get segfault here. Why????
write_data is called only through curl_easy_setopt. We have no idea
no idea how that is accomplished. What do buffer and userp point to?
return len;
}

int main(int argc, char **argv)
{
char *tablica="";
CURL *curl;
CURLcode success;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
sample url.
curl_easy_setopt is a function taking three arguments. The third one
has a type consistent with pointer to char. It could be pointer to
void or const qualified.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?
curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
success = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}

Thanks in advance for your help.

<<Remove the del for email>>
Nov 15 '05 #2
Barry Schwarz napisa³(a):
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?

I don't know if I understand you correctly, because I haven't heard
about variadic functions before. There's a paragraph in libcurl which says:
"Let's assume for a while that you want to receive data as the URL
identifies a remote resource you want to get here. Since you write a
sort of application that needs this transfer, I assume that you would
like to get the data passed to you directly instead of simply getting it
passed to stdout. So, you write your own function that matches this
prototype:

size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp);

You tell libcurl to pass all data to this function by issuing a function
similar to this:

curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data); "

Userp can be FILE, but I wanted it to be a variable.
curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);


I'm passing tablica to write_data, because I want all that is downloaded
by curl to be stored in this variable. I still have a lot to read about
local and global variables, so it might be not that necessary as I
think. Please tell me if there's any better solution to do what I want
to do.

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management:( Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

Best regards,
Jacek
Nov 15 '05 #3
I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management:( Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?


So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
....
tablica = (char *) malloc(buffersize * sizeof(char));
....
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.

Zach

Nov 15 '05 #4
On 5 Sep 2005 15:16:48 -0700, "elzacho" <el*****@gmail.com> wrote:
I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management:( Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?
So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
...
tablica = (char *) malloc(buffersize * sizeof(char));


Casting the return from malloc only serves to hide real errors.
...
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.


Consider the realloc() function.

<<Remove the del for email>>
Nov 15 '05 #5
On Mon, 05 Sep 2005 23:41:28 +0200, Shutdownrunner
<"shutdownrunner[NOSPAM]"@o2.pl> wrote:
Barry Schwarz napisa³(a):
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?

I don't know if I understand you correctly, because I haven't heard
about variadic functions before. There's a paragraph in libcurl which says:
"Let's assume for a while that you want to receive data as the URL
identifies a remote resource you want to get here. Since you write a
sort of application that needs this transfer, I assume that you would
like to get the data passed to you directly instead of simply getting it
passed to stdout. So, you write your own function that matches this
prototype:

snip

You need to discuss this in a newsgroup where libcurl is on-topic.
<<Remove the del for email>>
Nov 15 '05 #6
"elzacho" <el*****@gmail.com> writes:
I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management:( Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?
So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
...
tablica = (char *) malloc(buffersize * sizeof(char));


Better:

tablica = malloc(buffersize);
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.


Use realloc() for this.

One thing to watch out for is that realloc(), like malloc() can fail.
If you want to be able to recover from such an error, you need to
assign the result of realloc() to a separate variable; otherwise you
could lose your original data.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Nov 15 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Haluk Durmus | last post by:
Hello I checked out openssl,mm,apr,apr-util,apache 2,curl,libxml and php from cvs. php couse an ERROR I did the following steps:
0
by: Phil Powell | last post by:
I am having trouble retrieving URLs using curl for PHP whereby the URL requires a cookie to produce proper data. I wrote a wrapper class called Timer that will time the execution/download of a...
8
by: mrbog | last post by:
1. In order to make an http (or https) request with PHP, I need to recompile php with cURL. 2. In order to install CURL I have to upgrade my openssl rpm, even though I'm runing a version of...
9
by: Dave Martin | last post by:
I've successfully used CURL to maneuver around and through sites but a new site I've been trying to work with has got me stumped. To retrieve the data I'm trying to get a login is required. If...
3
by: Chris Fortune | last post by:
# uname -a Linux stargate.mxc-online.net 2.4.20-021stab022.2.777-smp #1 SMP Wed Jul 28 17:12:37 MSD 2004 i686 i686 i386 GNU/Linux I recompiled PHP with mcrypt, openssl, and curl phpinfo():...
0
by: nfhm2k | last post by:
I've been trying to find a solution to this for quite some time now... I even took a look at existing scripts... Including this one......
0
by: Joshua Ruppert | last post by:
I'm trying to figure out what I'm doing wrong with my setup Alternative PHP Cache. It's not saving the cache entries. When I look at the APC Info page (apc.php) the only entry I ever see is...
0
by: xerc | last post by:
I am trying to create a generic function I can call to download all files from a single remote FTP directory -- using CURL. I want to multi-thread it, but need to get the single thread functionality...
3
by: rottmanj | last post by:
I am re-writing my rets application in perl, and I have found a few modules that will help me on my way. One of them being WWW::Curl:easy. During my testing, I have tested both system curl and...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.