Trying to extract a string from HTTP::Request object

[nkg1234567]

I'm trying to extract HTML from a website in the form of a string, and then I want to extract particular elements from the string using the substr function:
here is some sample code that I have thus far:

use HTTP::Request::Common;
use LWP::UserAgent;
use LWP::Simple;
 
$ua = LWP::UserAgent->new;
 
$request = HTTP::Request->new(GET => 'http://www.cnn.com');
$response = $ua->request($request);
$content = $response->content();
 
my $result2 = substr $content, index($content, 'Headlines');
 

So, the variable $content seems to be an HTML object or something that is NOT a string. How can I convert $content to a string, so that I can use the substr function?

I have tried other methods including simpler code:

my $content = get('http://securities.stanford.edu//1014/TCHC00');
 

however, I am not able to process $content as a string.

I have even tried putting the contents into a text file, but I am not able to extract a string from a text file either?

any help is appreciated!!!

[rickumali]

I ran your program through the Perl debugger, and confirmed that $response->content() definitely contains HTML. When I put the output into an editor, I found that the content does NOT contain "Headlines. " Try another keyword, like "Weather."

If you want to examine variables without the debugger, use this code (provided your Perl has the Dumpvalue module):

use HTTP::Request::Common;
use LWP::UserAgent;
use LWP::Simple;
use Dumpvalue;
 
$dumper=new Dumpvalue;
 
$ua = LWP::UserAgent->new;
 
$request = HTTP::Request->new(GET => 'http://www.cnn.com');
$response = $ua->request($request);
$content = $response->content();
 
$dumper->dumpValue(\$response);
 
my $result2 = substr $content, index($content, 'Headlines');
 

Then when you run it, save the output to a text file. On my Windows box, with ActiveState Perl, I used this:

C:\cygwin\home\Rick\perl>perl getreq.pl > output.txt
 

In the Perl debugger, this is what I see when I used 'Weather':

main::(getreq.pl:15):   my $result2 = substr $content, index($content, 'Weather');
  DB<1>
main::(getreq.pl:17):   print $result2;
  DB<1> print length($result2)
105335
  DB<2> print substr $result2, 0, 20
Weather, Entertainme
 

You're on the right track. Prove what each line does, and learn the Perl debugger to get interactive.