473,405 Members | 2,344 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,405 software developers and data experts.

Mechanize module

Hi,

i am new to perl...

i tried this module mechanize. this is the script
Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2.  
  3. # Include the WWW::Mechanize module
  4. use WWW::Mechanize;
  5.  
  6. # What URL shall we retrieve? [I HAVE TRIED DIFFERENT URLS ALSO]
  7. $url = "http://www.example.com/bestselling.html";
  8.  
  9.  
  10. # Create a new instance of WWW::Mechanize
  11. # enabling autoheck checks each request to ensure it was successful,
  12. # producing an error if not.
  13. my $mechanize = WWW::Mechanize->new(autocheck => 1);
  14.  
  15. # Retrieve the page
  16. $mechanize->get($url);
  17.  
  18. # Assign the page content to $page
  19. my $page = $mechanize->content;
  20.  
  21. # Output the page
  22. print $page;
  23.  
  24. # Let's also save the page locally
  25. open(FH, ">bestsellers.txt");
  26.  
  27. print FH $page;
  28.  
  29. close(FH);
  30.  
the error is
================================================== ==============


Error GETing http://www.example.com/bestselling.html: Can't connect to www.example.com:80 (Bad hostname 'www.example.com').

SOLUTION TRIED FROM USER GROUPS
================================================== ==============

1) i have turned off firewall with /etc/init.d/iptables stop

2) i have checked the error log file
[Wed Feb 06 10:00:09 2008] [notice] core dump file size limit raised to 18446744073709551615 bytes
[Wed Feb 06 10:00:10 2008] [notice] Digest: generating secret for digest authentication ...
[Wed Feb 06 10:00:10 2008] [notice] Digest: done
[Wed Feb 06 10:00:11 2008] [notice] Apache/2.0.46 (Red Hat) configured -- resuming normal operations


================================================== ================

BUT STILL I DONT KNOW HOW TO RECTIFY


I HAVE TRIED LWP::Simple module also but this is showing error 500.


give me some suggestion to rectifiy this


Suresh
Feb 7 '08 #1
6 3348
debasisdas
8,127 Expert 4TB
Question move to PERL forum.
Feb 7 '08 #2
eWish
971 Expert 512MB
Your code works. Below is a slightly revised version.
Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl -T
  2.  
  3. use strict;
  4. use warnings;
  5.  
  6. use CGI;
  7. use CGI::Carp qw/fatalsToBrowser/;
  8.  
  9. my $q = CGI->new;
  10.  
  11. print $q->header();
  12. print $q->start_html();
  13.  
  14. # Include the WWW::Mechanize module
  15. use WWW::Mechanize;
  16.  
  17. # What URL shall we retrieve? [I HAVE TRIED DIFFERENT URLS ALSO]
  18. my $url = 'http://somesite.com';
  19.  
  20.  
  21. # Create a new instance of WWW::Mechanize
  22. # enabling autoheck checks each request to ensure it was successful,
  23. # producing an error if not.
  24. my $mechanize = WWW::Mechanize->new(autocheck => 1);
  25.  
  26. # Retrieve the page
  27. $mechanize->get($url);
  28.  
  29. # Assign the page content to $page
  30. my $page = $mechanize->content();
  31.  
  32. # Output the page
  33. print $page;
  34.  
  35. # Let's also save the page locally
  36. open(my $FH, '>', 'path/to/file/bestsellers.txt') || die "Can't open file $!\n";
  37.  
  38. print $FH $page;
  39.  
  40. close($FH);
  41.  
  42. print $q->end_html();
  43.  
  44. 1;
--Kevin
Feb 9 '08 #3
numberwhun
3,509 Expert Mod 2GB
Well, if the code works, then I would check to ensure that there isn't a firewall in the way, stopping you from getting out.

Regards,

Jeff
Feb 9 '08 #4
KevinADC
4,059 Expert 2GB
Well, if the code works, then I would check to ensure that there isn't a firewall in the way, stopping you from getting out.

Regards,

Jeff

Evidently he tried that already

SOLUTION TRIED FROM USER GROUPS
================================================== ==============

1) i have turned off firewall with /etc/init.d/iptables stop
Feb 9 '08 #5
numberwhun
3,509 Expert Mod 2GB
Evidently he tried that already
he he, missed that one in my tired, zen state. Thanks!
Feb 9 '08 #6
Hi,

Thanks for replies. The script is working now. My system admin has changed my system primary DNS server address. Now its working. Once again thanks for your prompt replies.


with regards
Suresh
Feb 12 '08 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

0
by: bruce | last post by:
hi... it appears that i'm running into a possible problem with mechanize/browser/python rgarding the "select_form" method. i've tried the following and get the error listed: br.select_form(nr...
2
by: bruce | last post by:
hi... i can do the following br = Browser br.open("www.yahoo.com") br.open("file:///foo") but can i do s = "..." <<<< qualified html text br.open(s)
0
by: bruce | last post by:
hi... update to an ongoing issue i've been having regarding html/Browser and selecting forms. i've created a basic test app, and created a stripped down page of html. the html has a single...
12
by: John | last post by:
I have to write a spyder for a webpage that uses html + javascript. I had it written using mechanize but the authors of the webpage now use a lot of javascript. Mechanize can no longer do the job....
2
by: Patrick C | last post by:
I'm trying to learn the basics of the mechanize module and i'm very very new to programming. Does anyone know of some good places to learn the very basics, say with some examples as well? I found...
1
numberwhun
by: numberwhun | last post by:
I am having an issue with understanding something in the WWW::Mechanize module. I have a website which I want to download a whole plethora of pdf files from. It is a site that I have paid to access...
0
by: bruce | last post by:
i'm getting the following error: mechanize._response.httperror_seek_wrapper: HTTP Error 500: i'm running python 5.1 and mechanize 0.1.7b I have no idea as to what I have to...
2
by: Rex | last post by:
Hello, I am working on an academic research project where I need to log in to a website (www.lexis.com) over HTTPS and execute a bunch of queries to gather a data set. I just discovered the...
1
by: tedpottel | last post by:
Hi, I can read the home page using the mechanize lib. Is there a way to load in web pages using filename.html instad of servername/ filename.html. Lots of time the links just have the file...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.