473,403 Members | 2,366 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,403 software developers and data experts.

Need time-out procedure for HTML perl procedure

I have the following perl script which works nicely. It takes a list of URLs from a file, looks up each one, and returns the meta-keywords for that domain. Problem is, sometimes it hangs because it goes to a site which does not respond. I'm wondering if there is a method by which if it times out, or gets an error response, or in the alternative, a success response, it will know to continue on to the next line in the file. Here is the current script which works, but, as I said, hangs upon grabbing meta-keywords from a bad site:

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl 
  2. #print "Content-type: text/html\n\n"; 
  3. use LWP::Simple; 
  4. use HTML::HeadParser; 
  5. open (OUTFILE, '>outfile.txt'); 
  6. open (MYFILE, 'url3.txt'); 
  7. foreach $line (<MYFILE>) { 
  8. chomp($line); 
  9. $URL = get($line); 
  10. $Head = HTML::HeadParser->new; 
  11. $Head->parse("$URL");  
  12. print OUTFILE $Head->header('X-Meta-Description') . "."; 
  13. close(MYFILE); 
  14. close(OUTFILE); 
  15. exit;
Aug 17 '10 #1
2 1566
toolic
70 Expert
I can't see a way to configure the timeout for LWP::Simple, but its POD directs you to LWP::UserAgent, which does seem to allow you to control the timeout.
Aug 17 '10 #2
RonB
589 Expert Mod 512MB
See: 'perldoc -f alarm
http://perldoc.perl.org/functions/alarm.html
Aug 17 '10 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

4
by: Brian | last post by:
I have 4 sites sharing an account on a server that is in the US Eastern time zone. 3 of those sites are for businesses/persons who live in the same time zone, but one is for a restaurant in the US...
4
by: deko | last post by:
I need code to run once a day (no cron service available) - when the first visitor of the day hits my site would be fine. What I have in mind is a Boolean function that identifies day boundaries...
7
by: Valiz | last post by:
Hi, I am updating system time with IRIG time source which sends irregular pulses as shown below 11000011111100111111111111111111....(1-IRIG present and 0-IRIG not present) I need to update...
7
by: mammothman42 | last post by:
hi everyone i need some creative input. i'm trying to enable the user to input a length of time, from one day to about 10 years. ideas i've had have been: - 3 pull down boxes, for days, months,...
30
by: nephish | last post by:
Hey there, i have tried about every graphing package for python i can get to work on my system. gnuplot, pychart, biggles, gdchart, etc.. (cant get matplot to work) so far, they all are working...
5
by: Jeremy | last post by:
I have a core VB service that monitors a database, and based on data in the records will execute code to send email notifications. Problem: I don't want my main program code to halt and wait for...
9
by: Bob Achgill | last post by:
I would like to use the timestamp on files to manage the currency of support files for my VB windows application. In this case I would only put the timestamp of the file in the management database...
11
by: greg | last post by:
Hi all, Is there a way to get the current theme name at design time? I'm trying to write a custom control for which I need to use images from the current theme. I have asigned a theme to the...
3
by: amanjsingh | last post by:
Hi, I have a Database (MSSQL) with some tables having multiple Time columns (which I imported from some Access DB). The DateTime format in the MSSQL is 1899-12-30 02:30:00.0 format. I populate...
1
deephill
by: deephill | last post by:
hi thr, i need Time based background image change script for website. ex: if user open in moring time background with sunrise* open in midday background vry bright* open in the evng background...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.