473,788 Members | 2,652 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

download website pages using perl

23 New Member
Is there any way that i could download all the webpages in a website using perl ?
Jan 26 '08 #1
3 1890
KevinADC
4,059 Recognized Expert Specialist
Yes, at least pages that are web accessible. What have you tried so far and why do you want to download entire websites?
Jan 26 '08 #2
numberwhun
3,509 Recognized Expert Moderator Specialist
Is there any way that i could download all the webpages in a website using perl ?
If you are just wanting to download a specific web page (that is accessible as Kevin mentioned), then use LWP::Simple module. It has a get() function that will do that. If you are wanting to do something a little more complex, then check out the WWW::Mechanize module then.

Regards,

Jeff
Jan 26 '08 #3
karthikeyanck
23 New Member
If you are just wanting to download a specific web page (that is accessible as Kevin mentioned), then use LWP::Simple module. It has a get() function that will do that. If you are wanting to do something a little more complex, then check out the WWW::Mechanize module then.

Regards,

Jeff
Thanks a ton for the inputs, i need to download the whole website coz i've a project on it
Jan 28 '08 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

2
5245
by: Matthew Sims | last post by:
Is it possible to force a download without using the readfile function? My website setup consists of my server that serves the web pages plus a high-speed file server elsewhere on the internet that servers my files. I'm currently using header("Location: abc.com") to redirect my users to this other site but can't force a download. I can get it to work using readfile() but that means the file moves from the high-speed server through my...
0
2327
by: jonathan.beckett | last post by:
Hi All, I have just made version 0.4.8 of the PluggedOut CMS Content Management System available for download - it's free, and covered by the GPL. It's still very much a work in progress (current release version is 0.4.8), but you should get a very good idea of what it's about by visiting the site (which uses it, funnily enough), or downloading a copy of it and trying it out.
0
11565
by: Antwerp | last post by:
Hi, I'm trying to create a perl script that will log into a website (the login form uses POST), navigate to several pages, and append the (html) content parsed from those pages to a seperate log file. I'm not very familiar with this aspect of perl, and have been having some trouble in the POSTing of the form data, while using cookies to log in. Visting the site automatically redirects you to a login page. Once you fill out the login...
5
6582
by: Joseph Ellis | last post by:
Hello all. I have a family website that I have been maintaining for the past year and a half or so. It includes a photo album that has grown quite large. So far I've displayed the photo album using frames, with thumbnails on the left side of the screen and full-size images displayed on the right. But now I'm sick of frames and I'd like to use javascript to make the whole shebang cleaner and tighter. I'd like to use a table,...
5
12309
by: Neil Rossi | last post by:
I have an issue with a particular ASP page on two web servers. Let's call these servers Dev1 and Beta1. Both Servers are running IIS 5, Windows 2000 SP4 with "almost" all of the latest patches. On Beta1, I am able to execute a particular page with no problem, that page opens up in the comes up just fine. On Win2kdev1, when I go to execute the same page, it opens a file download dialog and asks me whether I want to open or save the...
17
2290
by: stubbsie | last post by:
Hi, I have redesigned our official public government website in .net and it has taken me a few months to redo. I have been the sole designer of the website from its humble beginnning a few years ago when no one wanted to even mess with it. Since then I have been steadily maintaining it and improving it. I am about halfway done, and all of a sudden my supervisor won't let me make the website in .net. Our programmers use .net to make web...
5
1639
by: Pitaridis Aristotelis | last post by:
I have a large file which has to be downloaded from a user using the Internet Explorer. The problem is that when someone downloads this file, the server slows down and the other users can not see the web pages from the server because there is not available bandwidth. Is there any way to decrease the download speed of this file by reading pieces of the file and manually send each piece every 100 milliseconds in order to control the speed of...
25
2813
by: pereges | last post by:
Hello, I'm trying to build a database driven website for a library management system. The database is stored on a remote server which all of my team mates can access. I've installed MySQL, PHP and Apache on my machine. I'm a beginner and I really don't understand how to proceed. My biggest problem is how to connect to the database on remote mysql server ? Does php allow this kind of thing ? I'm interested in a web interface on my machine...
1
47489
KevinADC
by: KevinADC | last post by:
Note: You may skip to the end of the article if all you want is the perl code. Introduction Many websites have a form or a link you can use to download a file. You click a form button or click on a link and after a moment or two a file download dialog box pops-up in your web browser and prompts you for some instructions, such as “open” or “save“. I’m going to show you how to do that using a perl script. What You Need Any recent...
0
9656
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9498
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10366
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10173
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10110
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
7517
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6750
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5399
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5536
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.