473,508 Members | 2,289 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to Store web pages to local disk using httpwebrequest

Hi all,

I have a problem downloading web pages to my local system. I am using the
HttpebRequest class to query a web site and get the response from the same.
UndNow, when i asy a web page, all the images and support files such as JS
file, CSS files are also in picture.

I want to store the response stream to the local drive, say something like
"C:\Temp", and i want all the images and CSS files to be stored to this local
path.

Please mention any methods (sample codes will be of great help), to store
the web pages along with the images, to my local hard disk.

Thanks.
Indiresh.
Feb 10 '06 #1
3 2385
you need to parse the html for references, then fetch them. this can get
complex if javascript is modifing the html. in this case, you need to also
interpret (run) the javascript. also be sure to check if one of the
refernces is html, you need to do the same on it.

-- bruce

"Indiresh" <In******@discussions.microsoft.com> wrote in message
news:00**********************************@microsof t.com...
Hi all,

I have a problem downloading web pages to my local system. I am using the
HttpebRequest class to query a web site and get the response from the
same.
UndNow, when i asy a web page, all the images and support files such as JS
file, CSS files are also in picture.

I want to store the response stream to the local drive, say something like
"C:\Temp", and i want all the images and CSS files to be stored to this
local
path.

Please mention any methods (sample codes will be of great help), to store
the web pages along with the images, to my local hard disk.

Thanks.
Indiresh.

Feb 10 '06 #2
Not to mention the need to update the referenced URLs in the HTML now
that the relevant items are localized. Since you can't get around
changing the references, you could just convert any relative URLs to
absolute ones and not actually download the items.

Bruce Barker wrote:
you need to parse the html for references, then fetch them. this can get
complex if javascript is modifing the html. in this case, you need to also
interpret (run) the javascript. also be sure to check if one of the
refernces is html, you need to do the same on it.

-- bruce

"Indiresh" <In******@discussions.microsoft.com> wrote in message
news:00**********************************@microsof t.com...
Hi all,

I have a problem downloading web pages to my local system. I am using the
HttpebRequest class to query a web site and get the response from the
same.
UndNow, when i asy a web page, all the images and support files such as JS
file, CSS files are also in picture.

I want to store the response stream to the local drive, say something like
"C:\Temp", and i want all the images and CSS files to be stored to this
local
path.

Please mention any methods (sample codes will be of great help), to store
the web pages along with the images, to my local hard disk.

Thanks.
Indiresh.


Feb 11 '06 #3
Hi,
Thanks for the response. Actually, I can go for HTML parsing, but there are
many conditions which i need to check for. And while parsing, it can take in
some considerable time, whereas i am keeping in view the performace issue as
well.

Hence, i want to implement/Automate the Internet Explorer's SAVE AS
functionality, where i can point to my defined folder and store the images
and all other files at one shot.

Thanks,
Indiresh.

"Russell" wrote:
Not to mention the need to update the referenced URLs in the HTML now
that the relevant items are localized. Since you can't get around
changing the references, you could just convert any relative URLs to
absolute ones and not actually download the items.

Bruce Barker wrote:
you need to parse the html for references, then fetch them. this can get
complex if javascript is modifing the html. in this case, you need to also
interpret (run) the javascript. also be sure to check if one of the
refernces is html, you need to do the same on it.

-- bruce

"Indiresh" <In******@discussions.microsoft.com> wrote in message
news:00**********************************@microsof t.com...
Hi all,

I have a problem downloading web pages to my local system. I am using the
HttpebRequest class to query a web site and get the response from the
same.
UndNow, when i asy a web page, all the images and support files such as JS
file, CSS files are also in picture.

I want to store the response stream to the local drive, say something like
"C:\Temp", and i want all the images and CSS files to be stored to this
local
path.

Please mention any methods (sample codes will be of great help), to store
the web pages along with the images, to my local hard disk.

Thanks.
Indiresh.


Feb 13 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

21
2908
by: matvdl | last post by:
I have a system that was originally developed in asp - the pages are saved in SQL (there are over 10,000 pages) and saved to a temp directory in the server when requested by a client. I have...
4
1410
by: Steve Wolfie | last post by:
Hello all: Again, let me thank everyone who has helped in the past. Can't wait till I can help out with some advice of my own. Now, I am building an app that wishes to retrieve the output of...
2
1609
by: jens Jensen | last post by:
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(ConfigurationManager.AppSettings); req.Proxy = new System.Net.WebProxy(ConfigurationManager.AppSettings, true); req.ContentType =...
10
4234
by: Paul Cheetham | last post by:
Hi, I am developing an application that needs to store some machine-specific settings. The application is going to be published on the network in order to keep the clients on the latest version....
18
9121
by: siddharthkhare | last post by:
Hi All, what is the diference between these two cache control header. no-cache and no-store. I have read the w3.org explanation. So lets say I am using only no-cache ....my understanding is...
3
6443
by: ary | last post by:
I try to create a weblog host site! in this case i can't use cache for every page because that cause to be my Server ram full of caching page. but if I can save cache in hard disk my problem...
6
1477
by: emorgoch | last post by:
Hi there. I'm currently working on a project involving rebuilding our local intranet site. The site was originally developed using frames, defining a header, sidebar, and content area, which...
1
5894
by: Joe | last post by:
Hello, I'm currently using a C# class library which is also converted quickly to a console app by adding a MAIN and adjusting the building configuration. I'm using this page as a reference to the...
8
1648
by: ak | last post by:
Hi Guys, I was just wondering whether it is possible to translate JSP pages into ASP pages using XSLT. What I want is to be able to open a currently available website developed in JSP in a...
0
7224
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
7323
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
1
7039
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
7494
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
5626
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
5050
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
4706
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
3180
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
0
415
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.