473,324 Members | 1,856 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

Web site

Is there an example someplace of how I can turn a web site into a string of text so I can parse it?

I am trying to extract a returned value from an existing website that I have no control over. Specifically http://www.ffiec.gov/ratespread/default.aspx

I am trying to pull the rate spread field.

Thanks
Lance
Nov 20 '05 #1
2 1145
Check out this article:
http://www.devcity.net/net/article.a...=screen_scrape
Not too long ago, if you wanted some particular information off of a particular web site, you'd have to snake the HTML off a page and incorporate it into yours. Whether you did that manually via cut and paste or with a homegrown process was up to you - usually it involved some pain and misery to get it right.

Even today, as we teeter on the 'new age' of web services, we still have problems getting what we want from our favorite web pages - maybe we need some information that isn't exposed via a web service, and until the Frito chomping, Jolt drinking programmer that wrote the page shuts off 'Star Trek', gets up off the sofa and writes a web service, we'll have to do their job for them.

The idea of screen scraping isn't new, in fact, many unsavory types use some sort of screen scraping to retrieve email addresses and harvest images from unsuspecting sites. Actually, this is common practice on the web - one that is nefarious and ill received by most of the Internet community.

No, I'm not going to show you how to screen scrape email addresses off of pages, so don't ask me - instead, we'll do a little constructive scraping in order to put more content out on the web.
--
Greetz,
Jan
__________________________________
Read my weblog: http://weblogs.asp.net/jan
"Lance Geeck" <lg****@cox.net> schreef in bericht news:TbFJb.46058$m83.24557@fed1read01...
Is there an example someplace of how I can turn a web site into a string of text so I can parse it?

I am trying to extract a returned value from an existing website that I have no control over. Specifically http://www.ffiec.gov/ratespread/default.aspx

I am trying to pull the rate spread field.

Thanks
Lance
Nov 20 '05 #2
* "Lance Geeck" <lg****@cox.net> scripsit:
Is there an example someplace of how I can turn a web site into a string of text so I can parse it?


\\\
Import System.IO
Import System.Net

..
..
..

Public Function LoadTextFile(ByVal Url As String) As String
Dim wrq As WebRequest = WebRequest.Create(Url)
Dim wrp As HttpWebResponse = _
DirectCast(wrq.GetResponse(), HttpWebResponse)
Dim sr As StreamReader = _
New StreamReader(wrp.GetResponseStream)
Dim Text As String = sr.ReadToEnd()
sr.Close()
wrp.Close()
Return Text
End Function
///

--
Herfried K. Wagner [MVP]
<http://www.mvps.org/dotnet>
Nov 20 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Florent | last post by:
Hi, I run a few sites and I want to log in my main site database when/if there is a problem, (like a page not found or an unknown agent). But I don't want to give direct access to my database...
6
by: Brad | last post by:
I have a win2003 server workstation with multiple webs, each web has it's own ip address. In VS2005, if I select to open an existing web site, select Local IIS, the dialog correctly displays a...
0
by: HackingPSP | last post by:
I saw a lot of requests for a program like this, so I wrote it. Yeah, my site has "PSP software by Auri" but in this case it means "Pretty Sweet Programming" :) There's both a VS2005 add-in and a...
0
by: Wayne Sepega | last post by:
We are using windows 2003 standard server, VSS 2005 and VS 2005 (mostly team dev, but some team arch and suite) We run windows 2003 on our development boxes, this gives us the ability to host...
9
by: CK | last post by:
Hi All, I have a sharepoint site on a box. I can hit the box by going to the ip address http://10.0.0.1/default.aspx but I can not hit it by name. The password dialong box comes up and it rejects...
71
by: Murray R. Van Luyn | last post by:
Hi, Since I have made changes to my website it's been a complete flop. According to the logs, as soon as visitors have downloaded the index page they are off. I can't figure out why? ...
20
by: mike | last post by:
I help manage a large web site, one that has over 600 html pages... It's a reference site for ham radio folks and as an example, one page indexes over 1.8 gb of on-line PDF documents. The site...
3
by: DBLWizard | last post by:
Howdy All, Is it possible to have Visual Studio 2005 create a project form an existing hosted website? In other words I want to be able connect via ftp to my website structure and have it pull...
16
by: Ben Sehara | last post by:
Is there any way I can limit the access to my website? I have a site "A" and I want to allow access to it only from site "B" login user. If someone try to access site "A" directory, I want it...
3
by: John Kotuby | last post by:
Hi all, Within an IFRAME of a standard site constructed of mostly static HTM type pages, I am calling up one page from a large ASP.NET 3.5 site. I have precompiled the ASP.NET site and...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.