468,256 Members | 1,447 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,256 developers. It's quick & easy.

programmatically navigate websites

How can I programmatically navigate a website just as a user would? click
on buttons, hyperlinks, etc. I know how to read nodes using XPath but I
don't know what classes to use to navigate.

I have to write a program to automatically read mail from an html mail
system.
Thanks,
T

Oct 22 '08 #1
2 2992
it will depend on the website. you can use webclient to get the page.
fews sites render valid xml, so you will need an html parser to read the
page. some pages require javascript, or have javascript modifiy the dom.
this would be common with a mail system, but it may have a dumb mode
(maybe for mobile devices). for instance with aspx any thing but an
imagebutton or submit button require javascript to do postback. you will
need emulate this.

you should read the w3c http 1.1 protocol spec so you know how to get
and post data. some site (say an aspx site with viewstate), will require
you do a get and postback matching data.

you should get a tool like fiddler, so you can see the raw data passed
back and forth and what headers are required.

-- bruce (sqlwork.com)

Tina wrote:
How can I programmatically navigate a website just as a user would?
click on buttons, hyperlinks, etc. I know how to read nodes using XPath
but I don't know what classes to use to navigate.

I have to write a program to automatically read mail from an html mail
system.
Thanks,
T
Oct 22 '08 #2


"Tina" <Ti**********@nospamexcite.comwrote in message
news:ub**************@TK2MSFTNGP05.phx.gbl...
How can I programmatically navigate a website just as a user would? click
on buttons, hyperlinks, etc. I know how to read nodes using XPath but I
don't know what classes to use to navigate.

I have to write a program to automatically read mail from an html mail
system.
Thanks,
T
You have a few options:
* Automate IE - you can navigate to a page and use the DOM to fill out text
boxes, press buttons and read results
* Use MSXML2.XmlHttp.6.0 - you can make POST and GET calls and parse the
returned results using the 'HtmlFile' COM class to create an HTML document
* The same as above but using WebRequest and WebResponse from .NET
* Use a tool like Visual Studio - you can record macros that move you
through a site, then you refine them to use inputs from a database and make
decisions according to returned data
--

Joe Fawcett (MVP - XML)
http://joe.fawcett.name

Oct 22 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Matthias Leonhardt | last post: by
3 posts views Thread by greg | last post: by
2 posts views Thread by Monty | last post: by
reply views Thread by NPC403 | last post: by
reply views Thread by zattat | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.