HTTP Listener??

[Rusty Hill]

Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.

[Daniel Cigic]

IE Developer Toolbar

Rusty Hill wrote:

Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.

[Nicholas Paldino [.NET/C# MVP]]

Rusty,

There are a few programs out there that will let you do this (I'm sure
there is even a firefox extension if that is your browser of choice).

For any kind of web browser though, you can use something like Fiddler
(which is free). All you have to do is run the program, and then set your
proxy settings on your web browser to point to the port and ip that the
interceptor is running on (usually the local host).
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Rusty Hill" <Ru********@hotmail.comwrote in message
news:uE**************@TK2MSFTNGP02.phx.gbl...

Is there some tool that I can use to profile all the activity HTML,
cookies, headers, etc. that are sent back and forth to a site when I
access it using my web browser? Essentially I want to navigate a site
manually in my browser and get enough information so I can program
something to crawl the site in an automated fashion but need to know what
headers and stuff to send in my request. Thank you.

[Samuel R. Neff]

Charles will record an HTTP session (set of requests) and then replay
them. That sounds like what you're asking for, right?

http://www.xk72.com/charles/

I haven't used that aspec of Charles but in general it's a wonderful
app.

Sam

------------------------------------------------------------
We're hiring! B-Line Medical is seeking .NET
Developers for exciting positions in medical product
development in MD/DC. Work with a variety of technologies
in a relaxed team environment. See ads on Dice.com.

On Fri, 1 Jun 2007 16:23:26 -0400, "Rusty Hill"
<Ru********@hotmail.comwrote:

>Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.

[RedGrittyBrick]

Rusty Hill wrote:

Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.

Wireshark?

[Steve C. Orr [MCSD, MVP, CSM, ASP Insider]]

I suggest Fiddler.
It's free and good.
http://www.fiddler2.com

--
I hope this helps,
Steve C. Orr,
MCSD, MVP, CSM, ASPInsider
http://SteveOrr.net
"Rusty Hill" <Ru********@hotmail.comwrote in message
news:uE**************@TK2MSFTNGP02.phx.gbl...

Is there some tool that I can use to profile all the activity HTML,
cookies, headers, etc. that are sent back and forth to a site when I
access it using my web browser? Essentially I want to navigate a site
manually in my browser and get enough information so I can program
something to crawl the site in an automated fashion but need to know what
headers and stuff to send in my request. Thank you.