By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,602 Members | 1,473 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,602 IT Pros & Developers. It's quick & easy.

view http info when browsing

P: n/a
ben
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

Thanks for your help,

Ben
Jul 18 '05 #1
Share this Question
Share on Google+
8 Replies


P: n/a
"ben" <sa********@yahoo.com> wrote in message
news:56**************************@posting.google.c om...
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

Thanks for your help,

Ben


Try HTTPLook at www.httpsniffer.com . We've used it some on our current
project, to monitor SOAP message contents. I don't know about cookie
viewing, but it's free for 15-day (full featured) trial, less than $30 if
you want to keep it.

-- Paul
Jul 18 '05 #2

P: n/a
In article <56**************************@posting.google.com >, ben wrote:
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.


from urllib import urlopen
r = urlopen("http://www.python.org/")
print r.headers

You can also take a look at mozilla firefox and download the excellent
webdeveloper extension. It also allows you to view headers and lots of other
stuff.

Hth,

PterK
--
Peter van Kampen
pterk -- at -- datatailors.com
Jul 18 '05 #3

P: n/a
sa********@yahoo.com (ben) writes:
Does anybody know a good way of view the raw http?


I always just use a proxy server for that. www.junkbuster.org has a
free and convenient one that has options for logging the http stream.
Jul 18 '05 #4

P: n/a
sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>...
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

Thanks for your help,

Ben

TCPwatch should do what you want. It's a proxy serverso you'll have to
point your browser at that - it shows headers and page content
etc.....

If you're handling cookies then ClientCookie (or cookielib) could well
do what you want without you having to worry about it (if you're using
python why do you need a browser - why can't you just print the
headers ?)

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html
Jul 18 '05 #5

P: n/a
Lad
sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>...
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

Thanks for your help,

Ben


I downloaded a "sniffer" in Python. Let me know if you are interested
Jul 18 '05 #6

P: n/a
give www.debugbar.com a bash

the trial version is fully functional and it is cheap
Jul 18 '05 #7

P: n/a
sa********@yahoo.com (ben) writes:
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

[...]

from http://wwwsearch.sourceforge.net/bits/GeneralFAQ.html :

| I want to see what my web browser is doing, but standard network
| sniffers like ethereal or netcat (nc) don't work for HTTPS. How do I
| sniff HTTPS traffic?
|
| Three good options:
|
| Mozilla plugin: livehttpheaders.
| ieHTTPHeaders does the same for MSIE.
| Use lynx -trace, and filter out the junk with a script.
|
| I'm told you can also use a proxy like proxomitron (never tried it
| myself). There's also a commercial MSIE plugin.

URLs:

http://www.ethereal.com/

http://livehttpheaders.mozdev.org/

http://www.blunck.info/iehttpheaders.html

http://lynx.browser.org/

http://www.proxomitron.info/

http://www.simtec.ltd.uk/
There are plugins that show what session & permanent cookies your
browser has stored, too. Both Mozilla and Konqueror (& Safari and
Opera too, I guess?) do this as standard, but for IE:

http://www.codeproject.com/shell/coo...0&select=21591
John
Jul 18 '05 #8

P: n/a
Ben,

I am using iewatch (iewatch.com) quite a bit and it works great in
showing me the cookie information, both for request cookies and
response cookies (look for the Cookie and Set-Cookie headers). There
is a fully functional 21 day trial version available and it only costs
a fraction of what httpwatch is charging.

Michael

sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>...
I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?

I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.

Does anybody know a good way of view the raw http?

Thanks for your help,

Ben

Jul 18 '05 #9

This discussion thread is closed

Replies have been disabled for this discussion.