I'm trying to write a web client script in python to log onto a web
page and pull some information off of it. The page has quite a few
behind the scenes http things going on that are making it difficult to
write the python script. I think if I could see the raw http data
that comes to my browser life would be much easier. Does anybody know
of programs that would run alongside my browser and show the http data
that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks
good but it costs quite a bit of money, and the trial is not fully
functional. iewatch has a free trial, but they don't show cookie
info, which is important.
Does anybody know a good way of view the raw http?
Thanks for your help,
Ben 8 2206
"ben" <sa********@yahoo.com> wrote in message
news:56**************************@posting.google.c om... I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
Does anybody know a good way of view the raw http?
Thanks for your help,
Ben
Try HTTPLook at www.httpsniffer.com . We've used it some on our current
project, to monitor SOAP message contents. I don't know about cookie
viewing, but it's free for 15-day (full featured) trial, less than $30 if
you want to keep it.
-- Paul
In article <56**************************@posting.google.com >, ben wrote: I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
from urllib import urlopen
r = urlopen("http://www.python.org/")
print r.headers
You can also take a look at mozilla firefox and download the excellent
webdeveloper extension. It also allows you to view headers and lots of other
stuff.
Hth,
PterK
--
Peter van Kampen
pterk -- at -- datatailors.com sa********@yahoo.com (ben) writes: Does anybody know a good way of view the raw http?
I always just use a proxy server for that. www.junkbuster.org has a
free and convenient one that has options for logging the http stream. sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>... I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
Does anybody know a good way of view the raw http?
Thanks for your help,
Ben
TCPwatch should do what you want. It's a proxy serverso you'll have to
point your browser at that - it shows headers and page content
etc.....
If you're handling cookies then ClientCookie (or cookielib) could well
do what you want without you having to worry about it (if you're using
python why do you need a browser - why can't you just print the
headers ?)
Regards,
Fuzzy http://www.voidspace.org.uk/atlantib...thonutils.html sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>... I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
Does anybody know a good way of view the raw http?
Thanks for your help,
Ben
I downloaded a "sniffer" in Python. Let me know if you are interested
give www.debugbar.com a bash
the trial version is fully functional and it is cheap sa********@yahoo.com (ben) writes: I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
Does anybody know a good way of view the raw http?
[...]
from http://wwwsearch.sourceforge.net/bits/GeneralFAQ.html :
| I want to see what my web browser is doing, but standard network
| sniffers like ethereal or netcat (nc) don't work for HTTPS. How do I
| sniff HTTPS traffic?
|
| Three good options:
|
| Mozilla plugin: livehttpheaders.
| ieHTTPHeaders does the same for MSIE.
| Use lynx -trace, and filter out the junk with a script.
|
| I'm told you can also use a proxy like proxomitron (never tried it
| myself). There's also a commercial MSIE plugin.
URLs: http://www.ethereal.com/ http://livehttpheaders.mozdev.org/ http://www.blunck.info/iehttpheaders.html http://lynx.browser.org/ http://www.proxomitron.info/ http://www.simtec.ltd.uk/
There are plugins that show what session & permanent cookies your
browser has stored, too. Both Mozilla and Konqueror (& Safari and
Opera too, I guess?) do this as standard, but for IE: http://www.codeproject.com/shell/coo...0&select=21591
John
Ben,
I am using iewatch (iewatch.com) quite a bit and it works great in
showing me the cookie information, both for request cookies and
response cookies (look for the Cookie and Set-Cookie headers). There
is a fully functional 21 day trial version available and it only costs
a fraction of what httpwatch is charging.
Michael sa********@yahoo.com (ben) wrote in message news:<56**************************@posting.google. com>... I'm trying to write a web client script in python to log onto a web page and pull some information off of it. The page has quite a few behind the scenes http things going on that are making it difficult to write the python script. I think if I could see the raw http data that comes to my browser life would be much easier. Does anybody know of programs that would run alongside my browser and show the http data that is send back and forth?
I've looked at plugins like httpwatch and iewatch. httpwatch looks good but it costs quite a bit of money, and the trial is not fully functional. iewatch has a free trial, but they don't show cookie info, which is important.
Does anybody know a good way of view the raw http?
Thanks for your help,
Ben This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: lori3laz |
last post by:
How do you disable the right click>view source option on web pages so people
can't view your coding and copy it? What's the html I need to include in my
website to utilize this feature?
Thank...
|
by: Tomas Vera |
last post by:
Hello All,
I'm posting this for the benefit of the search engines.
I've been developing web apps on my computer for about 18 months using VS2002.
This morning, I was showing a colleague how to...
|
by: Brad |
last post by:
I have an intranet app that has just started sporadically getting the
following error
"The viewstate is invalid for this page and might be corrupted."
By sproadic I mean 3-4 times during the past...
|
by: Paolo Pignatelli |
last post by:
I have an aspx code behind page that goes something like this in the HTML
view:
<asp:HyperLink id=HyperLink1 runat="server"
NavigateUrl='<%#"mailto:" &...
|
by: Zack Sessions |
last post by:
Has anyone tried to create a SQL7 view using the CREATE VIEW command
and ADO.NET? If so, is there a trick in trapping a SQL error when
trying to create the view? I have a VB.NET app that, amoung...
|
by: Jacob Donajkowski |
last post by:
Once the user logs in I want to have the users switch from the Roster
View to the Profile View and enter their profile infomation and save
it. Then the next time they login and go to the Profile...
|
by: CBidwill |
last post by:
Hello, I'm a ASP.NET developer with a little experience in IIS but I'm having difficulties getting a little ASP.NET application I wrote up on my home intranet.
Here's the details:
- I can view...
|
by: =?Utf-8?B?UmljaA==?= |
last post by:
I am using a web browser control in my program for bringing up files/folders
in Windows Explorer on a given user's workstatino -- searching special
files/folders on the company server. This works...
|
by: Arno R |
last post by:
Hi all.
When I need to search for pictures, I always have too choose thumbnail-view manually.
Is it possible to open the common dialog in thumbnail-view programmatically?
Example ??
At the...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
| |