I am writing a webcrawler, so to speak, and for some reason, this one
particular type of page on a site is returning a 400 (bad request), but I
can view the page in IE. Other pages on the site works. Now the issue with
this site is that it uses the "referrer" to determine if the proper links
were followed. What I don't know is if there are other fields header fields
that might be important, or other stuff like cookies.
Is there any app out there that will hook IE or my internet connection and
allow me to see the entire HTTP request, including headers and cookies so
that I can see what happens when I navigate through the site? 3 1078
look at network trace tool. (netmon comes with SMS).
-- bruce (sqlwork.com)
"Peter Rilling" <pe***@nospam.rilling.net> wrote in message
news:ur****************@tk2msftngp13.phx.gbl... I am writing a webcrawler, so to speak, and for some reason, this one particular type of page on a site is returning a 400 (bad request), but I can view the page in IE. Other pages on the site works. Now the issue with this site is that it uses the "referrer" to determine if the proper links were followed. What I don't know is if there are other fields header fields that might be important, or other stuff like cookies.
Is there any app out there that will hook IE or my internet connection and allow me to see the entire HTTP request, including headers and cookies so that I can see what happens when I navigate through the site?
Check out ieHttpHeaders, Peter. http://www.blunck.info/iehttpheaders.html
Juan T. Llibre, ASP.NET MVP
ASP.NET FAQ : http://asp.net.do/faq/
ASPNETFAQ.COM : http://www.aspnetfaq.com/
Foros de ASP.NET en Español : http://asp.net.do/foros/
======================================
"Peter Rilling" <pe***@nospam.rilling.net> wrote in message
news:ur****************@tk2msftngp13.phx.gbl... I am writing a webcrawler, so to speak, and for some reason, this one particular type of page on a site is returning a 400 (bad request), but I can view the page in IE. Other pages on the site works. Now the issue with this site is that it uses the "referrer" to determine if the proper links were followed. What I don't know is if there are other fields header fields that might be important, or other stuff like cookies.
Is there any app out there that will hook IE or my internet connection and allow me to see the entire HTTP request, including headers and cookies so that I can see what happens when I navigate through the site?
Peter Rilling wrote: I am writing a webcrawler, so to speak, and for some reason, this one particular type of page on a site is returning a 400 (bad request), but I can view the page in IE. Other pages on the site works. Now the issue with this site is that it uses the "referrer" to determine if the proper links were followed. What I don't know is if there are other fields header fields that might be important, or other stuff like cookies.
Is there any app out there that will hook IE or my internet connection and allow me to see the entire HTTP request, including headers and cookies so that I can see what happens when I navigate through the site? www.fiddlertool.com is my favorite.
Cheers,
-- http://www.joergjooss.de
mailto:ne********@joergjooss.de This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Michael Foord |
last post by:
#!/usr/bin/python -u
# 15-09-04
# v1.0.0
# auth_example.py
# A simple script manually demonstrating basic authentication.
# Copyright Michael Foord
# Free to use, modify and relicense.
#...
|
by: Andreas Klemt |
last post by:
Hello,
I get this error Message "cannot redirect after http headers have been sent"
when I do this
response.redirect ("home.aspx")
How can I find out with vb.net if already a http header has...
|
by: Clif |
last post by:
Hi,
I would like to find out exactly what is being sent when a web page uses the
post request method. My thought was to use the webbrowser control together
with the HTTPWebRequest class, but am...
|
by: Gabriel Lozano-Morán |
last post by:
Here is my problem description:
We have a Java Portal from there a user can click on a certain aspx page.
There is some kind of authentication mechanism that can be compared to a...
|
by: jl |
last post by:
I have a J2ME cell phone application that via the AMS (Application
Manager) delivers a status report to my server indicating whether the
suite was successfully installed and identifying the reason...
|
by: omantawy |
last post by:
Hi,
I have some legacy ASP web applications that use an unmanaged COM
component to connect to a third party application. The third part
application has moved to the managed code in the current...
|
by: Boldgeek |
last post by:
I am trying to develop an app that will allow automatic updating of a
web form which uses multipart/form-data enctype (as it MIGHT be sending
an image)
I have an example form, which when...
|
by: Reporter |
last post by:
Here is an example from the PHP Manual
<?php
if ((!isset($_SERVER)) || (1==1)) {
header('WWW-Authenticate: Basic realm="My Realm"');
header('HTTP/1.0 401 Unauthorized');
echo 'Text to send...
|
by: JohnH |
last post by:
Hi,
In my application which has two or more threads calling web
service or just HttpWebRequest I am seeing some HTTP 400 errors retrun
in the response. All the calls are going through the same ISA...
|
by: CindyH |
last post by:
Hi
I'm not sure whether I should send this as a new message or use the one I've
been using but...
I'm using vb.net 2.0 -
My problem is I need to send
something like this: 'dim encodedstring =...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: jfyes |
last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
|
by: ArrayDB |
last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: Shællîpôpï 09 |
last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
|
by: Faith0G |
last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
| |