https proxy

[Paul Sweeney]

Does anyone know of a working (python) https proxy which allows viewing of
unencrypted data being sent from my browser to an https site?

I've worked my way through most on the list at
http://xhaus.com/alan/python/proxies.html, but while many claim to support
https, if you actually point your browser at the proxies, they work fine for
http, but not for https pages.

TIA

Paul Sweeney

[Simon Dahlbacka]

Paul Sweeney wrote:

Does anyone know of a working (python) https proxy which allows viewing of
unencrypted data being sent from my browser to an https site?

hmm, I thought the _purpose_ of using https was to make it relatively
impossible to view the unencrypted data being the "man in the middle"..

[Paul Sweeney]

Simon Dahlbacka wrote:

hmm, I thought the _purpose_ of using https was to make it relatively
impossible to view the unencrypted data being the "man in the middle"..

It's certainly not impossible, there are tools like Paros for java which do
the job, the browser sets up an http connection with the proxy (using the
proxy's built in certificate), and the proxy then sets up an https
connection with the destination server, but the data is unencrypted in the
proxy before being re-encrypted to send to the destination server.

What is (virtually) impossible is to intercept and do a "man in the middle"
attack on an existing connect. I don't want to intercept stuff on the net,
just see what the browser on my machine is sending/receiving

HTH :-)

[Peter Hansen]

Paul Sweeney wrote:

Simon Dahlbacka wrote:

hmm, I thought the _purpose_ of using https was to make it relatively
impossible to view the unencrypted data being the "man in the middle"..

It's certainly not impossible, there are tools like Paros for java which do
the job, the browser sets up an http connection with the proxy (using the
proxy's built in certificate), and the proxy then sets up an https
connection with the destination server, but the data is unencrypted in the
proxy before being re-encrypted to send to the destination server.

What is (virtually) impossible is to intercept and do a "man in the middle"
attack on an existing connect. I don't want to intercept stuff on the net,
just see what the browser on my machine is sending/receiving

It sounds like you want either to see the raw data stream (the
encrypted stuff), or you want to see the unencrypted data that
the browser would be sending if it weren't using https. It's
still unclear. If the latter, why not use Paros, since you seem
to know about it and how it works?

(The reason your request is unclear is because your first message
talks about seeing the "unencrypte d data being sent from [your]
browser to an https site" and yet obviously, as you know, there
is no unencrypted data going to the https site... But since you
certainly know this, it makes it unclear just which you are
requesting.)

-Peter

[Paul Sweeney]

Peter Hansen wrote:

Paul Sweeney wrote:

Simon Dahlbacka wrote:

hmm, I thought the _purpose_ of using https was to make it relatively
impossible to view the unencrypted data being the "man in the middle"..

It's certainly not impossible, there are tools like Paros for java which do the job, the browser sets up an http connection with the proxy (using the proxy's built in certificate), and the proxy then sets up an https
connection with the destination server, but the data is unencrypted in the proxy before being re-encrypted to send to the destination server.

What is (virtually) impossible is to intercept and do a "man in the middle" attack on an existing connect. I don't want to intercept stuff on the net, just see what the browser on my machine is sending/receiving

It sounds like you want either to see the raw data stream (the
encrypted stuff), or you want to see the unencrypted data that
the browser would be sending if it weren't using https. It's
still unclear. If the latter, why not use Paros, since you seem
to know about it and how it works?

(The reason your request is unclear is because your first message
talks about seeing the "unencrypte d data being sent from [your]
browser to an https site" and yet obviously, as you know, there
is no unencrypted data going to the https site... But since you
certainly know this, it makes it unclear just which you are
requesting.)

-Peter

Ok, apologies, let me try to explain it more clearly.

I wish to access an https:// site using my browser and click around it. I
wish to see the http(s) GET urls and http(s) POST urls and data which are
being sent to the https server. If the site were an http:// site I would
use a tool like ethereal to see what was going on, but this data is sent
encrypted from the browser due to the https connection, so I'll just see the
encrypted data. So I need a tool to view the unencrypted form of the data
which is being sent encrypted (sorry for being unclear about the "unecrypted
data" in my last post, I hope this is better).

I have and could use a tool like Paros, but it is java and (need I say
more?):

a) I love Python (:-D) ...
b) ... seriously, I'd like to be able to play with the source code to record
the gets and posts for later replay in a python based retriever tool.

Thanks for your interest in this thread.

Paul Sweeney

[Peter Hansen]

Paul Sweeney wrote:

I wish to access an https:// site using my browser and click around it. I
wish to see the http(s) GET urls and http(s) POST urls and data which are
being sent to the https server. If the site were an http:// site I would
use a tool like ethereal to see what was going on, but this data is sent
encrypted from the browser due to the https connection, so I'll just see the
encrypted data. So I need a tool to view the unencrypted form of the data
which is being sent encrypted (sorry for being unclear about the "unecrypted
data" in my last post, I hope this is better).
Based on what you say here, I think the answer is back to being
"impossible ". The encryption is going on in the browser. If
the browser thinks it's talking HTTPS, then it will encrypt
stuff and talk HTTPS. Nothing between the browser end of the
socket and the server end of the socket will be able to
decrypt it.

If you had a proxy that used its own certificate and established
an https connection when the client tried to connect with http
to something, then you could at least start the whole thing off
without encrypted data on the browser side. As soon as you tried
to click on an embedded https link, however, the browser would
try to establish an encrypted link, and the proxy wouldn't work
any more.

What about just writing a tool using urllib2 and basically
simulating the browser activity in Python? Then you can
bypass the whole issue, and have programmatic access to the
unencrypted data before or after it hits the encrypted socket
(depending on the direction of travel).
I have and could use a tool like Paros, but it is java and (need I say
more?):

a) I love Python (:-D) ...
b) ... seriously, I'd like to be able to play with the source code to record
the gets and posts for later replay in a python based retriever tool.

I love Python too, but that doesn't stop me from using a more
effective tool when the need arises. Surely something like Paros,
if it can do what you need (and I don't see how it can), also
has the ability to save the data to a file for later perusal,
even if the utility for retrieving it were in Python.

-Peter

[Dietmar Schwertberger]

In article <ce**********@t itan.btinternet .com>, Paul Sweeney
<URL:mailto:re* *************** ***@nothypgnal. delrest.co.uk> wrote:

Does anyone know of a working (python) https proxy which allows viewing of
unencrypted data being sent from my browser to an https site?

I've worked my way through most on the list at
http://xhaus.com/alan/python/proxies.html, but while many claim to support
https, if you actually point your browser at the proxies, they work fine for
http, but not for https pages.

I wrote a http proxy server some time ago to use Netscape/Firefox for
homebanking through the modem connection of my RISC OS computer (I don't
like connecting my Windows PC to the internet....).
I don't think a simple proxy server will help you at all as it just waits
for the browser connecting, receives a "CONNECT ..." request, builds the
requested connection and then just passes on the data between the browser
and the website until the connection is closed from either side ...

Anyway it's available from my homepage; it's in the RISC OS additions
archive - the name is SSLRelay.

Regards,

Dietmar

[Paul Rubin]

"Paul Sweeney" <re************ *******@nothypg nal.delrest.co. uk> writes:

I wish to access an https:// site using my browser and click around
it. I wish to see the http(s) GET urls and http(s) POST urls and
data which are being sent to the https server. If the site were an
http:// site I would use a tool like ethereal to see what was going
on, but this data is sent encrypted from the browser due to the
https connection, so I'll just see the encrypted data. So I need a
tool to view the unencrypted form of the data which is being sent
encrypted (sorry for being unclear about the "unecrypted data" in my
last post, I hope this is better).

You could do that but I don't know of already-existing Python code
that does it. Basically it would be a man-in-the-middle attack against
the browser/server pair:

browser---(SSL)---proxy---(SSL)---server

There'd be two separate SSL sessions and the proxy would of course
have to present its own server certificate to the browser, which you'd
configure the browser to accept. The server application could take
various measures to detect this subterfuge and stop it from happening,
but I doubt any normal server apps actually bother to do that. It's
not a true security break, since the browser user has to accept the
proxy certificate, thereby giving "permission " for the interception.

[John Hazen]

* Peter Hansen <pe***@engcorp. com> [2004-07-27 13:55]:

Paul Sweeney wrote:

I wish to access an https:// site using my browser and click around it. I
wish to see the http(s) GET urls and http(s) POST urls and data which are
being sent to the https server. If the site were an http:// site I would
use a tool like ethereal to see what was going on, but this data is sent
encrypted from the browser due to the https connection, so I'll just see
the
encrypted data. So I need a tool to view the unencrypted form of the data
which is being sent encrypted (sorry for being unclear about the
"unecrypted
data" in my last post, I hope this is better).

Based on what you say here, I think the answer is back to being
"impossible ". The encryption is going on in the browser. If
the browser thinks it's talking HTTPS, then it will encrypt
stuff and talk HTTPS. Nothing between the browser end of the
socket and the server end of the socket will be able to
decrypt it.

Although, if this is just for testing, I think you could trick the
browser by running a proxy on a different machine that accepts an HTTPS
connection, and creates another HTTPS connection with the destination
server. Then run your own DNS server that hands back bogus answers to
every request resolving to the proxy's IP address.

The browser will talk HTTPS to the proxy, thinking it's talking to the
target server. Of course, this will trigger the "certificat e doesn't
match IP address" type of warning, but I think most browsers will let
you continue after warning you. (If not, you could probably still get
around it with the gross hack of having the DNS server communicate with
the proxy, so the proxy could generate on the fly a certificate for the
server name that the browser is trying to reach. This would get you the
"certificat e can't be verified" type of error, which is definitely one
that you can continue after acknowledging.)

So, if it's just for testing and research, that might work. Obviously,
it's not a viable setup if you're trying to use it to snoop on the
conversations of unsuspecting users (which I'm not accusing you of, but
it's a much harder problem).

-John

[Paul Rubin]

jo**@hazen.net (John Hazen) writes:

Although, if this is just for testing, I think you could trick the
browser by running a proxy on a different machine that accepts an HTTPS
connection, and creates another HTTPS connection with the destination
server. Then run your own DNS server that hands back bogus answers to
every request resolving to the proxy's IP address.
You can use /etc/hosts for that. No need for running your own DNS.
But why bother? Just configure the browser send requests through the
proxy.
The browser will talk HTTPS to the proxy, thinking it's talking to the
target server. Of course, this will trigger the "certificat e doesn't
match IP address" type of warning, but I think most browsers will let
you continue after warning you.

Yes, or you can generate a cert with the right hostname in it, and
install it in the browser as a trusted cert.

To the OP, are you trying to debug a server-side app? If you're not
trying to send actual confidential data through the SSL connection and
if you can configure the server, it's probably simplest to just open a
non-SSL port, unless you're trying to debug something SSL-specific.