Connecting Tech Pros Worldwide Forums | Help | Site Map

Python Behind a Squid Corporate Proxy on Windows

Larry Hale
Guest
 
Posts: n/a
#1: Jul 17 '08
Greetings, Pythonistas!

My employer has a Squid Proxy between my Python programs and The
Internet.

I've searched high-and-low, and can only find examples online of how
to do basic authentication to connect TO an external Web Server, but
not how to go THROUGH a (corporate) Proxy, thus my question here.
I've tried all sorts of permutations based on my findings, but
(obviously) nothing's working for me. I've come across conflicting
information, including that urllib2 can't/won't do what I'm needing,
though this statement was a couple years old.

I'm running Python 2.5 on Windows XP SP2 64-bit. The Proxy is Squid
(don't know what version, but can find out if this is relevant).

urllib2 grabs the Proxy info fine: 'http': 'http://webfilter.xyz.local:
3128' (.getproxies()). There's the obvious necessity of telling
"something" my ID/pass *somehow*.

An example of one iteration:

######################

import urllib2

proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:@webfilter.xyz.local:3128" } )
opener = urllib2.build_opener( proxy_handler )

urllib2.install_opener( opener )

response = opener.open( "http://python.org" )

print response

######################

I've tried situations using HTTPPasswordMgrWithDefaultRealm and its
add_password method, also to no avail. (Again, THIS sort of thing
seems related to the -external- Web Server, _not_ going *through* a
Proxy. 'Course, I could be wrong. But if I am, I couldn't get this
to work, either. ;)

I've looked at what's happening with Wireshark; Py/my program/urllib2
makes the initial page request, Proxy replies HTTP Error 407: Proxy
Authentication Required, and Py/my program/urllib2 simply stops at
that, reporting the message/error.

Please forgive me; I've been programming in Python for a few years
now, but never had to deal with Web Proxies. I *have* tried to RTFMs,
but the "documentation" for non-basic stuffs with urllib2 is... uh...
well, I'm sure ye olde pro's know... :)


Cheers, and *any* help would be appreciated!
-Larry

Larry Hale
Guest
 
Posts: n/a
#2: Jul 17 '08

re: Python Behind a Squid Corporate Proxy on Windows


Err, the line above should be:

proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:password@webfilter.xyz.local:3128" } )

(Sorry! :)
Chris
Guest
 
Posts: n/a
#3: Jul 18 '08

re: Python Behind a Squid Corporate Proxy on Windows


On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
Quote:
Err, the line above should be:
>
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
>
(Sorry! *:)
some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me

import urllib2, urllib

proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)

conn = urllib2.urlopen('http://python.org')
return_str = conn.read()

fp = open('python.html','w').write(return_str)
Larry Hale
Guest
 
Posts: n/a
#4: Jul 18 '08

re: Python Behind a Squid Corporate Proxy on Windows


Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry


On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
Quote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
>
Quote:
Err, the line above should be:
>
Quote:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
>
Quote:
(Sorry! *:)
>
some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me
>
import urllib2, urllib
>
proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
>
conn = urllib2.urlopen('http://python.org')
return_str = conn.read()
>
fp = open('python.html','w').write(return_str)
Larry Hale
Guest
 
Posts: n/a
#5: Jul 18 '08

re: Python Behind a Squid Corporate Proxy on Windows


Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry


On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
Quote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
>
Quote:
Err, the line above should be:
>
Quote:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
>
Quote:
(Sorry! *:)
>
some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me
>
import urllib2, urllib
>
proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
>
conn = urllib2.urlopen('http://python.org')
return_str = conn.read()
>
fp = open('python.html','w').write(return_str)
Larry Hale
Guest
 
Posts: n/a
#6: Jul 18 '08

re: Python Behind a Squid Corporate Proxy on Windows


Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry


On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
Quote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
>
Quote:
Err, the line above should be:
>
Quote:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
>
Quote:
(Sorry! *:)
>
some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me
>
import urllib2, urllib
>
proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
>
conn = urllib2.urlopen('http://python.org')
return_str = conn.read()
>
fp = open('python.html','w').write(return_str)
Danilo
Guest
 
Posts: n/a
#7: Aug 21 '08

re: Python Behind a Squid Corporate Proxy on Windows


Hi,

do you manage to go through the proxy? I'm having the same problem
with error 407. I've tried many solutions from the web but it always
show the same error. So, if you solved your problem, please, post it
here.

Thanks
Danilo

On 18 jul, 15:43, Larry Hale <larz...@hotmail.comwrote:
Quote:
Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request,Squidreplies "Error 407:Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)
>
I do appreciate your time, though! :)
-Larry
>
On Jul 18, 3:08 am, Chris <cwi...@gmail.comwrote:
>
Quote:
On Jul 17, 6:40 pm, Larry Hale <larz...@hotmail.comwrote:
>
Quote:
Quote:
Err, the line above should be:
>
Quote:
Quote:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
>
Quote:
Quote:
(Sorry! :)
>
Quote:
some old code I wrote to download public domain info for our company,
also through asquidproxyand this still works for me
>
Quote:
import urllib2, urllib
>
Quote:
proxy= urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
>
Quote:
conn = urllib2.urlopen('http://python.org')
return_str = conn.read()
>
Quote:
fp = open('python.html','w').write(return_str)
Closed Thread