By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
457,931 Members | 1,436 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 457,931 IT Pros & Developers. It's quick & easy.

Python Behind a Squid Corporate Proxy on Windows

P: n/a
Greetings, Pythonistas!

My employer has a Squid Proxy between my Python programs and The
Internet.

I've searched high-and-low, and can only find examples online of how
to do basic authentication to connect TO an external Web Server, but
not how to go THROUGH a (corporate) Proxy, thus my question here.
I've tried all sorts of permutations based on my findings, but
(obviously) nothing's working for me. I've come across conflicting
information, including that urllib2 can't/won't do what I'm needing,
though this statement was a couple years old.

I'm running Python 2.5 on Windows XP SP2 64-bit. The Proxy is Squid
(don't know what version, but can find out if this is relevant).

urllib2 grabs the Proxy info fine: 'http': 'http://webfilter.xyz.local:
3128' (.getproxies()). There's the obvious necessity of telling
"something" my ID/pass *somehow*.

An example of one iteration:

######################

import urllib2

proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:@webfilter.xyz.local:3128" } )
opener = urllib2.build_opener( proxy_handler )

urllib2.install_opener( opener )

response = opener.open( "http://python.org" )

print response

######################

I've tried situations using HTTPPasswordMgrWithDefaultRealm and its
add_password method, also to no avail. (Again, THIS sort of thing
seems related to the -external- Web Server, _not_ going *through* a
Proxy. 'Course, I could be wrong. But if I am, I couldn't get this
to work, either. ;)

I've looked at what's happening with Wireshark; Py/my program/urllib2
makes the initial page request, Proxy replies HTTP Error 407: Proxy
Authentication Required, and Py/my program/urllib2 simply stops at
that, reporting the message/error.

Please forgive me; I've been programming in Python for a few years
now, but never had to deal with Web Proxies. I *have* tried to RTFMs,
but the "documentation" for non-basic stuffs with urllib2 is... uh...
well, I'm sure ye olde pro's know... :)
Cheers, and *any* help would be appreciated!
-Larry
Jul 17 '08 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Err, the line above should be:

proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:pa******@webfilter.xyz.local:3128" } )

(Sorry! :)
Jul 17 '08 #2

P: n/a
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
Err, the line above should be:

proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )

(Sorry! *:)
some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me

import urllib2, urllib

proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)

conn = urllib2.urlopen('http://python.org')
return_str = conn.read()

fp = open('python.html','w').write(return_str)
Jul 18 '08 #3

P: n/a
Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry
On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
Err, the line above should be:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
(Sorry! *:)

some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me

import urllib2, urllib

proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)

conn = urllib2.urlopen('http://python.org')
return_str = conn.read()

fp = open('python.html','w').write(return_str)
Jul 18 '08 #4

P: n/a
Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry
On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
Err, the line above should be:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
(Sorry! *:)

some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me

import urllib2, urllib

proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)

conn = urllib2.urlopen('http://python.org')
return_str = conn.read()

fp = open('python.html','w').write(return_str)
Jul 18 '08 #5

P: n/a
Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request, Squid replies "Error 407: Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry
On Jul 18, 3:08*am, Chris <cwi...@gmail.comwrote:
On Jul 17, 6:40*pm, Larry Hale <larz...@hotmail.comwrote:
Err, the line above should be:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
(Sorry! *:)

some old code I wrote to download public domain info for our company,
also through a squid proxy and this still works for me

import urllib2, urllib

proxy = urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)

conn = urllib2.urlopen('http://python.org')
return_str = conn.read()

fp = open('python.html','w').write(return_str)
Jul 18 '08 #6

P: n/a
Hi,

do you manage to go through the proxy? I'm having the same problem
with error 407. I've tried many solutions from the web but it always
show the same error. So, if you solved your problem, please, post it
here.

Thanks
Danilo

On 18 jul, 15:43, Larry Hale <larz...@hotmail.comwrote:
Thank you so much for the reply, but alas: I get the same results.
(urllib2 makes the initial request,Squidreplies "Error 407:Proxy
Authentication Required", which urllib2 dutifully fails on/raises said
error...)

I do appreciate your time, though! :)
-Larry

On Jul 18, 3:08 am, Chris <cwi...@gmail.comwrote:
On Jul 17, 6:40 pm, Larry Hale <larz...@hotmail.comwrote:
Err, the line above should be:
proxy_handler = urllib2.ProxyHandler( { "http": "http://
myusername:passw...@webfilter.xyz.local:3128" } )
(Sorry! :)
some old code I wrote to download public domain info for our company,
also through asquidproxyand this still works for me
import urllib2, urllib
proxy= urllib2.ProxyHandler({'http': 'http://
username:password@proxy_url:proxy_port'})
auth = urllib2.HTTPBasicAuthHandler()
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
conn = urllib2.urlopen('http://python.org')
return_str = conn.read()
fp = open('python.html','w').write(return_str)
Aug 21 '08 #7

This discussion thread is closed

Replies have been disabled for this discussion.