Until now, i know that ftplib doesn't support proxies and that i have
to use urllib2. But i don't know how to use the urllib2 correct. I
found some examples, but i don't understand them.
Is there anyone who can help me? 4 21074
You need to install a proxyhandler, authhandler,
ftphandler and httphandler. Then build yourself
an opener, that opens the doors for you ... :-)
The following does the trick.
proxy_handler = urllib2.ProxyHandler( {'http': 'myhttpproxy:80',
'https' : 'myhttpsproxy:443',
'ftp' : 'myftpproxy:21' } )
opener= urllib2.build_opener(proxy_handler, urllib2.HTTPBasicAuthHandler(),
urllib2.HTTPHandler, urllib2.HTTPSHandler,
urllib2.FTPHandler)
# install this opener
urllib2.install_opener(opener)
# Go ahead, knock knock!
req=urlli2.Request('ftp://ftp.gnu.org')
data=urllib2.urlopen(req).read()
Of course, replace the arbit proxy values I wrote
with your proxy values. If your proxy need authentication
, you will need to do a bit more here.
proxyauth='http://' + username + '@' + password + 'myproxy:myproxport'
Then your proxy handler becomes ( I am assuming a generic proxy
for all protocols here!)
proxy_handler = urllib2.ProxyHandler ( {'http' : proxyauth,
'https' : proxyauth,
'ftp' : proxyauth } )
HTH.
-Anand jj*@pobox.com (John J. Lee) wrote in message news:<87************@pobox.com>... Za***********@gmx.de (O. Koch) writes:
Until now, i know that ftplib doesn't support proxies and that i have to use urllib2. But i don't know how to use the urllib2 correct. I found some examples, but i don't understand them.
Is there anyone who can help me?
import urllib2 response = urllib2.urlopen("ftp://ftp.example.com/pub/myfile") data = response.read() response.close()
Does that do the trick?
John py*******@Hotpop.com (Anand Pillai) writes: You need to install a proxyhandler, authhandler, ftphandler and httphandler. Then build yourself an opener, that opens the doors for you ... :-)
The following does the trick.
proxy_handler = urllib2.ProxyHandler( {'http': 'myhttpproxy:80', 'https' : 'myhttpsproxy:443', 'ftp' : 'myftpproxy:21' } )
opener= urllib2.build_opener(proxy_handler, urllib2.HTTPBasicAuthHandler(), urllib2.HTTPHandler, urllib2.HTTPSHandler, urllib2.FTPHandler)
# install this opener urllib2.install_opener(opener)
# Go ahead, knock knock!
req=urlli2.Request('ftp://ftp.gnu.org') data=urllib2.urlopen(req).read()
A couple of things to add: you don't need to add handlers that already
get added by default by build_opener (FTPHandler and HTTPHandler, for
example). ProxyHandler is one of these default handlers, so if your
environment is set up for it (http_proxy, etc. environment variables),
you don't need to supply a ProxyHandler (of course, if your
environ. *isn't*, then you do need to supply one, to give it the proxy
details). You don't need Request objects (unless you want to add
headers to a Request, or pass Requests around). You don't need to
install a global opener, unless your code expects it -- it's just a
convenience (or an inconvenience, sometimes).
Actually, did the OP say proxy basic auth. was involved? Don't
recall. I've never needed it for proxies, but there seems to be a
ProxyBasicAuthHandler in urllib2, so I guess that's what you meant to
use, rather than HTTPBasicAuthHandler (which is for website auth., not
proxy auth).
So, after all that, you end up with:
opener = urllib2.build_opener(urllib2.ProxyBasicAuthHandler )
data = opener.open('ftp://ftp.gnu.org').read()
(I like to close the response explicitly, though)
John jj*@pobox.com (John J. Lee) wrote in message news:<87************@pobox.com>... py*******@Hotpop.com (Anand Pillai) writes:
The code obviously caters to the bottom-line, in case
the programmer does not want to worry about environment
variables, would like to install the handlers himself
and his proxy needs authentication.
In the last case, the HTTP_PROXY env variable does not
help and you need to install the proxy handler yourself.
I copied this stuff from my program which needs all these
and more ( a USER-AGENT header for example ).
-Anand You need to install a proxyhandler, authhandler, ftphandler and httphandler. Then build yourself an opener, that opens the doors for you ... :-)
The following does the trick.
proxy_handler = urllib2.ProxyHandler( {'http': 'myhttpproxy:80', 'https' : 'myhttpsproxy:443', 'ftp' : 'myftpproxy:21' } )
opener= urllib2.build_opener(proxy_handler, urllib2.HTTPBasicAuthHandler(), urllib2.HTTPHandler, urllib2.HTTPSHandler, urllib2.FTPHandler)
# install this opener urllib2.install_opener(opener)
# Go ahead, knock knock!
req=urlli2.Request('ftp://ftp.gnu.org') data=urllib2.urlopen(req).read()
A couple of things to add: you don't need to add handlers that already get added by default by build_opener (FTPHandler and HTTPHandler, for example). ProxyHandler is one of these default handlers, so if your environment is set up for it (http_proxy, etc. environment variables), you don't need to supply a ProxyHandler (of course, if your environ. *isn't*, then you do need to supply one, to give it the proxy details). You don't need Request objects (unless you want to add headers to a Request, or pass Requests around). You don't need to install a global opener, unless your code expects it -- it's just a convenience (or an inconvenience, sometimes).
Actually, did the OP say proxy basic auth. was involved? Don't recall. I've never needed it for proxies, but there seems to be a ProxyBasicAuthHandler in urllib2, so I guess that's what you meant to use, rather than HTTPBasicAuthHandler (which is for website auth., not proxy auth).
So, after all that, you end up with:
opener = urllib2.build_opener(urllib2.ProxyBasicAuthHandler ) data = opener.open('ftp://ftp.gnu.org').read()
(I like to close the response explicitly, though)
John py*******@Hotpop.com (Anand Pillai) writes: jj*@pobox.com (John J. Lee) wrote in message news:<87************@pobox.com>... py*******@Hotpop.com (Anand Pillai) writes: The code obviously caters to the bottom-line, in case the programmer does not want to worry about environment variables, would like to install the handlers himself and his proxy needs authentication.
Sure, I understand that. I was just elaborating, no criticism was
intended!
It seems people are frightened to use urllib2 (perhaps partly because
the documentation has lots of sections -- though they're all very
short), so I like to point out that most of the time it's just a
matter of:
import urllib2
urllib2.urlopen("http://example.com/")
And most of the rest of the time, the only complications above that
are passing a Request instead of a URL (for adding headers and passing
around requests as first-class objects), and building your own opener
with build_opener (for choosing what features you want and don't want,
like authentication -- and adding your own features, of course).
John This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: bmiras |
last post by:
I've got a problem using urllib2 to get a web page.
I'm going through a proxy using user/password authentification
and i'm trying to get a page asking for a HTTP authentification.
And I'm using...
|
by: Fuzzyman |
last post by:
urllib2 (under windows) will auto-detect your proxy settings and use
those.
Normally that's a good thing (I guess), except when it's not !
How do I switch off this behaviour ? I'm behind a...
|
by: Ray Slakinski |
last post by:
Hello,
I have defined a function to set an opener for urllib2, this opener
defines any proxy and http authentication that is required.
If the proxy has authencation itself and requests an...
|
by: Alejandro Dubrovsky |
last post by:
I see from googling around that this is a popular topic, but I haven't seen
anyone saying "ah, yes, that works", so here it goes.
How does one connect through a proxy which requires basic...
|
by: itay_k |
last post by:
Hi,
I am running the following simple code (just open connection to some
https page with proxy):
proxy= '666.179.227.666:80'
proxy=urllib2.ProxyHandler({"https":'https://'+proxy})
opener =...
| |
by: Alessandro Fachin |
last post by:
I write this simply code that should give me the access to private page with
htaccess using a proxy, i don't known because it's wrong...
import urllib,urllib2
#input url...
|
by: cp.finances.gouv |
last post by:
Hello all,
I'm facing a strange behavior of urllib2 trying to access gmail
account behind a proxy (Squid).
The following works perfectly :
wget --save-cookies cookies...
|
by: Astan Chee |
last post by:
Hi,
Im trying to implement the logic from
http://www.hypothetic.org/docs/msn/general/http_connections.php to a
simple python code using urllib2 and some parts of urllib. Im behind a
http proxy...
|
by: Jack |
last post by:
I'm trying to use a proxy server with urllib2.
So I have managed to get it to work by setting the environment
variable:
export HTTP_PROXY=127.0.0.1:8081
But I wanted to set it from the code....
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...
| |