473,406 Members | 2,867 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

loading a url using urllib2

I have the following code to load a url (address).

When I have a url like this,

http://www.testcom.co.uk/dev_12345/www.cnn.com

i get an error "Failed to open http://www.testcom.co.uk/dev_12345/www.cnn.com".
Is there something wrong with my URL? or something wrong with my code?

Thank you for any help.

cj = cookielib.LWPCookieJar()

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(c j))
urllib2.install_opener(opener)
txheaders = {'User-agent': 'Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT)'}
try:
print "address", address
print "txdata", txdata
print "txheaders", txheaders
req = Request(address, txdata, txheaders)
handle = urlopen(req)
except IOError, e:
print address
print 'Failed to open %s' % address
data = ""

Mar 31 '07 #1
1 1354
On Mar 31, 7:21 am, "ying...@gmail.com" <ying...@gmail.comwrote:
I have the following code to load a url (address).

When I have a url like this,

http://www.testcom.co.uk/dev_12345/www.cnn.com

i get an error "Failed to openhttp://www.testcom.co.uk/dev_12345/www.cnn.com".
Is there something wrong with my URL? or something wrong with my code?
With the URL.
See http://www.dnsstuff.com/tools/dnsrep....testcom.co.uk
Thank you for any help.

cj = cookielib.LWPCookieJar()

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(c j))
urllib2.install_opener(opener)

txheaders = {'User-agent': 'Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT)'}
try:
print "address", address
print "txdata", txdata
print "txheaders", txheaders
req = Request(address, txdata, txheaders)
handle = urlopen(req)
I suppose you've added the appropriate imports for Request and
urlopen.
except IOError, e:
print address
print 'Failed to open %s' % address
data = ""
--
Kushal

Mar 31 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Andre Bocchini | last post by:
I'm having some trouble using proxy authentication. I can't figure out how to authenticate with a Squid proxy. I know for a fact the proxy is using Basic instead of Digest for the authentication....
4
by: bmiras | last post by:
I've got a problem using urllib2 to get a web page. I'm going through a proxy using user/password authentification and i'm trying to get a page asking for a HTTP authentification. And I'm using...
2
by: John F Dutcher | last post by:
Can anyone comment on why the code shown in the Python error is in some way incorrect...or is there a problem with Python on my hoster's site ?? The highlites don't seem to show here...but line...
0
by: jacob c. | last post by:
When I request a URL using urllib2, it appears that urllib2 always makes the request using HTTP 1.0, and not HTTP 1.1. I'm trying to use the "If-None-Match"/"ETag" HTTP headers to conserve...
0
by: Pieter Edelman | last post by:
Hi all, I'm trying to submit some data using a POST request to a HTTP server with BASIC authentication with python, but I can't get it to work. Since it's driving me completely nuts, so here's...
11
by: Johnny Lee | last post by:
Hi, I was using urllib to grab urls from web. here is the work flow of my program: 1. Get base url and max number of urls from user 2. Call filter to validate the base url 3. Read the source...
0
by: Phoe6 | last post by:
Hi All, I am able to use urlib2 through proxy. I give proxy credentials and use # Set the Proxy Address proxy_ip = "10.0.1.1:80" proxy_user = 'senthil_or' proxy_password_orig='password'
6
by: Jack | last post by:
I'm trying to use a proxy server with urllib2. So I have managed to get it to work by setting the environment variable: export HTTP_PROXY=127.0.0.1:8081 But I wanted to set it from the code....
3
by: Alexnb | last post by:
Okay, I tried to follow that, and it is kinda hard. But since you obviously know what you are doing, where did you learn this? Or where can I learn this? Maric Michaud wrote:...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.