473,231 Members | 1,709 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,231 software developers and data experts.

urllib2.HTTPError: HTTP Error 204: NoContent

I am getting the following error trying to download an html page using
urllib2.

urllib2.HTTPError: HTTP Error 204: NoContent

The url is of this type:

http://www.amazon.com/gp/offer-listi...N%3DB000KJX3A0

I can open it in my browser without problems.Any ideas on a solution?
Oct 19 '08 #1
2 3895

On Oct 19, 2008, at 6:13 AM, silk.odyssey wrote:
I am getting the following error trying to download an html page using
urllib2.

urllib2.HTTPError: HTTP Error 204: NoContent

The url is of this type:

http://www.amazon.com/gp/offer-listi...N%3DB000KJX3A0

I can open it in my browser without problems.Any ideas on a solution?
Are you changing the user-agent? Some sites sniff user agents and
return different results to browsers than to suspected bots.

I'd try it from here if you post a self-contained sample that
demonstrates the problem. Should only take a couple of lines.

Oct 19 '08 #2
On Oct 19, 9:49*am, Philip Semanchuk <phi...@semanchuk.comwrote:
On Oct 19, 2008, at 6:13 AM, silk.odyssey wrote:
I am getting the following error trying to download an html page using
urllib2.
urllib2.HTTPError: HTTP Error 204: NoContent
The url is of this type:
http://www.amazon.com/gp/offer-listi...scriptionId%3D...
I can open it in my browser without problems.Any ideas on a solution?

Are you changing the user-agent? Some sites sniff user agents and *
return different results to browsers than to suspected bots.

I tried it.
>>import urllib2
url = 'http://www.amazon.com/gp/offer-listing/B000KJX3A0%3FSubscriptionId%3D183VXJS74KNQ89D0NRR2 %26tag%3Dws%26linkCode%3Dxm2%26camp%3D2025%26creat ive%3D386001%26creativeASIN%3DB000KJX3A0'
op = urllib2.urlopen(url)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/urllib2.py", line 121, in urlopen
return _opener.open(url, data)
File "/usr/lib/python2.5/urllib2.py", line 380, in open
response = meth(req, response)
File "/usr/lib/python2.5/urllib2.py", line 491, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.5/urllib2.py", line 418, in error
return self._call_chain(*args)
File "/usr/lib/python2.5/urllib2.py", line 353, in _call_chain
result = func(*args)
File "/usr/lib/python2.5/urllib2.py", line 499, in
http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 204: NoContent
>>headers = {}
headers['User-Agent'] = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3'
ro = urllib2.Request(url, None, headers)
op = urllib2.urlopen(ro)
page = op.read()
page
(lots of HTML)

So the answer is as Philip suggests - amazon.com doesn't like 'Python-
urllib/2.5' as a User-Agent. You have to give it something that looks
like a browser.

--
(for email use this address please - you can figure it out)

Mark Sapiro mark at msapiro net Any clod can have the facts;
San Francisco Bay Area, California having opinions is an art. -
C. McCabe, The Fearless
Spectator
Oct 19 '08 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Matthew Wilson | last post by:
I am writing a script to check on my router's external IP address. My ISP refreshes my IP very often and I use dyndns for the hostname for my computer. My Netgear mr814 router has a webserver that...
2
by: John F Dutcher | last post by:
Can anyone comment on why the code shown in the Python error is in some way incorrect...or is there a problem with Python on my hoster's site ?? The highlites don't seem to show here...but line...
1
by: Doug Farrell | last post by:
Hi all, I'm trying to build a web page crawler to help us build our websites, which are driven by static pages after they are called the first time. Anyway, I can use urllib2.urlopen() no...
5
by: Pascal | last post by:
Hello, I want to acces my OWA (Outlook Web Acces - http Exchange interface) server with urllib2 but, when I try, I've always a 401 http error. Can someone help me (and us)? Thanks. ...
0
by: jacob c. | last post by:
When I request a URL using urllib2, it appears that urllib2 always makes the request using HTTP 1.0, and not HTTP 1.1. I'm trying to use the "If-None-Match"/"ETag" HTTP headers to conserve...
0
by: Ali.Sabil | last post by:
hello all, I just maybe hit a bug in both urllib and urllib2, actually urllib doesn't support proxy authentication, and if you setup the http_proxy env var to...
1
by: Alessandro Fachin | last post by:
I write this simply code that should give me the access to private page with htaccess using a proxy, i don't known because it's wrong... import urllib,urllib2 #input url...
1
by: Magnus.Moraberg | last post by:
Hi, I have the following code - import urllib2 from BeautifulSoup import BeautifulSoup proxy_support = urllib2.ProxyHandler({"http":"http:// 999.999.999.999:8080"}) opener =...
6
by: robean | last post by:
Hi everyone, I have a question about using urllib2. I like urllib2 better than urllib at least in part because it has more elaborate support for handling errors: there is built in support for...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.