473,320 Members | 1,865 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Re: urllib accept-language doesn't have any effect

Hey Philip,

thanks for the snipplet, but I have tried that code already. It does
indeed give me a swedish version.. of www.google.de :) That's the beauty
about Google that they have all languages for all domains available.

However if I try it with www.gizmodo.com (a tech blog in several
languages) I still get the German version.

Both sites obviously redirect the client to the country-based version
according to the IP first, and Google presents that page in the desired
language AFTER that.. most other multihost sites won't have a Swedish
version of the .de site, so this doesn't quit help :(

Thanks anyway,

Martin
>
On Oct 16, 2008, at 6:50 AM, Martin Bachwerk wrote:
>Hmm, thanks for the ideas,

I've checked the requests in Firefox one more time after deleting all
the cookies and both google.com and gizmodo.com do indeed forward me
to the German site without caring about the browser settings.

wget shows me that the server does a 302 redirect straight away.. soo..

I'm not sure what you mean by this. In my experiment with wget, Google
respects the Accept-Language header. On other words, this returns a
Swedish page even though I'm executing it from a U.S. IP address:

wget "--header=Accept-Language: sv" http://www.google.com/
I see the same behavior from urllib2, although my code is slightly
different from yours. Here's my code. If I use "sv" in the header I
get Swedish, "pl" gives me Polish, etc. I get the same result when I
add your Mozilla user-agent string.

----------------------------------------
import urllib2

headers = { "Accept-Language" : "sv" }

req = urllib2.Request("http://www.google.com/", None, headers)
f = urllib2.urlopen(req)
content = f.read()
f.close()

print content
----------------------------------------
Do you get different results with this same code in Germany?

Cheers
Philip
>>
>>>
On Oct 15, 2008, at 9:50 AM, Martin Bachwerk wrote:

Hello,

I'm trying to load a couple of pages using the urllib2 module. The
problem is that I live in Germany and some sites seem to look at
the IP of the client and forward him to a localized page.. Here's
an example of the code, how I want to access google.com main
english page, but get German instead. (For those of you who live in
US, you will probably get correct results.. try emulating with 'fr'
in accepted languages or something)

opener = urllib2.build_opener()
opener.addheaders = [('Host', 'www.google.com'),
('Accept-Language','en-gb,en;q=0.5'), ('User-agent', 'Mozilla/5.0
(Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.1) Gecko/2008070208
Firefox/3.0.1')]
webfile = opener.open(url)

Martin,
It looks to me like what you're sending is correct. Debugging
suggestions --

- Set up a Web server on 127.0.0.1 and see what that server receives
when your Python code connects to it. Maybe you're not sending quite
what you think.
- Try emulating your Python code with wget or a similar command line
tool that lets you set headers.
- Sniff the conversation you're having with google using Wireshark.
Maybe you're getting redirected by the remote server.

Good luck
Philip

Oct 16 '08 #1
1 2379
In message <ma**************************************@python.o rg>, Martin
Bachwerk wrote:
It does indeed give me a swedish version.. of www.google.de :) That's the
beauty about Google that they have all languages for all domains
available.

However if I try it with www.gizmodo.com (a tech blog in several
languages) I still get the German version.
Sounds like a bug in the gizmodo.com site.
Oct 17 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Richard Shea | last post by:
Hi - I'm new to Python. I've been trying to use URLLIB and the 'tidy' function (part of the mx.tidy package). There's one thing I'm having real difficulties understanding. When I did this ... ...
11
by: Pater Maximus | last post by:
I am trying to implement the recipe listed at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/211886 However, I can not get to first base. When I try to run import urllib...
0
by: Shane Hathaway | last post by:
I started experimenting with SOAPpy yesterday and immediately hit a snag. Both web services I tried simply hung and never replied. After a lot of digging, I found out what was going wrong:...
0
by: Pieter Edelman | last post by:
Hi all, I'm trying to submit some data using a POST request to a HTTP server with BASIC authentication with python, but I can't get it to work. Since it's driving me completely nuts, so here's...
1
by: Timothy Wu | last post by:
Hi, I'm trying to fill the form on page http://www.cbs.dtu.dk/services/TMHMM/ using urllib. There are two peculiarities. First of all, I am filling in incorrect key/value pairs in the...
4
by: william | last post by:
I've got a strange problem on windows (not very familiar with that OS). I can ping a host, but cannot get it via urllib (see here under). I can even telnet the host on port 80. Thus network...
6
by: justsee | last post by:
Hi, I'm using Python 2.3 on Windows for the first time, and am doing something wrong in using urllib to retrieve images from urls embedded in a csv file. If I explicitly specify a url and image...
8
by: Gabriel Zachmann | last post by:
Here is a very simple Python script utilizing urllib: import urllib url = "http://commons.wikimedia.org/wiki/Commons:Featured_pictures/chronological" print url print file = urllib.urlopen(...
1
by: evanpmeth | last post by:
I have tried multiple ways of posting information to a website and have failed. I have seen this problem on other forums can someone explain or point me to information on how POST works through...
5
by: supercooper | last post by:
I am downloading images using the script below. Sometimes it will go for 10 mins, sometimes 2 hours before timing out with the following error: Traceback (most recent call last): File...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.