473,574 Members | 2,393 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Re: urllib accept-language doesn't have any effect

Hey Philip,

thanks for the snipplet, but I have tried that code already. It does
indeed give me a swedish version.. of www.google.de :) That's the beauty
about Google that they have all languages for all domains available.

However if I try it with www.gizmodo.com (a tech blog in several
languages) I still get the German version.

Both sites obviously redirect the client to the country-based version
according to the IP first, and Google presents that page in the desired
language AFTER that.. most other multihost sites won't have a Swedish
version of the .de site, so this doesn't quit help :(

Thanks anyway,

Martin
>
On Oct 16, 2008, at 6:50 AM, Martin Bachwerk wrote:
>Hmm, thanks for the ideas,

I've checked the requests in Firefox one more time after deleting all
the cookies and both google.com and gizmodo.com do indeed forward me
to the German site without caring about the browser settings.

wget shows me that the server does a 302 redirect straight away.. soo..

I'm not sure what you mean by this. In my experiment with wget, Google
respects the Accept-Language header. On other words, this returns a
Swedish page even though I'm executing it from a U.S. IP address:

wget "--header=Accept-Language: sv" http://www.google.com/
I see the same behavior from urllib2, although my code is slightly
different from yours. Here's my code. If I use "sv" in the header I
get Swedish, "pl" gives me Polish, etc. I get the same result when I
add your Mozilla user-agent string.

----------------------------------------
import urllib2

headers = { "Accept-Language" : "sv" }

req = urllib2.Request ("http://www.google.com/", None, headers)
f = urllib2.urlopen (req)
content = f.read()
f.close()

print content
----------------------------------------
Do you get different results with this same code in Germany?

Cheers
Philip
>>
>>>
On Oct 15, 2008, at 9:50 AM, Martin Bachwerk wrote:

Hello,

I'm trying to load a couple of pages using the urllib2 module. The
problem is that I live in Germany and some sites seem to look at
the IP of the client and forward him to a localized page.. Here's
an example of the code, how I want to access google.com main
english page, but get German instead. (For those of you who live in
US, you will probably get correct results.. try emulating with 'fr'
in accepted languages or something)

opener = urllib2.build_o pener()
opener.addhe aders = [('Host', 'www.google.com '),
('Accept-Language','en-gb,en;q=0.5'), ('User-agent', 'Mozilla/5.0
(Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.1) Gecko/2008070208
Firefox/3.0.1')]
webfile = opener.open(url )

Martin,
It looks to me like what you're sending is correct. Debugging
suggestions --

- Set up a Web server on 127.0.0.1 and see what that server receives
when your Python code connects to it. Maybe you're not sending quite
what you think.
- Try emulating your Python code with wget or a similar command line
tool that lets you set headers.
- Sniff the conversation you're having with google using Wireshark.
Maybe you're getting redirected by the remote server.

Good luck
Philip

Oct 16 '08 #1
1 2410
In message <ma************ *************** ***********@pyt hon.org>, Martin
Bachwerk wrote:
It does indeed give me a swedish version.. of www.google.de :) That's the
beauty about Google that they have all languages for all domains
available.

However if I try it with www.gizmodo.com (a tech blog in several
languages) I still get the German version.
Sounds like a bug in the gizmodo.com site.
Oct 17 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
3980
by: Richard Shea | last post by:
Hi - I'm new to Python. I've been trying to use URLLIB and the 'tidy' function (part of the mx.tidy package). There's one thing I'm having real difficulties understanding. When I did this ... finA= urllib.urlopen('http://www.python.org/') foutA=open('C:\\testout.html','w') tidy(finA,foutA,None) I get ...
11
5034
by: Pater Maximus | last post by:
I am trying to implement the recipe listed at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/211886 However, I can not get to first base. When I try to run import urllib fo=urllib.urlopen("http://www.dictionary.com/") page = fo.read() I get:
0
1286
by: Shane Hathaway | last post by:
I started experimenting with SOAPpy yesterday and immediately hit a snag. Both web services I tried simply hung and never replied. After a lot of digging, I found out what was going wrong: urllib.urlopen() is issuing an HTTP/1.0 request, but Microsoft IIS 5 ignores the client HTTP version and replies with an HTTP/1.1 response. This is a...
0
3575
by: Pieter Edelman | last post by:
Hi all, I'm trying to submit some data using a POST request to a HTTP server with BASIC authentication with python, but I can't get it to work. Since it's driving me completely nuts, so here's my cry for help. The server is an elog logbook server (http://midas.psi.ch/elog/). It is protected with a password and an empty username. I can...
1
2063
by: Timothy Wu | last post by:
Hi, I'm trying to fill the form on page http://www.cbs.dtu.dk/services/TMHMM/ using urllib. There are two peculiarities. First of all, I am filling in incorrect key/value pairs in the parameters on purpose because that's the only way I can get it to work.. For "version" I am suppose to leave it unchecked, having value of empty string....
4
3582
by: william | last post by:
I've got a strange problem on windows (not very familiar with that OS). I can ping a host, but cannot get it via urllib (see here under). I can even telnet the host on port 80. Thus network seems good, but not for python ;-(. Does any windows specialist can guide me (a poor linux user) to get Network functionalitiies with python on...
6
5813
by: justsee | last post by:
Hi, I'm using Python 2.3 on Windows for the first time, and am doing something wrong in using urllib to retrieve images from urls embedded in a csv file. If I explicitly specify a url and image name it works fine(commented example in the code), but if I pass in variables in this for loop it throws errors: --- The script: import csv,...
8
3486
by: Gabriel Zachmann | last post by:
Here is a very simple Python script utilizing urllib: import urllib url = "http://commons.wikimedia.org/wiki/Commons:Featured_pictures/chronological" print url print file = urllib.urlopen( url ) mime = file.info() print mime
1
496
by: evanpmeth | last post by:
I have tried multiple ways of posting information to a website and have failed. I have seen this problem on other forums can someone explain or point me to information on how POST works through urllib an different broweser (what is the difference). my first attempt was out of the docs: Code: import httplib, urllib params =...
5
4663
by: supercooper | last post by:
I am downloading images using the script below. Sometimes it will go for 10 mins, sometimes 2 hours before timing out with the following error: Traceback (most recent call last): File "ftp_20070326_Downloads_cooperc_FetchLibreMapProjectDRGs.py", line 108, i n ? urllib.urlretrieve(fullurl, localfile) File "C:\Python24\lib\urllib.py", line...
0
7794
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7710
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8040
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
8092
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6437
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
5296
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3738
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3745
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1332
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.