473,372 Members | 814 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,372 software developers and data experts.

urllib urllib2 what is the difference ?

Could somebody please explain the difference between these two modules
and explain why they are both required, and if there will ever be a
unification of them?
Jul 18 '05 #1
1 4076
On 7 Jan 2004 04:30:34 -0800, ch********@spritenote.co.uk (Chris Lyon)
Could somebody please explain the difference between these two modules
and explain why they are both required, and if there will ever be a
unification of them?

The main difference for me is that urllib2.urlopen can accept as an
argument, not only a URL name, but an instance of a Request class. The
Request class allows you to specify headers in the URL request. This is
helpful, for example, if you are retrieving a URL from a site that takes
different actions based on, say, the referral URL or the name of your
"browser", both of which are specified in request headers.

urllib.urlopen can only take a url string, and doesn't allow you (as far
as I can tell) to specify headers.

Another difference is the whole build_opener stuff that urllib2 provides,
but I've never had occasion to use or try to understand that, so I'll
leave that to someone else to explain.

On the flip side, urllib implements some things (like urlretrieve) that
urllib2 does not have; so urllib2 is not just a stepped-up urllib. That's
probably the reason they're not unified, at least for now (I have no
insight on future features).
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Max Khesin | last post by:
Is there a way to access an htaccess-protected directory with urllib, password being known? thanks, max -- ======================================== Max Khesin, software developer -...
by: Gary Feldman | last post by:
I think I've found a deficiency in the design of urllib related to https. In order to complete an https connection, it appears that URLOpener and hence FancyURLOpener require the key and cert...
by: Volker M. | last post by:
Hey, I want to open a list of URLs with Pythons urllib and the fuction open(URL) automatically. It is important that the program open ONLY normal http-sites and no https-sites with...
by: Pieter Edelman | last post by:
Hi all, I'm trying to submit some data using a POST request to a HTTP server with BASIC authentication with python, but I can't get it to work. Since it's driving me completely nuts, so here's...
by: Johnny Lee | last post by:
Hi, I was using urllib to grab urls from web. here is the work flow of my program: 1. Get base url and max number of urls from user 2. Call filter to validate the base url 3. Read the source...
by: Ali.Sabil | last post by:
hello all, I just maybe hit a bug in both urllib and urllib2, actually urllib doesn't support proxy authentication, and if you setup the http_proxy env var to...
by: John Nagle | last post by:
I thought I had all the timeout problems with urllib worked around, but no. socket.setdefaulttimeout is useful, but not always effective. I'm setting that to 15 seconds. If the host end won't...
by: O.R.Senthil Kumaran | last post by:
Hi, There is an Open Tracker item against urllib2 library python.org/sf/735515 which states that. urllib / urllib2 should cache the results of 301 (permanent) redirections. This shouldn't break...
by: johnpollard | last post by:
For some reason this script isn't working and I dont know what it is. I believe the problem lies in the following lines of code since the script works with a different website and username/password...
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.