473,800 Members | 2,648 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

urllib2 - iteration over non-sequence

im trying to get urllib2 to work on my server which runs python
2.2.1. When i run the following code:
import urllib2
for line in urllib2.urlopen ('www.google.co m'):
print line
i will always get the error:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence
Anyone have any answers?

Jun 9 '07
10 4171
En Sun, 10 Jun 2007 02:54:47 -0300, Erik Max Francis <ma*@alcyone.co m>
escribió:
Gary Herron wrote:
>Certainly there's are cases where xreadlines or read(bytecount) are
reasonable, but only if the total pages size is *very* large. But for
most web pages, you guys are just nit-picking (or showing off) to
suggest that the full read implemented by readlines is wasteful.
Moreover, the original problem was with sockets -- which don't have
xreadlines. That seems to be a method on regular file objects.
There is absolutely no reason to read the entire file into memory (which
is what you're doing) before processing it. This is a good example of
the principle of there is one obvious right way to do it -- and it isn't
to read the whole thing in first for no reason whatsoever other than to
avoid an `x`.
The problem is -and you appear not to have noticed that- that the object
returned by urlopen does NOT have a xreadlines() method; and even if it
had, a lot of pages don't contain any '\n' so using xreadlines would read
the whole page in memory anyway.

Python 2.2 (the version that the OP is using) did include a xreadlines
module (now defunct) but on this case it is painfully slooooooooooooo w -
perhaps it tries to read the source one character at a time.

So the best way would be to use (as Paul Rubin already said):

for line in iter(lambda: f.read(4096), ''): print line

--
Gabriel Genellina

Jun 10 '07 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
21120
by: O. Koch | last post by:
Until now, i know that ftplib doesn't support proxies and that i have to use urllib2. But i don't know how to use the urllib2 correct. I found some examples, but i don't understand them. Is there anyone who can help me?
5
7413
by: Pascal | last post by:
Hello, I want to acces my OWA (Outlook Web Acces - http Exchange interface) server with urllib2 but, when I try, I've always a 401 http error. Can someone help me (and us)? Thanks. here's my pyscript:
0
2079
by: Benjamin Schollnick | last post by:
Folks, With Windows XP, and Python v2.41 I am running into a problem.... The following code gives me an unknown protocol error.... And I am not sure how to resolve it... I have a API written for DocuShare for a non-SSL server, and I wanted to update it
1
3374
by: Ray Slakinski | last post by:
Hello, I have defined a function to set an opener for urllib2, this opener defines any proxy and http authentication that is required. If the proxy has authencation itself and requests an authenticated file I get a HTTP status code of 401 (Unauthorized access of the file being requested) I do see in the headers the Proxy-authorization and the Authorization headers being sent for the request.
6
11679
by: Alejandro Dubrovsky | last post by:
I see from googling around that this is a popular topic, but I haven't seen anyone saying "ah, yes, that works", so here it goes. How does one connect through a proxy which requires basic authorisation? The following code, stolen from somewhere, fails with a 407: proxy_handler = urllib2.ProxyHandler({"http" : "http://the.proxy.address:3128"}) proxy_auth_handler = urllib2.ProxyBasicAuthHandler() proxy_auth_handler.add_password("The...
4
6700
by: Bo Yang | last post by:
Hi , Recently I use python's urllib2 write a small script to login our university gateway . Usually , I must login into the gateway in order to surf the web . So , every time I start my computer , it is my first thing to do that open a browser to login the gateway ! So , I decide to write such a script , sending some post information to the webserver
2
4474
by: Ant | last post by:
Hi all, I have just moved to a new machine, and so have installed the latest version of Python (2.4.3 - previously I believe I was running 2.4.2). Unfortunately this seems to have broken urllib2... An app I wrote for testing our web application makes heavy use of urllib2 against the localhost, and I am getting the following problem (minimal code sample):
1
5762
by: Alessandro Fachin | last post by:
I write this simply code that should give me the access to private page with htaccess using a proxy, i don't known because it's wrong... import urllib,urllib2 #input url url="http://localhost/private/file" #proxy set up
2
12402
by: ken | last post by:
Hi, i have the following code to load a url. My question is what if I try to load an invalide url ("http:// www.heise.de/"), will I get an IOException? or it will wait forever? Thanks for any help. opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) urllib2.install_opener(opener)
1
1995
by: Larry Hale | last post by:
Since it seems I have a "unique" problem, I wonder if anyone could point me in the general/right direction for tracking down the issue and resolving it myself. See my prior post @ http://groups.google.com/group/comp.lang.python/browse_thread/thread/44775994a6b55161?hl=en# for more info. (Python 2.5.2 on Win XP 64 ==>Squid Proxy requiring Authentication ==>Internet not working.) I've looked the urllib2 source over, but am having...
0
10501
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10250
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10032
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9085
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6811
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5469
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5603
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
3764
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2944
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.