im trying to get urllib2 to work on my server which runs python
2.2.1. When i run the following code:
import urllib2
for line in urllib2.urlopen ('www.google.co m'):
print line
i will always get the error:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence
Anyone have any answers?
Jun 9 '07
10 4171
En Sun, 10 Jun 2007 02:54:47 -0300, Erik Max Francis <ma*@alcyone.co m>
escribió:
Gary Herron wrote:
>Certainly there's are cases where xreadlines or read(bytecount) are reasonable, but only if the total pages size is *very* large. But for most web pages, you guys are just nit-picking (or showing off) to suggest that the full read implemented by readlines is wasteful. Moreover, the original problem was with sockets -- which don't have xreadlines. That seems to be a method on regular file objects.
There is absolutely no reason to read the entire file into memory (which
is what you're doing) before processing it. This is a good example of
the principle of there is one obvious right way to do it -- and it isn't
to read the whole thing in first for no reason whatsoever other than to
avoid an `x`.
The problem is -and you appear not to have noticed that- that the object
returned by urlopen does NOT have a xreadlines() method; and even if it
had, a lot of pages don't contain any '\n' so using xreadlines would read
the whole page in memory anyway.
Python 2.2 (the version that the OP is using) did include a xreadlines
module (now defunct) but on this case it is painfully slooooooooooooo w -
perhaps it tries to read the source one character at a time.
So the best way would be to use (as Paul Rubin already said):
for line in iter(lambda: f.read(4096), ''): print line
--
Gabriel Genellina This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: O. Koch |
last post by:
Until now, i know that ftplib doesn't support proxies and that i have
to use urllib2. But i don't know how to use the urllib2 correct. I
found some examples, but i don't understand them.
Is there anyone who can help me?
|
by: Pascal |
last post by:
Hello,
I want to acces my OWA (Outlook Web Acces - http Exchange interface)
server with urllib2 but, when I try, I've always a 401 http error.
Can someone help me (and us)?
Thanks.
here's my pyscript:
|
by: Benjamin Schollnick |
last post by:
Folks,
With Windows XP, and Python v2.41 I am running into a problem....
The following code gives me an unknown protocol error.... And I am not
sure how
to resolve it...
I have a API written for DocuShare for a non-SSL server, and I wanted
to update it
|
by: Ray Slakinski |
last post by:
Hello,
I have defined a function to set an opener for urllib2, this opener
defines any proxy and http authentication that is required.
If the proxy has authencation itself and requests an authenticated file
I get a HTTP status code of 401 (Unauthorized access of the file being
requested) I do see in the headers the Proxy-authorization and the
Authorization headers being sent for the request.
|
by: Alejandro Dubrovsky |
last post by:
I see from googling around that this is a popular topic, but I haven't seen
anyone saying "ah, yes, that works", so here it goes.
How does one connect through a proxy which requires basic authorisation?
The following code, stolen from somewhere, fails with a 407:
proxy_handler = urllib2.ProxyHandler({"http" :
"http://the.proxy.address:3128"})
proxy_auth_handler = urllib2.ProxyBasicAuthHandler()
proxy_auth_handler.add_password("The...
| |
by: Bo Yang |
last post by:
Hi ,
Recently I use python's urllib2 write a small script to login our
university gateway .
Usually , I must login into the gateway in order to surf the web . So ,
every time I
start my computer , it is my first thing to do that open a browser to
login the gateway !
So , I decide to write such a script , sending some post information to
the webserver
|
by: Ant |
last post by:
Hi all,
I have just moved to a new machine, and so have installed the latest
version of Python (2.4.3 - previously I believe I was running 2.4.2).
Unfortunately this seems to have broken urllib2...
An app I wrote for testing our web application makes heavy use of
urllib2 against the localhost, and I am getting the following problem
(minimal code sample):
|
by: Alessandro Fachin |
last post by:
I write this simply code that should give me the access to private page with
htaccess using a proxy, i don't known because it's wrong...
import urllib,urllib2
#input url
url="http://localhost/private/file"
#proxy set up
|
by: ken |
last post by:
Hi,
i have the following code to load a url.
My question is what if I try to load an invalide url ("http://
www.heise.de/"), will I get an IOException? or it will wait forever?
Thanks for any help.
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)
|
by: Larry Hale |
last post by:
Since it seems I have a "unique" problem, I wonder if anyone could
point me in the general/right direction for tracking down the issue
and resolving it myself.
See my prior post @ http://groups.google.com/group/comp.lang.python/browse_thread/thread/44775994a6b55161?hl=en#
for more info. (Python 2.5.2 on Win XP 64 ==>Squid Proxy requiring
Authentication ==>Internet not working.)
I've looked the urllib2 source over, but am having...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |