473,322 Members | 1,307 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

how to judge urllib.Request is finished?

currently I am using
Expand|Select|Wrap|Line Numbers
  1. req=urllib2.Request(url)
  2. data=''
  3. if '</htm>' not in data:
  4. fd=urllib2.urlopen(req)
  5. data=fd.read()
  6. time.sleep(10)
  7. time.sleep(10)
  8. blahblah
  9.  
Is there any other ready-to-use function? And what if the internet
connection is chocked so that the html page can not be loaded
entirely?
Jul 1 '08 #1
1 1045
On Tue, 01 Jul 2008 15:16:03 +0800, oyster wrote:
currently I am using
Expand|Select|Wrap|Line Numbers
  1. req=urllib2.Request(url)
  2. data=''
  3. if '</htm>' not in data:
  4.         fd=urllib2.urlopen(req)
  5.         data=fd.read()
  6.         time.sleep(10)
  7. time.sleep(10)
  8. blahblah
  9.  
That looks very strange. Why the ``if`` and the `sleep()`\s? The ``if``
condition is always true, so it's completely unnecessary. The `read()`
call is blocking, that means it returns iff the complete data is read. So
there's no need to wait or to check if all is read. If there's a problem
with the connection an exception should be raised by `urllib2`.

Ciao,
Marc 'BlackJack' Rintsch
Jul 1 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Chris Lyon | last post by:
Could somebody please explain the difference between these two modules and explain why they are both required, and if there will ever be a unification of them?
11
by: Pater Maximus | last post by:
I am trying to implement the recipe listed at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/211886 However, I can not get to first base. When I try to run import urllib...
0
by: Shane Hathaway | last post by:
I started experimenting with SOAPpy yesterday and immediately hit a snag. Both web services I tried simply hung and never replied. After a lot of digging, I found out what was going wrong:...
0
by: Pieter Edelman | last post by:
Hi all, I'm trying to submit some data using a POST request to a HTTP server with BASIC authentication with python, but I can't get it to work. Since it's driving me completely nuts, so here's...
3
by: Chris Tavares | last post by:
Hi all. I'm currently tracking down a problem in a little script I have, and I was hoping that those more experienced than myself could weigh in. The script's job is to grab the status page off a...
8
by: Gabriel Zachmann | last post by:
Here is a very simple Python script utilizing urllib: import urllib url = "http://commons.wikimedia.org/wiki/Commons:Featured_pictures/chronological" print url print file = urllib.urlopen(...
1
by: John Nagle | last post by:
If you try to open a password protected page with "urllib.urlopen()", you get "Enter username for EnterPassword at example.com:" on standard output, followed by a read for input! This seems to...
6
by: O.R.Senthil Kumaran | last post by:
Hi, There is an Open Tracker item against urllib2 library python.org/sf/735515 which states that. urllib / urllib2 should cache the results of 301 (permanent) redirections. This shouldn't break...
0
by: johnpollard | last post by:
For some reason this script isn't working and I dont know what it is. I believe the problem lies in the following lines of code since the script works with a different website and username/password...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.