473,386 Members | 1,708 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Downloading files using urllib in a for loop?

Hi,
I'm using Python 2.3 on Windows for the first time, and am doing
something wrong in using urllib to retrieve images from urls embedded
in a csv file. If I explicitly specify a url and image name it works
fine(commented example in the code), but if I pass in variables in this
for loop it throws errors:

--- The script:

import csv, urllib
reader = csv.reader(open("source.csv"))
for x,y,z,imagepath in reader
theurl = imagepath[:55]
theimage = imagepath[55:-8]
urllib.urlretrieve(theurl, theimage)
#urllib.urlretrieve("http://someurl/image.gif", "image.gif") # works!

--- The errors:

This throws the following errors:
File "getimages.py", line 9, in ?
urllib.urlretrieve(theurl,theimage)
File "C:\Python23\lib\urllib.py", line 83, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "C:\Python23\lib\urllib.py", line 213, in retrieve
fp = self.open(url, data)
File "C:\Python23\lib\urllib.py", line 181, in open
return getattr(self, name)(url)
File "C:\Python23\lib\urllib.py", line 410, in open_file
return self.open_local_file(url)
File "C:\Python23\lib\urllib.py", line 420, in open_local_file
raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 2] No such file or directory: ''

---

Would really appreciate some pointers on the right way to loop through
and retrieve images, as I've tried various other solutions but am
clearly missing something simple!

Thanks,

justin.

Feb 15 '06 #1
6 5780

ju*****@gmail.com wrote:
Hi,
I'm using Python 2.3 on Windows for the first time, and am doing
something wrong in using urllib to retrieve images from urls embedded
in a csv file. If I explicitly specify a url and image name it works
fine(commented example in the code), but if I pass in variables in this
for loop it throws errors:

--- The script:

import csv, urllib
reader = csv.reader(open("source.csv"))
for x,y,z,imagepath in reader
theurl = imagepath[:55]
theimage = imagepath[55:-8]
"No such file or directory: ''" sounds to me like you are trying
to open a file called '' (empty string)

try adding some debugging

print theimage, imagepath
urllib.urlretrieve(theurl, theimage)
#urllib.urlretrieve("http://someurl/image.gif", "image.gif") # works!

--- The errors:

This throws the following errors:
File "getimages.py", line 9, in ?
urllib.urlretrieve(theurl,theimage)
File "C:\Python23\lib\urllib.py", line 83, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "C:\Python23\lib\urllib.py", line 213, in retrieve
fp = self.open(url, data)
File "C:\Python23\lib\urllib.py", line 181, in open
return getattr(self, name)(url)
File "C:\Python23\lib\urllib.py", line 410, in open_file
return self.open_local_file(url)
File "C:\Python23\lib\urllib.py", line 420, in open_local_file
raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 2] No such file or directory: ''

---

Would really appreciate some pointers on the right way to loop through
and retrieve images, as I've tried various other solutions but am
clearly missing something simple!

Thanks,

justin.


Feb 15 '06 #2
Thanks - but have printed and verified they are valid paths and
filenames. One correction to the code I listed:
theurl = imagepath[:-8]

For some reason the values aren't being passed through
urllib.urlretrieve properly but this makes no sense to me?

Feb 15 '06 #3
Martin Franklin wrote:
"No such file or directory: ''" sounds to me like you are trying
to open a file called '' (empty string)

try adding some debugging

print theimage, imagepath


or, better:

print repr(theimage), repr(imagepath)

</F>

Feb 15 '06 #4
Martin Franklin wrote:
ju*****@gmail.com wrote:
Hi,
I'm using Python 2.3 on Windows for the first time, and am doing
something wrong in using urllib to retrieve images from urls embedded
in a csv file. If I explicitly specify a url and image name it works
fine(commented example in the code), but if I pass in variables in this
for loop it throws errors:

--- The script:

import csv, urllib
reader = csv.reader(open("source.csv"))
for x,y,z,imagepath in reader
I just noticed the code you sent will not work... notice the lack of a
colon ( : ) and the end of the 'for' line....

please post an exact copy of your code and also the results with the
included print debugging line (with or without repr ;) )

Cheers
Martin

theurl = imagepath[:55]
theimage = imagepath[55:-8]


"No such file or directory: ''" sounds to me like you are trying
to open a file called '' (empty string)

try adding some debugging

print theimage, imagepath
urllib.urlretrieve(theurl, theimage)
#urllib.urlretrieve("http://someurl/image.gif", "image.gif") # works!

--- The errors:

This throws the following errors:
File "getimages.py", line 9, in ?
urllib.urlretrieve(theurl,theimage)
File "C:\Python23\lib\urllib.py", line 83, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "C:\Python23\lib\urllib.py", line 213, in retrieve
fp = self.open(url, data)
File "C:\Python23\lib\urllib.py", line 181, in open
return getattr(self, name)(url)
File "C:\Python23\lib\urllib.py", line 410, in open_file
return self.open_local_file(url)
File "C:\Python23\lib\urllib.py", line 420, in open_local_file
raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 2] No such file or directory: ''

---

Would really appreciate some pointers on the right way to loop through
and retrieve images, as I've tried various other solutions but am
clearly missing something simple!

Thanks,

justin.


Feb 15 '06 #5
ju*****@gmail.com wrote:
Thanks - but have printed and verified they are valid paths and
filenames. One correction to the code I listed:
theurl = imagepath[:-8]

For some reason the values aren't being passed through
urllib.urlretrieve properly but this makes no sense to me?


A working (for me!) example:-
import urllib

paths = ["http://www.python.org/index.html", ]

for remotepath in paths:
# keep only last 10 chars (index.html)
# for local file name
localpath = remotepath[-10:]

print remotepath, localpath

urllib.urlretrieve(remotepath, localpath)
Feb 15 '06 #6
ah - thanks for your help, but what is happening is the first line
being returned contains the field names from the csv file! Schoolboy
errors :-)

Feb 15 '06 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Blaktyger | last post by:
I would like to download some mp3 files from a web site. There is to much of them and I had the idea of writing a script to do it for me. Code: import string import urllib f =...
1
by: NewFilmFan | last post by:
I use Python 2.3 on Windows XP. I wrote this program: import httplib conn = httplib.HTTPConnection("www.x.net") conn.request("GET", "/x/y.jpg") r1 = conn.getresponse() print r1.status,...
1
by: Oyvind Ostlund | last post by:
Hello, I have to download a lot of files. And at the moment i am trying with URLLib. But sometimes it doesn't download the whole file. It looks like it stops half way through or something. Is it...
1
by: Greg L. | last post by:
I apologize if this is not the correct group.... Using the system.net namespace I want to download an entire directory's worth of files. I've been researching furiously and can't seem to find...
10
by: mwt | last post by:
This code works fine to download files from the web and write them to the local drive: import urllib f = urllib.urlopen("http://www.python.org/blah/blah.zip") g = f.read() file =...
4
by: aldonnelley | last post by:
Hi there: a bit of a left-field question, I think. I'm writing a program that analyses image files downloaded with a basic crawler, and it's slow, mainly because I only want to analyse files...
9
by: Frank Potter | last post by:
I want to find a multithreaded downloading lib in python, can someone recommend one for me, please? Thanks~
7
by: Ehsan | last post by:
I foundd this code in ASPN Python Cookbook for downloading files in python but when it finished downloading files the files became corrupted and didn't open, the files in internet havn't any...
7
by: Jetus | last post by:
I am able to download this page (enclosed code), but I then want to download a pdf file that I can view in a regular browser by clicking on the "view" link. I don't know how to automate this next...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.