473,386 Members | 1,715 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Re: Agnostic fetching

>----- Original Message -----
>From: "Diez B. Roggisch" <de***@nospam.web.de>
To: py*********@python.org
Sent: Saturday, August 2, 2008 11:05:07 AM GMT -06:00 US/Canada Central
Subject: Re: Agnostic fetching
>Bruce Frederiksen schrieb:
On Fri, 01 Aug 2008 17:05:00 -0700, jorpheus wrote:
>OK, that sounds stupid. Anyway, I've been learning Python for some
time now, and am currently having fun with the urllib and urllib2
modules, but have run into a problem(?) - is there any way to fetch
(urllib.retrieve) files from a server without knowing the filenames?
For instance, there is smth like folder/spam.egg, folder/
unpredictable.egg and so on. If not, perhaps some kind of glob to
create a list of existing files? I'd really appreciate some help,
since I'm really out of my (newb) depth here.

You might try the os.path module and/or the glob module in the standard
python library.
>Not on remote locations. The only work on your local filesystem.
>Diez
Here's a function I wrote for checking remote or local file existence. It works for me but admittedly I haven't tested many cases with it. Also its currently specific to an http URI scheme.

def fileExists(self, fileUrlPath):
fileExists = False
if "http:" in fileUrlPath.lower():
#We don't want to open the file so ask the header if the
#file exists
urlParts = urlparse(fileUrlPath)
host = urlParts[1]
http = httplib.HTTP(host)
http.putrequest("HEAD", fileUrlPath)
http.putheader("Host", host)
http.endheaders()
errorcode, errormessage, headers = http.getreply()
if errorcode == 200:
fileExists = True
else:
fileExists = path.exists(fileUrlPath)

return fileExists

--
Ivan Ven Osdel
http://datasyncsuite.com
Aug 4 '08 #1
1 1218
Ivan Ven Osdel wrote:
>>----- Original Message -----
From: "Diez B. Roggisch" <de***@nospam.web.de>
To: py*********@python.org
Sent: Saturday, August 2, 2008 11:05:07 AM GMT -06:00 US/Canada Central
Subject: Re: Agnostic fetching
>>Bruce Frederiksen schrieb:
On Fri, 01 Aug 2008 17:05:00 -0700, jorpheus wrote:
>>OK, that sounds stupid. Anyway, I've been learning Python for some
time now, and am currently having fun with the urllib and urllib2
modules, but have run into a problem(?) - is there any way to fetch
(urllib.retrieve) files from a server without knowing the filenames?
For instance, there is smth like folder/spam.egg, folder/
unpredictable.egg and so on. If not, perhaps some kind of glob to
create a list of existing files? I'd really appreciate some help,
since I'm really out of my (newb) depth here.

You might try the os.path module and/or the glob module in the standard
python library.
>>Not on remote locations. The only work on your local filesystem.
>>Diez

Here's a function I wrote for checking remote or local file existence. It
works for me but admittedly I haven't tested many cases with it. Also its
currently specific to an http URI scheme.
<snip/>

The idea was to *not* know the filenames beforehand, but instead get a
directory-listing. Which is impossible to get.

Diez
Aug 4 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Shujun Huang | last post by:
Hi, I am working on converting Informix database to Postgre. I have one question for fetching records using PostgreSQL. The record I am fetching is a variable size text string. Before fetching...
2
by: Max André Bündchen | last post by:
Hi, I'm starting a project that will work in many clients and a requiriment is a plataform agnostic system. The idea behind the project is a winform client based interface with a webservice...
22
by: Sandman | last post by:
So, I have this content management system I've developed myself. The system has a solid community part where members can register and then participate in forums, write weblogs and a ton of other...
5
by: Bhavesh | last post by:
Hello genious people, I m trying to insert a LARGE text from Multiline Textbox into my table of sqlserver2000. I m using vs-2005. Please note that I dont want to store blob data From FILE...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.