473,805 Members | 1,972 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Re: Agnostic fetching

>----- Original Message -----
>From: "Diez B. Roggisch" <de***@nospam.w eb.de>
To: py*********@pyt hon.org
Sent: Saturday, August 2, 2008 11:05:07 AM GMT -06:00 US/Canada Central
Subject: Re: Agnostic fetching
>Bruce Frederiksen schrieb:
On Fri, 01 Aug 2008 17:05:00 -0700, jorpheus wrote:
>OK, that sounds stupid. Anyway, I've been learning Python for some
time now, and am currently having fun with the urllib and urllib2
modules, but have run into a problem(?) - is there any way to fetch
(urllib.retrie ve) files from a server without knowing the filenames?
For instance, there is smth like folder/spam.egg, folder/
unpredictable. egg and so on. If not, perhaps some kind of glob to
create a list of existing files? I'd really appreciate some help,
since I'm really out of my (newb) depth here.

You might try the os.path module and/or the glob module in the standard
python library.
>Not on remote locations. The only work on your local filesystem.
>Diez
Here's a function I wrote for checking remote or local file existence. It works for me but admittedly I haven't tested many cases with it. Also its currently specific to an http URI scheme.

def fileExists(self , fileUrlPath):
fileExists = False
if "http:" in fileUrlPath.low er():
#We don't want to open the file so ask the header if the
#file exists
urlParts = urlparse(fileUr lPath)
host = urlParts[1]
http = httplib.HTTP(ho st)
http.putrequest ("HEAD", fileUrlPath)
http.putheader( "Host", host)
http.endheaders ()
errorcode, errormessage, headers = http.getreply()
if errorcode == 200:
fileExists = True
else:
fileExists = path.exists(fil eUrlPath)

return fileExists

--
Ivan Ven Osdel
http://datasyncsuite.com
Aug 4 '08 #1
1 1234
Ivan Ven Osdel wrote:
>>----- Original Message -----
From: "Diez B. Roggisch" <de***@nospam.w eb.de>
To: py*********@pyt hon.org
Sent: Saturday, August 2, 2008 11:05:07 AM GMT -06:00 US/Canada Central
Subject: Re: Agnostic fetching
>>Bruce Frederiksen schrieb:
On Fri, 01 Aug 2008 17:05:00 -0700, jorpheus wrote:
>>OK, that sounds stupid. Anyway, I've been learning Python for some
time now, and am currently having fun with the urllib and urllib2
modules, but have run into a problem(?) - is there any way to fetch
(urllib.retri eve) files from a server without knowing the filenames?
For instance, there is smth like folder/spam.egg, folder/
unpredictable .egg and so on. If not, perhaps some kind of glob to
create a list of existing files? I'd really appreciate some help,
since I'm really out of my (newb) depth here.

You might try the os.path module and/or the glob module in the standard
python library.
>>Not on remote locations. The only work on your local filesystem.
>>Diez

Here's a function I wrote for checking remote or local file existence. It
works for me but admittedly I haven't tested many cases with it. Also its
currently specific to an http URI scheme.
<snip/>

The idea was to *not* know the filenames beforehand, but instead get a
directory-listing. Which is impossible to get.

Diez
Aug 4 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1709
by: Shujun Huang | last post by:
Hi, I am working on converting Informix database to Postgre. I have one question for fetching records using PostgreSQL. The record I am fetching is a variable size text string. Before fetching the record, a preallocated static buffer size will have to be declared beforehand in the declaration section. Since the record sizes vary a lot, instead of using the static buffer, is there a dynamic way to fetch these records? Informix (Oracle...
2
1361
by: Max André Bündchen | last post by:
Hi, I'm starting a project that will work in many clients and a requiriment is a plataform agnostic system. The idea behind the project is a winform client based interface with a webservice communication middle tier. In .Net 2.0, it's very easy to develop and deploy for Windows. My question is about the initiatives to run complex .Net systems in agnostic plataform way (like Linux). There's real perspectives to make this work fine (some...
22
2750
by: Sandman | last post by:
So, I have this content management system I've developed myself. The system has a solid community part where members can register and then participate in forums, write weblogs and a ton of other things. So, in instances where I list, for example, the latest weblogs. I list the headline of the weblog, the date and the name of the member who wrote it. Now, the name isn't just "Smith", but rather Smith's online status, his nick and his...
5
3001
by: Bhavesh | last post by:
Hello genious people, I m trying to insert a LARGE text from Multiline Textbox into my table of sqlserver2000. I m using vs-2005. Please note that I dont want to store blob data From FILE TO TABLE, like storing IMAGE into DB.
0
10607
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10359
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10364
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10104
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6875
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5541
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5677
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
3843
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3007
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.