473,785 Members | 3,157 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

PEP ? os.listdir enhancement

Hi,
I noticed that when I use os.listdir I need to work with absolute paths
90% of times.
While I can use a for cycle, I'd prefere to use a list comprehension,
but it becomes too long.

I propose to add an 'abs' keyword which would make os.listdir return the
absolute path of files instead of a relative path.
This would bring only advantages, I think.

I show two examples

from os import listdir
from os.path import isdir,join,gets ize,isfile

### e.g. 1 part 1 - getting a list of directories ###
dirs=[]
for i in os.listdir(path ):
tmp_path=os.pat h.join(path,i)
if os.path.isdir(t mp_path):
dirs.append(tmp _path)

### e.g. 1 part 2 ###
dirs=[join(path,x) for x in listdir(path) if isdir(join(path ,x))]

here the list comprehension is still clear, but only because we have
direct references to join and friends. moreover whe need to use join twice
for each directory.

### e.g. 2 part 1 - getting a list of (file,size) tuples ###
path_size=[]
for i in os.listdir(path ):
tmp_path=os.pat h.join(path,i)
if os.path.isfile( tmp_path):
path_size.appen d((tmp_path,get size(tmp_path))

### e.g. 2 part 2 ###
dirs=[(join(path,x),g etsize(join(pat h,x)) for x in listdir(path) if
isfile(join(pat h,x))]
now list comprehension is way too long, and in the worst case we must use
join 3 times for each iteration.

adding an 'abs' keyword to os.listdir would give benefits both to for
cycle and list comprehensions.
for cycle would lose the tmp_path assignment and list comprehensions ...
### e.g. 1 part 2 bis ###
dirs=[x for x in listdir(path,ab s=True) if isdir(x)]

here we gain clearness and speed.

### e.g. 2 part 2 bis ###
dirs=[(x,getsize(x)) for x in listdir(path,ab s=True) if isfile(x)]

and here we gain clearness, speed and a truely _usable_ list comprehension

What do you think about this ?

Thanks for reading,
Riccardo

--
Riccardo Galli
Sideralis Programs
http://www.sideralis.net
Jul 19 '05 #1
15 2581
Why not just define the function yourself? Not every 3-line function
needs to be built in.

def listdir_joined( path):
return [os.path.join(pa th, entry) for entry in os.listdir(path )]

dirs = [x for x in listdir_joined( path) if os.path.isdir(x )]

path_size = [(x, getsize(x)) for x in listdir_joined( path) if os.path.isfile( x)]
Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFCuZFYJd0 1MZaTXX0RAqz2AK Cozk1iJjrTwFRv4 up5NdeVXqUNUQCe KniG
Q4Sl42Sp6dMnMFL 0u607gwI=
=QEy2
-----END PGP SIGNATURE-----

Jul 19 '05 #2
Riccardo Galli wrote:
I noticed that when I use os.listdir I need to work with absolute paths
90% of times.
While I can use a for cycle, I'd prefere to use a list comprehension,
but it becomes too long.

### e.g. 1 part 1 - getting a list of directories ###
dirs=[]
for i in os.listdir(path ):
tmp_path=os.pat h.join(path,i)
if os.path.isdir(t mp_path):
dirs.append(tmp _path)

### e.g. 1 part 2 ###
dirs=[join(path,x) for x in listdir(path) if isdir(join(path ,x))]


Using Jason Orendorff's "path" module, all this code basically collapses
down to this beauty (with your variable "path" renamed to myPath to
avoid a name collision):

from path import path
dirs = path(myPath).ab spath().dirs()

-Peter
Jul 19 '05 #3
Peter Hansen wrote:
Using Jason Orendorff's "path" module, all this code basically collapses
down to this beauty (with your variable "path" renamed to myPath to
avoid a name collision):


This has to be the non-stdlib library I use the most. It's a great module.
--
Michael Hoffman
Jul 19 '05 #4
On Wed, 22 Jun 2005 11:27:06 -0500, Jeff Epler wrote:
Why not just define the function yourself? Not every 3-line function
needs to be built in.


Of course I can code such a function, and I agree with the second
sentence, but I think that obtaining absolutes path is a task so commonly
needed that adding a keyword to an existing function would give a plus to
the library without adding complexity (speaking of number of modules).

Usually when you use os.listdir do you end using os.path.join to obtain
absolutes path? I'm interested to know if the task is so common as I
think, or if I'm wrong.

Thank you,
Riccardo
--
Riccardo Galli
Sideralis Programs
http://www.sideralis.net
Jul 19 '05 #5
What's wrong with

(os.path.join(d , x) for x in os.listdir(d))

It's short, and easier to understand then some obscure option ;)

Andreas

On Thu, Jun 23, 2005 at 11:05:57AM +0200, Riccardo Galli wrote:
On Wed, 22 Jun 2005 11:27:06 -0500, Jeff Epler wrote:
Why not just define the function yourself? Not every 3-line function
needs to be built in.


Of course I can code such a function, and I agree with the second
sentence, but I think that obtaining absolutes path is a task so commonly
needed that adding a keyword to an existing function would give a plus to
the library without adding complexity (speaking of number of modules).

Usually when you use os.listdir do you end using os.path.join to obtain
absolutes path? I'm interested to know if the task is so common as I
think, or if I'm wrong.

Thank you,
Riccardo
--
Riccardo Galli
Sideralis Programs
http://www.sideralis.net
--
http://mail.python.org/mailman/listinfo/python-list

Jul 19 '05 #6
On 6/22/05, Riccardo Galli <riccardo_cut1@ cut2_sideralis. net> wrote:
I propose to add an 'abs' keyword which would make os.listdir return the
absolute path of files instead of a relative path.


What about os.listdir(dir= 'relative/path', abs=True)? Should listdir
call abspath on results? Should we add another keyword rel? Would it
complicate listdir unnecessarily?

- kv
Jul 19 '05 #7
On Thu, 23 Jun 2005 11:34:02 +0200, Andreas Kostyrka wrote:
What's wrong with

(os.path.join(d , x) for x in os.listdir(d))

It's short, and easier to understand then some obscure option ;)

Andreas


how does it help in using list comprehension, as the ones in the first
post?
--
Riccardo Galli
Sideralis Programs
http://www.sideralis.net
Jul 19 '05 #8
Riccardo Galli wrote:
On Thu, 23 Jun 2005 12:56:08 +0300, Konstantin Veretennicov wrote:
What about os.listdir(dir= 'relative/path', abs=True)? Should listdir call
abspath on results? Should we add another keyword rel? Would it complicate
listdir unnecessarily?


keyword dir not exists (don't know if you added as example or not) and
abs set to true would return abspath on result. What else could it do ?


He probably meant that a 'join' option would be more natural than an
'abs' option. After all, your examples use os.path.join to create a
valid path that can be used as the argument to other module os
functions. Whether the results are absolute or relative should depend on
the initial argument to os.listdir.

Daniel
Jul 19 '05 #9
Konstantin Veretennicov wrote:
On 6/22/05, Riccardo Galli <riccardo_cut1@ cut2_sideralis. net> wrote:
I propose to add an 'abs' keyword which would make os.listdir return the
absolute path of files instead of a relative path.


What about os.listdir(dir= 'relative/path', abs=True)? Should listdir
call abspath on results? Should we add another keyword rel? Would it
complicate listdir unnecessarily?

- kv


I'm in favour of Riccardo's suggestion, but I think he's got the name of the
keyword wrong. The signature should be

listdir(path, with_path=False )

and cater absolute and relative paths alike.

The need for such an enhancement is not particularly pressing, as
workarounds abound. Here's another one:
glob.glob("/usr/lib/games/../games/*")

['/usr/lib/games/../games/schwarzerpeter']
Peter

Jul 19 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

11
23037
by: Jason Kratz | last post by:
OK. I've search on google groups and around the web for this and I haven't found an answer. I'm a Python newbie and have what I assume is a basic question. os.listdir takes a pathname as an arg but it doesn't actually list the contents of the dir I pass in. it always operates on the current dir (wherever the script is run) and I have to chdir beforehand. Is that how its supposed to work? If so what is the point in passing in a...
8
2450
by: Hannu Kankaanp?? | last post by:
This may be a bug or simply a strange result of undefined behaviour, but this is what I get with Python 2.3.2 on Windows XP: >>> import os >>> os.listdir('') >>> os.listdir(u'')
0
465
by: Ishwor | last post by:
hi check your seperator variable in the os module. :) for example >>> import os >>> os.sep '\\' Now what you do is :- >> os.listdir("D:" + os.sep + "any_other_folder_name" + os.sep); :)
1
2829
by: Doran_Dermot | last post by:
Hi All, I'm currently using "os.listdir" to obtain the contents of some slow Windows shares. I think I've seen another way of doing this using the win32 library but I can't find the example anymore. My main problem with using "os.listdir" is that it hangs my gui application. The tread running the "os.listdir" appears to block all other threads when it calls this function.
7
3875
by: Kenneth Pronovici | last post by:
I have some confusion regarding the relationship between locale, os.listdir() and unicode pathnames. I'm running Python 2.3.5 on a Debian system. If it matters, all of the files I'm dealing with are on an ext3 filesystem. The real code this problem comes from takes a configured set of directories to deal with and walks through each of those directories using os.listdir(). Today, I accidentally ran across a directory containing three...
1
3270
by: kai | last post by:
Hello, I use dircache.listdir(myDir) in my module repeatedly. On OS WIN 2000 listdir() will re-read the directory structure! But on AIX, listdir() will not re-read the directory structure (see Python Library Reference). I work with python version 2.2. Now my 2 questions: Why does dircache.listdir() work different?
6
17362
by: Stef Mientki | last post by:
hello, I want to find all files with the extension "*.txt". From the examples in "Learning Python, Lutz and Asher" and from the website I see examples where you also may specify a wildcard filegroup. But when I try this files = os.listdir('D:\\akto_yk\\yk_controle\\*.txt') I get an error message
3
8936
by: vedrandekovic | last post by:
Hello Here is my simple listdir example: Here is my error: WindowsError: The system cannot find the path specified: 'l/ *.*'
0
1825
by: scriptmann | last post by:
Hi, I'm trying to use os.listdir() to list directories with simplified chinese filenames. However, while I can see the filenames correctly in windows explorer, I am getting ? in the filename strings returned by os.listdir() for some chinese characters. What should I do to make os.listdir return what I see in Windows Explorer (i.e. the full simplified chinese filename)? I am confused because os.listdir() is supposed to have unicode...
0
9480
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10325
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9950
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8972
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5381
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5511
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4053
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3646
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2879
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.