This function is intended to remove unwanted files and dirs from
os.walk(). It will return correctly *IF* I leave the 'for fs in
fs_objects' statement out (basically leave out the entire purpose of the
function).
It's odd, when the program goes into that statment... even when only a
'pass', and nothing else is present, nothing is returned. Why is that?
I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on
either platform.
def build_clean_list(self, path):
file_skip_list = ['search_results.txt']
dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
## for fs in fs_objects:
##
## for f in fs[2]:
## if f in file_skip_list:
## print f
## fs[2].remove(f)
##
## for d in fs[1]:
## if d in dir_skip_list:
## print d
## fs[1].remove(d)
return fs_objects 7 1700
rbt wrote: This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function).
It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform.
def build_clean_list(self, path):
file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True) ## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
return fs_objects
Just to clarify, it's wrong of me to say that 'nothing is returned'...
in either case, this is what is returned:
Here's what was returned and its type:
----------------------------------------
<generator object at 0x407dbe4c>
<type 'generator'>
----------------------------------------
But, I can't iterate over the returned object when I descend into the
for statement I mentioned above.
That's an easy one: fs_objects is not modified by your ode, so you get
it back as created by os.walk
"rbt" <rb*@athop1.ath.vt.edu> wrote in message
news:cu**********@solaris.cc.vt.edu... rbt wrote: This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function).
It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform.
def build_clean_list(self, path):
file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True) ## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
return fs_objects
Just to clarify, it's wrong of me to say that 'nothing is returned'... in either case, this is what is returned:
Here's what was returned and its type: ---------------------------------------- <generator object at 0x407dbe4c> <type 'generator'> ----------------------------------------
But, I can't iterate over the returned object when I descend into the for statement I mentioned above.
What do you mean by not being able to iterate over the returned object?
What kind of error are you getting? Have you tried to debug the code?
BTW, os.walk indeed returns a generator. You should familiarize yourself
with generators and iterators if you haven't done so yet.
"rbt" <rb*@athop1.ath.vt.edu> wrote in message
news:cu**********@solaris.cc.vt.edu... def build_clean_list(self, path):
file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True) ## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
return fs_objects
Rather as an aside, the idiom for using os.walk is
for dirpath, dirnames, dirfiles in os.walk(path):
for f in dirnames:
if f in file_skip_list:
print f
filenames.remove(f)
if d in dir_skip_list:
print d
dirnames.remove(f)
More crucially for your code, returning the generator object after having
iterated all the way through it will not do you any good. The generator has
an internal state that puts it at "the end of the iteration" so you cannot
use it to iterate again.
rbt wrote: rbt wrote:
This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function).
It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform.
def build_clean_list(self, path):
file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
fs_objects is a generator, not a list. This loop is exhausting fs_objects, so when you return
fs_objects is at the end of iteration, there is nothing left.
## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
Add this here:
yield fs
and take out the return. This turns build_clean_list() into a generator function and you will be
able to iterate the result.
Kent return fs_objects
Just to clarify, it's wrong of me to say that 'nothing is returned'... in either case, this is what is returned:
Here's what was returned and its type: ---------------------------------------- <generator object at 0x407dbe4c> <type 'generator'> ----------------------------------------
But, I can't iterate over the returned object when I descend into the for statement I mentioned above.
Kent Johnson wrote: rbt wrote:
rbt wrote:
This function is intended to remove unwanted files and dirs from os.walk(). It will return correctly *IF* I leave the 'for fs in fs_objects' statement out (basically leave out the entire purpose of the function).
It's odd, when the program goes into that statment... even when only a 'pass', and nothing else is present, nothing is returned. Why is that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on either platform.
def build_clean_list(self, path):
file_skip_list = ['search_results.txt'] dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
fs_objects is a generator, not a list. This loop is exhausting fs_objects, so when you return fs_objects is at the end of iteration, there is nothing left.
That makes sense. Thanks for the explanation. I've never used generators
before.
## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
Add this here: yield fs
and take out the return. This turns build_clean_list() into a generator function and you will be able to iterate the result.
I'll try this.
Will the changes I made (file and dir removals from os.walk()) be
reflected in the generator object? Is it safe to remove objects this way
and pass the results in a generator on to another function? Sorry for
all the questions, I just like to fully understand something before I
start doing it with confidence.
rbt Kent
return fs_objects
Just to clarify, it's wrong of me to say that 'nothing is returned'... in either case, this is what is returned:
Here's what was returned and its type: ---------------------------------------- <generator object at 0x407dbe4c> <type 'generator'> ----------------------------------------
But, I can't iterate over the returned object when I descend into the for statement I mentioned above.
rbt wrote: ## for fs in fs_objects: ## ## for f in fs[2]: ## if f in file_skip_list: ## print f ## fs[2].remove(f) ## ## for d in fs[1]: ## if d in dir_skip_list: ## print d ## fs[1].remove(d)
Will the changes I made (file and dir removals from os.walk()) be reflected in the generator object? Is it safe to remove objects this way and pass the results in a generator on to another function? Sorry for all the questions, I just like to fully understand something before I start doing it with confidence.
Yes. The docs for os.walk() explicitly state, "When topdown is true, the caller can modify the
dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into
the subdirectories whose names remain in dirnames."
So changes to the dir list affect the iteration; changes to the file list directly affect the value
you return to the caller.
Kent This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: mike420 |
last post by:
In the context of LATEX, some Pythonista asked what the big
successes of Lisp were. I think there were at least three *big*
successes.
a. orbitz.com web site uses Lisp for algorithms, etc.
b....
|
by: hokieghal99 |
last post by:
This script is not recursive... in order to make it recursive, I have to
call it several times (my kludge... hey, it works). I thought os.walk's
sole purpose was to recursively walk a directory...
|
by: Frans Englich |
last post by:
Hello,
Have a look at this recursive function:
def walkDirectory( directory, element ):
element = element.newChild( None, "directory", None ) # automatically
appends to parent...
|
by: rbt |
last post by:
Could someone demonstrate the correct/proper way to use os.walk() to skip certain
files and folders while walking a specified path? I've read the module docs and
googled to no avail and posted here...
|
by: sparks |
last post by:
We get more and more data done in excel and then they want it imported
into access.
The data is just stupid....values of 1 to 5 we get a lot of 0's ok
that alright but 1-jan ?
we get colums...
|
by: sopranos2 |
last post by:
Dear IBM DB2 support,
I wish CLP would have been more functional. Without living the command
line I would like to get sql return code explanations, scalar function
definitions etc..
Looking...
|
by: Sky |
last post by:
I have been looking for a more powerful version of GetType(string) that will
find the Type no matter what, and will work even if only supplied
"{TypeName}", not the full "{TypeName},{AssemblyName}"...
|
by: KraftDiner |
last post by:
The os.walk function walks the operating systems directory tree.
This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?
for root, dirs,...
|
by: =?Utf-8?B?U3dhcHB5?= |
last post by:
Can anyone suggest me to pass more parameters other than two parameter for
events like the following?
Event:
Onbutton_click(object sender, EventArgs e)"
Event handler:
button.Click += new...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
| |