473,326 Members | 2,173 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Can This Code Be Made Faster?

I have a problem, in that the following code is sometimes failing. I am
stat'ing a list of files, but, sometimes, one of the files may be moved
by an external process, after the os.path.exists and before the lstat,
causing the script to fail. My question, is there anything I can do to
speed it up, or put in a trap, to stop the stat from failing?

fileTimes = [ ( os.lstat( os.path.join( rootPath, fileName )
)[stat.ST_MTIME], fileName) for fileName in fileList \

if os.path.exists( os.path.join( rootPath, fileName ) ) ]

I'd appreciate any pointers.

John

Jul 18 '05 #1
3 1577
>>>>> "John" == John Abel <jo*******@pa.press.net> writes:
I have a problem, in that the following code is sometimes failing. I
am stat'ing a list of files, but, sometimes, one of the files may be
moved by an external process, after the os.path.exists and before the
lstat, causing the script to fail. My question, is there anything I
can do to speed it up, or put in a trap, to stop the stat from failing?
fileTimes = [ ( os.lstat( os.path.join( rootPath, fileName )
)[stat.ST_MTIME], fileName) for fileName in fileList \
if
os.path.exists( os.path.join( rootPath, fileName ) ) ]

Then there's no point in doing os.path.exists(), you might as well call
os.lstat directly. Some like this should work:

======
fileTimes = []
for fileName in fileList:
try:
fileTimes.append(
(os.lstat(os.path.join(rootPath, fileName)).st_mtime, fileName)
)
except:
pass

======

Ganesan

--
Ganesan R
Jul 18 '05 #2

"John Abel" <jo*******@pa.press.net> schrieb im Newsbeitrag
news:ma*********************************@python.or g...
I have a problem, in that the following code is sometimes failing. I am
stat'ing a list of files, but, sometimes, one of the files may be moved
by an external process, after the os.path.exists and before the lstat,
causing the script to fail. My question, is there anything I can do to
speed it up, or put in a trap, to stop the stat from failing?

I think best help is with try - except (Ganesan already posted...)
But by all means NEVER use except: without any explicit exception; in this
case
except OSError:
would be appropriate.

If you by chance are using Windows only, a much better (and much faster)
suport will be given by
win32api.FindFiles
Of course this is not portable....

Kindly
Michael P
Jul 18 '05 #3
"Ganesan R" <rg******@myrealbox.com> wrote in message
news:ou************@andlx-anamika.cisco.com...
>> "John" == John Abel <jo*******@pa.press.net> writes:
[snip] ======
fileTimes = []
for fileName in fileList:
try:
fileTimes.append(
(os.lstat(os.path.join(rootPath, fileName)).st_mtime, fileName) )
except:
pass

======


def getFileTimes(fileList, rootPath):
# using a function can speed things up (I seem to recall)
fileTimes = []
# use aliases to save some look-up time inside loop
append = fileTimes.append; lstat = os.lstat; joinPath = os.path.join;
for fileName in fileList:
try:
append((lstat(joinPath(rootPath, fileName)).st_mtime, fileName))
except OSError:
pass
return fileTimes
I don't know if function composition would help speed (you'd have to
timeit.py),
but it might help readability:

# from "Python Cookbook", pg. 467
def compose(f, g, *orig_args, **orig_kwds):
def nested_function(*more_args, **more_kwds):
return f(g(*more_args, **more_kwds), *orig_args, **orig_kwds)
return nested_function

def getFileTimes2(fileList, rootPath):
fileTimes = []
append = fileTimes.append
lstat = compose(os.lstat, os.path.join)
for fileName in fileList:
try:
append( (lstat(rootPath, fileName).st_mtime, fileName) )
except OSError:
pass
return fileTimes
HTH
Sean
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

242
by: James Cameron | last post by:
Hi I'm developing a program and the client is worried about future reuse of the code. Say 5, 10, 15 years down the road. This will be a major factor in selecting the development language. Any...
354
by: Montrose... | last post by:
After working in c# for a year, the only conclusion I can come to is that I wish I knew c. All I need is Linux, the gnu c compiler and I can do anything. Web services are just open sockets...
22
by: Liang Chen | last post by:
Is the file "bcopy.c" in the "libitery" directory the implement of the GNU C library function "bcopy"? If so, how can it run so fast?(copy-by-byte rather than copy-by-word) When I copy the code of...
11
by: hasadh | last post by:
Hi, is the assemly code for if..else and switch statements similar. I would like to know if switch also uses value comparison for each case internally or does it jump to the case directly at...
14
by: joshc | last post by:
I'm writing some C to be used in an embedded environment and the code needs to be optimized. I have a question about optimizing compilers in general. I'm using GCC for the workstation and Diab...
11
by: Eigenvector | last post by:
I apologize if this is a trivial question, but it's always made me wonder when I have to compile my code. There are some #includes that you don't really need to reference in your library and...
23
by: Python Maniac | last post by:
I am new to Python however I would like some feedback from those who know more about Python than I do at this time. def scrambleLine(line): s = '' for c in line: s += chr(ord(c) | 0x80)...
4
by: Rick | last post by:
Access2003 in XP I'm using the code below to append any new records from (tbl_From_Mainframe) into (tbl_Appended_Data). It takes more than a minute to search 7000 records for a dozen new records....
30
by: galiorenye | last post by:
Hi, Given this code: A** ppA = new A*; A *pA = NULL; for(int i = 0; i < 10; ++i) { pA = ppA; //do something with pA
1
by: samoukos | last post by:
Hello i had to do this project but at school they tell me that it will be faster using stdio insteed of fstream... Is that right??? if it is faster can anyone suggest how this code will be using...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.