473,602 Members | 2,811 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Anything faster than stat() ?

Help!

I'm recursing through 10,000 directories where some have as few as 5 files
and others have 100's. I'm then performing a few things based on the modify
date of the file. Needless to say, this script is going to run a long time
regardless of what I do. But if I remove the stat, the script runs 500%
faster (duh). That being said, I've set out to find something faster than
stat if at all possible (since stat gets much more info than I need).

I've noted that doing a file check (-f) slows down execution significantly
so I've removed it in favor of regular expressions, but I can't find another
way to get the modify date of each file other than stat and -M.

Anyone have any other ideas? I'm running this in Win32, so I'm sure that's
part of the cause, but perhaps there's a stat-like command that ONLY grabs
modify date, or perhaps there's something in a Win32 module that I haven't
found.

Here's a snippet of code in case it helps.

# open the current directory
opendir D, "$checkpath ";

# loop through files in directory
while (defined($file = readdir D)) {
chomp($file);
# if there's a . in the file, it must be real (weak but works for
now)
if ($file=~ /\./) {
# get the time, in seconds, of the file since Jan 1 1970
$filetime = int((stat($file ))[9]); <--- SLOW!

# do something with it...
}
}

Thanks in advance!
-Ken
Jul 19 '05 #1
1 3034
Ken Tucker <ke*@nospam.iit .net> wrote:
Forget stat(). I just used regular expressions to parse through
a DOS directory listing ... it went from 256 seconds to 16 seconds
That will be specific to Windows.
So - word to the wise -


[ insert joke here ]

--
Steve
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
2305
by: Fred Pacquier | last post by:
First off, sorry for this message-in-a-bottle-like post... I haven't been able to phrase my questions well enough to get a meaningful answer from Google in my research. OTOH, it is standard flattery (but true) that this group has a bunch of the nicest and most knowledgeable Usenet people around, and I know for a fact that there are some pretty good spam- related tools written in Python, so I thought I might get away with it :-) Yes,...
8
3791
by: fuzzylollipop | last post by:
I am trying to get the number of bytes used by files in a directory. I am using a large directory ( lots of stuff checked out of multiple large cvs repositories ) and there is lots of wasted time doing multiple os.stat() on dirs and files from different methods.
3
9521
by: Steven T. Hatton | last post by:
I found the following two statements in the file linked below: struct stat st; stat(fileName.c_str(), &st); http://websvn.kde.org/branches/work/kdevelop4-parser/main.cpp?rev=420247&view=markup This is from /usr/include/sys/stat.h which I'm pretty sure is what #include <sys/stat.h> is pulling in. I plain and simply do not understand what the above two lines of code do. Can someone please explain?
10
5261
by: David Farning | last post by:
I am new to c and not yet sure of the boundary between c and it's implementations. So please redirect me if this should be asked elsewhere. I am working on a function to determine if a directory exists. The idea comes from a snippet I found in a samba source file. bool dirExist(char *dname) { struct stat *stbuf; //stbuf = (struct stat*)malloc(sizeof(struct stat));
5
3152
by: diadia | last post by:
#include <sys/types.h> #include <sys/stat.h> #include "ctype.h" #include <stdio.h> int estimateLen(struct stat buf, FILE *fp) { size_t _size = buf.st_size / 4;
3
3939
by: David Bear | last post by:
I'm trying to use os.chmod and am refered to the stat module. Is there are explanation of: * S_ISUID * S_ISGID * S_ENFMT * S_ISVTX * S_IREAD * S_IWRITE * S_IEXEC
2
5038
by: Michael Glassford | last post by:
The Python 2.5 News at http://www.python.org/download/releases/2.5/NEWS.txt states that Python 2.5 was changed to "Use Win32 API to implement os.stat/fstat. As a result, subsecond timestamps are reported, the limit on path name lengths is removed, and stat reports WindowsError now (instead of OSError)." Although not mentioned in the Python 2.5 News, apparently there was a similar change on Mac that I'm having some problems with. On the...
24
6092
by: Joe Salmeri | last post by:
I just upgraded from Python 2.4.2 to Python 2.5.1 and have found some unexpected behavior that appears to be a bug in the os.stat module. My OS is Windows XP SP2 + all updates. I have several programs that have worked flawlessly on all previous Python versions for years and they are now producing incorrect results in the code that uses os.stat. Searching through the 2.5.1 release notes I found the following:
3
6989
by: Deniz Dogan | last post by:
Hello. I have a question regarding the st_mode field of the "stat" struct in sys/stat.h. I'd like to know how to use the S_IRWXU, S_IRWXG and S_IRWXO flags to mask the mode_t value into human readable form such as 755, 644, etc. Currently I do something similar to this: usr_t = (mod & S_IRWXU); usr_r = (mod & S_IRUSR);
0
7993
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
7920
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8401
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8268
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6730
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5440
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
1
2418
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1510
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1254
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.