473,385 Members | 1,379 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

alternative to fstat ?

I have thousands of files whose exitence needs to be checked.
I think fstat is too costly for this.
Is there any better way for the same ?
Dec 11 '07 #1
11 5105
"solarisss" <sa**************@gmail.comschrieb im Newsbeitrag
news:d6**********************************@s19g2000 prg.googlegroups.com...
>I have thousands of files whose exitence needs to be checked.
I think fstat is too costly for this.
Is there any better way for the same ?
OT here, but in POSIX you'd have stat() (which would save you the fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).

Bye, Jojo
Dec 11 '07 #2
In article <fj**********@online.de>,
Joachim Schmitz <jo**@schmitz-digital.dewrote:
>OT here, but in POSIX you'd have stat() (which would save you the fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).
I doubt access() would be significantly cheaper since it still has
to do a system call and get the inode of the file.

-- Richard
--
:wq
Dec 11 '07 #3
"Richard Tobin" <ri*****@cogsci.ed.ac.ukschrieb im Newsbeitrag
news:fj***********@pc-news.cogsci.ed.ac.uk...
In article <fj**********@online.de>,
Joachim Schmitz <jo**@schmitz-digital.dewrote:
>>OT here, but in POSIX you'd have stat() (which would save you the fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).

I doubt access() would be significantly cheaper since it still has
to do a system call and get the inode of the file.
It is significantly cheaper on a system I frequently work with.

Bye, Jojo
Dec 11 '07 #4
solarisss <sa**************@gmail.comwrote:
# I have thousands of files whose exitence needs to be checked.
# I think fstat is too costly for this.
# Is there any better way for the same ?

stat information is not generally stored as part of the directory;
getting inode information generally generally requires a random
disc access. Some file systems might allow store inodes to improve
reading them, but that would be implementation dependent.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
There are subtler ways of badgering a witness.
Dec 11 '07 #5
solarisss schrieb:
I have thousands of files whose exitence needs to be checked.
I think fstat is too costly for this.
Is there any better way for the same ?
if the files are located in few common directories, fstatat might be an
option on Solaris. Don't know about other systems...

- Thomas
Dec 11 '07 #6
Joachim Schmitz wrote:
>
"solarisss" <sa**************@gmail.comschrieb im Newsbeitrag
news:d6**********************************@s19g2000 prg.googlegroups.com...
I have thousands of files whose exitence needs to be checked.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I think fstat is too costly for this.
Is there any better way for the same ?
OT here, but in POSIX you'd have stat() (which would save you the fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).
Well, if you have a file handle to pass to fstat(), I think we can
pretty much agree that the file exists. (Even if POSIX is OT here.)

:-)

And, if I recall, a discussion in the past noted that a failed access()
call doesn't necessarily mean the file doesn't exist. (For example,
you may simply not have permission to check for the existence.) Also,
there was a discussion about some sort of "cryptographic filesystem"(?)
in which the existence of a file cannot be determined.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>

Dec 11 '07 #7
"Kenneth Brody" <ke******@spamcop.netschrieb im Newsbeitrag
news:47***************@spamcop.net...
Joachim Schmitz wrote:
>>
"solarisss" <sa**************@gmail.comschrieb im Newsbeitrag
news:d6**********************************@s19g200 0prg.googlegroups.com...
>I have thousands of files whose exitence needs to be checked.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I think fstat is too costly for this.
Is there any better way for the same ?
OT here, but in POSIX you'd have stat() (which would save you the
fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).

Well, if you have a file handle to pass to fstat(), I think we can
pretty much agree that the file exists. (Even if POSIX is OT here.)

:-)
Oops...
And, if I recall, a discussion in the past noted that a failed access()
call doesn't necessarily mean the file doesn't exist. (For example,
you may simply not have permission to check for the existence.) Also,
Very true, so you would have to check errno in case access() returns a -1.

printf("%s does%s exist\n", filename, ((access(filename) == -1) && (errno==
ENOFILE))? "n't":"");

The same is true for stat BTW(). And for fstat() resp. the open() it needs.

Bye, Jojo
Dec 11 '07 #8
Joachim Schmitz wrote:
>
"Kenneth Brody" <ke******@spamcop.netschrieb im Newsbeitrag
news:47***************@spamcop.net...
Joachim Schmitz wrote:
[... use fstat() to check for file existence ...]
Well, if you have a file handle to pass to fstat(), I think we can
pretty much agree that the file exists. (Even if POSIX is OT here.)

:-)
Oops...
And, if I recall, a discussion in the past noted that a failed access()
call doesn't necessarily mean the file doesn't exist. (For example,
you may simply not have permission to check for the existence.) Also,
Very true, so you would have to check errno in case access() returns a -1.

printf("%s does%s exist\n", filename, ((access(filename) == -1) && (errno==
ENOFILE))? "n't":"");

The same is true for stat BTW(). And for fstat() resp. the open() it needs.
(I'm not sure why we're discussing POSIX here, but...)

What if the file doesn't exist in a directory to which you do not
have the proper permissions? I believe the error will be EPERM
(or, at least, not ENOFILE), and you will display "does exist".

The point is, an error other than ENOFILE does not imply that the
file does exist. I believe the same holds true for stat() et al.
The only thing you can reliably check for is whether you can open
the file, by seeing what fopen() returns. (Yes, you can use
something like access() to see if you have permission to a file
_at_the_time_you_call_access()_, but you have race conditions to
take into account.)

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>

Dec 11 '07 #9
solarisss <sa**************@gmail.comwrites:
I have thousands of files whose exitence needs to be checked.
I think fstat is too costly for this.
Is there any better way for the same ?
Since fstat() is defined by POSIX, I suggest that comp.unix.programmer
would provide better answers to your question.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Looking for software development work in the San Diego area.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Dec 11 '07 #10
"Kenneth Brody" <ke******@spamcop.netschrieb im Newsbeitrag
news:47***************@spamcop.net...
Joachim Schmitz wrote:
>>
"Kenneth Brody" <ke******@spamcop.netschrieb im Newsbeitrag
news:47***************@spamcop.net...
Joachim Schmitz wrote:
[... use fstat() to check for file existence ...]
Well, if you have a file handle to pass to fstat(), I think we can
pretty much agree that the file exists. (Even if POSIX is OT here.)

:-)
Oops...
And, if I recall, a discussion in the past noted that a failed access()
call doesn't necessarily mean the file doesn't exist. (For example,
you may simply not have permission to check for the existence.) Also,
Very true, so you would have to check errno in case access() returns
a -1.

printf("%s does%s exist\n", filename, ((access(filename) == -1) &&
(errno==
ENOFILE))? "n't":"");
ENOENT, not ENOFILE
>The same is true for stat BTW(). And for fstat() resp. the open() it
needs.

(I'm not sure why we're discussing POSIX here, but...)
Because ANSI/ISO C doesn't have fstat(), so the thread was OT from it's very
beginning
8-)
What if the file doesn't exist in a directory to which you do not
have the proper permissions? I believe the error will be EPERM
(or, at least, not ENOFILE), and you will display "does exist".
The point is, an error other than ENOFILE does not imply that the
file does exist. I believe the same holds true for stat() et al.
OK, make it this then:
printf("%s %s exist\n", filename, ((access(filename) == -1) && (errno==
ENOENT))? "doesn't":"may");
The only thing you can reliably check for is whether you can open
the file, by seeing what fopen() returns. (Yes, you can use
something like access() to see if you have permission to a file
_at_the_time_you_call_access()_, but you have race conditions to
take into account.)
The OP was looking for an alternative to fstat() to check for the existence
of a file, that request seems entirly unneccessary, as fstat() requires the
file to have been open()ed first, hence proving that it exists and is
accessible.

To bring it back to topicallity, how about this:
#include <stdio.h>
#include <errno.h>

/*
* return 1 if file could be opened, proving it's existence, 0 otherwise,
* which is not a secure sign that the file doesn't exist!
*/
int faccess(char *filename)
{
FILE *fp;
/* mode got to be "r" or "r+", anything else would create it */
fp=fopen(filename, "r");
if (fp)
fclose(fp);

return fp?1:0;
}

/*
* returns 0 if filename does not exist, 1 if it exists and -1 if unsure
* i.e. if couldn't be opened for reading, but this could be due to
permissions
* to the file or some component of the file's path
*/
int fexist(char *filename)
{
errno=0;
(void)faccess(filename);
perror("");
switch (errno)
{
case 0: return 1;
case ENOENT: return 0;
default: return -1;
}
}

Bye, Jojo
Dec 12 '07 #11
>"solarisss" <sa**************@gmail.comschrieb im Newsbeitrag
>news:d6**********************************@s19g200 0prg.googlegroups.com...
>>I have thousands of files whose exitence needs to be checked.
I think fstat is too costly for this.
Is there any better way for the same ?
In article <fj**********@online.deJoachim Schmitz
<jo**@schmitz-digital.dewrote:
>OT here, but in POSIX you'd have stat() (which would save you the fopen())
and access() (which should be much cheaper as it doesn't have to fill the
entire struct stat).
Using access() is usually the wrong approach, although this is not
the place to discuss why.

In this case, the best approach is probably to read the directory
(or directories) in which those "thousands of files" might be
stored, and then do your own in-memory lookups ... but this too is
off-topic, because ANSI C does not even support "read a directory"
type operations.
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.
Dec 13 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

99
by: Paul McGuire | last post by:
There are a number of messages on the python-dev mail list that indicate that Guido is looking for some concensus to come from this list as to what *one* alternative syntax for decorators we would...
28
by: Paul McGuire | last post by:
Well, after 3 days of open polling, the number of additional votes have dropped off pretty dramatically. Here are the results so far: Total voters: 55 (with 3 votes each) Votes for each choice...
4
by: Klaus Füller | last post by:
I would like to use the unix,et.al-fstat() system-call on some sort of stream which is already open and connected to a disk file. An example: I have an open stream and want to know the numerical...
1
by: prasaddevivara | last post by:
I am using the outerHTML property to modify the HTML of existin elements in a web page in Internet Explorer. But same outerHTM property is not working in firefox browser, Anybody can tell me a...
2
by: Martijn | last post by:
Hi, I have an open file handle from which I would like to retrieve the file descriptor for use with fstat. Is this possible or should I redesign my algorithm a bit (I have the feeling I am...
1
by: Mark Fink | last post by:
Hi there, unfortunately I am new to Jython and my "Jython Essentials" book is still in the mail. I looked into the Jython API Doc but could not find the information. I am porting a Python...
3
by: Will McGugan | last post by:
Hi, Is there a naming convention regarding alternative constructors? ie static methods where __new__ is called explicity. I use lower_case for methods in general, but thought maybe CamelCase...
0
by: sachintandon | last post by:
Hello all, Thanks in advance for your help I have a problem in sending emails, my requirement is to send multipart alternative emails with attachments, I'm able to send text with attachments or...
1
by: Lars B | last post by:
Hey guys, I have written a C++ program that passes data from a file to an FPGA board and back again using software and DMA buffers. In my program I need to compare the size of a given file against...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.