Telling an empty binary file from a "full" one

[Michel Rouzic]

I have a binary file used to store the values of variables in order to
use them again. I easily know whether the file exists or not, but the
problem is, in case the program has been earlier interupted before it
could write the variables to the file, the file is gonna be empty, and
then it's gonna load a load of crap into variables, which i want to
avoid.

That file is always 36 bytes big (it contains 4 double-precision floats
and one integer) and i'd like to be able to test whether it is 36 bytes
long or not, but it seems like quite a big problem to get to do it in a
portable way.

I thought that using fseek and ftell could work if the end of file
could be told but i read that "Setting the file position indicator to
end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior
for a binary stream (because of possible trailing null characters)"

My file has lots of zero bytes in it, so I guess it means it can't tell
then end of file reliably, right? I'd just like to know how I can, in a
reliable and portable way, tell the size of my binary file, and if not,
tell whether my file is empty or not

[Walter Roberson]

In article <11************ **********@o13g 2000cwo.googleg roups.com>,
Michel Rouzic <Mi********@yah oo.fr> wrote:

I have a binary file used to store the values of variables in order to
use them again. I easily know whether the file exists or not, but the
problem is, in case the program has been earlier interupted before it
could write the variables to the file, the file is gonna be empty, and
then it's gonna load a load of crap into variables, which i want to
avoid.
In that case, your load routine is programmed without due regard
to the circumstances.

Each fread() or fgetc() or fscanf() that you perform returns a
status. If there is any serious chance that the file might not
be of the proper size, you should be testing those return statuses,
and taking appropriate steps if you do not get enough data.
That file is always 36 bytes big (it contains 4 double-precision floats
and one integer)

Probably the easiest portable well to tell if the file is the right
size would be to attempt to fread) 37 bytes, and see whether you were
handed fewer bytes (file truncated), 36 bytes (right size), or 37 bytes
(file is too long.)

I would, though, make the point that you have emphasized portability
for the test, but the size of double-precision floats is not certain
to be 8 bytes, and integers are not certain to be 4 bytes.
It also appears that you might not have left room for any flags
to indicate representation format and to indicate which "endian"
the data is in. Portably stiching together a double from a binary
number is no fun -- fixed point or printable text or XDR are easier
to deal with in that regard.
--
Feep if you love VT-52's.

[Christopher Benson-Manica]

Michel Rouzic <Mi********@yah oo.fr> wrote:

That file is always 36 bytes big (it contains 4 double-precision floats
and one integer) and i'd like to be able to test whether it is 36 bytes
long or not, but it seems like quite a big problem to get to do it in a
portable way.

If you're assuming it will always be 36 bytes in size, you've already
left the realms of strict portability. Is there any particular reason
you're unable to simply store what sounds like a small amount of data
as text? It would make many things easier, in any case.

--
Christopher Benson-Manica | I *should* know what I'm talking about - if I
ataru(at)cybers pace.org | don't, I need to know. Flames welcome.

[Anonymous 7843]

In article <11************ **********@o13g 2000cwo.googleg roups.com>,
Michel Rouzic <Mi********@yah oo.fr> wrote:

My file has lots of zero bytes in it, so I guess it means it can't tell
then end of file reliably, right? I'd just like to know how I can, in a
reliable and portable way, tell the size of my binary file, and if not,
tell whether my file is empty or not

I'm curious as to what existing OS's do not accurately
report the lengths of binary files. Does anyone
have any examples?

The only situation like this that I've encountered is
that systems with unix-like device mapping can often
be coerced into opening a raw disk partition. Since
there is no file system present, fseek(SEEK_END) will
often position to the end of the partition whether
you have written meaningful data or not.

[Kenneth Brody]

Christopher Benson-Manica wrote:

Michel Rouzic <Mi********@yah oo.fr> wrote:

That file is always 36 bytes big (it contains 4 double-precision floats
and one integer) and i'd like to be able to test whether it is 36 bytes
long or not, but it seems like quite a big problem to get to do it in a
portable way.

If you're assuming it will always be 36 bytes in size, you've already
left the realms of strict portability. Is there any particular reason
you're unable to simply store what sounds like a small amount of data
as text? It would make many things easier, in any case.

s/36 bytes/4*sizeof(double )/

In my mind, there's a world of difference between "portable code" and
"code that generates portable data files". (One could also argue that
writing the file as text isn't 100% portable, as ASCII files won't read
correctly on an EBCDIC system.)

In any case, the "real" answer here is to check the return values from
the fread() calls to make sure the data was there to be read.

--
+-------------------------+--------------------+-----------------------------+
| Kenneth J. Brody | www.hvcomputer.com | |
| kenbrody/at\spamcop.net | www.fptech.com | #include <std_disclaimer .h> |
+-------------------------+--------------------+-----------------------------+
Don't e-mail me at: <mailto:Th***** ********@gmail. com>

[Gordon Burditt]

>> My file has lots of zero bytes in it, so I guess it means it can't tell

then end of file reliably, right? I'd just like to know how I can, in a
reliable and portable way, tell the size of my binary file, and if not,
tell whether my file is empty or not
I'm curious as to what existing OS's do not accurately
report the lengths of binary files. Does anyone
have any examples?

Under CP/M, the length of a file is a multiple of 128 bytes (the
size of a single-density floppy disk sector). A text file used ^Z
as an end-of-file marker, which was used when the file was opened
in text mode, to give a more fine-grained end-of-file. No such
marker could be used in binary mode, as ^Z is a legitimate binary
value that the file could contain. The file size was a number of
disk sectors.

C implementations (and there were some, although many of them left
out stuff like floating point) had to deal with the imprecise size
of binary files. A few used highly non-standard implementation
decisions (like int = 8 bits, on a z80 or 8080 processor). Others
were much closer to Standard C, but rather cramped as the machine
generally had only a 64k address space total (although some had
memory paging setups).
The only situation like this that I've encountered is
that systems with unix-like device mapping can often
be coerced into opening a raw disk partition. Since
there is no file system present, fseek(SEEK_END) will
often position to the end of the partition whether
you have written meaningful data or not.

That's another example.

Gordon L. Burditt

[Skarmander]

Kenneth Brody wrote:

Christopher Benson-Manica wrote:

Michel Rouzic <Mi********@yah oo.fr> wrote:

That file is always 36 bytes big (it contains 4 double-precision floats
and one integer) and i'd like to be able to test whether it is 36 bytes
long or not, but it seems like quite a big problem to get to do it in a
portable way.

If you're assuming it will always be 36 bytes in size, you've already
left the realms of strict portability. Is there any particular reason
you're unable to simply store what sounds like a small amount of data
as text? It would make many things easier, in any case.

s/36 bytes/4*sizeof(double )/

In my mind, there's a world of difference between "portable code" and
"code that generates portable data files". (One could also argue that
writing the file as text isn't 100% portable, as ASCII files won't read
correctly on an EBCDIC system.)

And by extension of that argument, "100% portable data" does not exist.
There is only data that is read more and less easily by various
languages on various platforms. But the intent is probably to literally
convey the sense of effort one has to expend to hoist it from one end to
another. ASCII-encoded integers are portable. Doubles encoded by
someone's C implementation are too heavy.

If your program is writing files, it's doing so because it needs to
communicate something across process boundaries. By some Rule or Law
someone no doubt coined, the mere potential to do things inspires the
desire to have them done. Therefore, it's wise to accommodate as broad a
range of processes as you can afford.

Writing binary data in the native format of your C implementation is
probably the narrowest range possible, and only justifiable by laziness.
It may be justifiable laziness, of course, but it's still laziness. Know
that it only works if the process on the other side of the boundary is
a program compiled by the exact same C implementation, running on the
exact same platform. Even upgrading your C library is taking chances --
very mild ones, but you should nevertheless be aware of them.

What am I saying? Oh yes, right. What the other posters said. Use a text
file. It's really not much more involved and saves you ever so much
potential grief.

S.

[Keith Thompson]

Kenneth Brody <ke******@spamc op.net> writes:

Christopher Benson-Manica wrote:

Michel Rouzic <Mi********@yah oo.fr> wrote:

> That file is always 36 bytes big (it contains 4 double-precision floats
> and one integer) and i'd like to be able to test whether it is 36 bytes
> long or not, but it seems like quite a big problem to get to do it in a
> portable way.

If you're assuming it will always be 36 bytes in size, you've already
left the realms of strict portability. Is there any particular reason
you're unable to simply store what sounds like a small amount of data
as text? It would make many things easier, in any case.

s/36 bytes/4*sizeof(double )/

s/4*sizeof(double )/4*sizeof(double )+sizeof(int)/

(assuming that "one integer" means "one int").

The most sensible approach *if* you don't care about portability of
the file is probably to declare a struct type

struct foo {
double a;
double b;
double c;
double d;
int e;
}

and use fread/fwrite to read and write values of type struct foo
directly to the file (in binary mode, of course). Never refer to
"36"; always use "sizeof(str uct foo)" or "sizeof obj" where obj is of
type struct foo.

The code should be portable to other platforms, but the data file will
not be; it will only be usable on the system where it was created.
That's likely to be good enough. (If it isn't, use some portable
external representation of the data; plain text is a good choice.)

And, of course, choose more descripive names, than a, b, c, d, e, and
foo.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Michel Rouzic]

Skarmander wrote:

Writing binary data in the native format of your C implementation is
probably the narrowest range possible, and only justifiable by laziness.
It may be justifiable laziness, of course, but it's still laziness. Know
that it only works if the process on the other side of the boundary is
a program compiled by the exact same C implementation, running on the
exact same platform.

No, I don't have this problem. The reason for that is that it's a
configuration file, it writes to a file whats in memory in order to use
it later. so it works both on big endian and little endian machines,
and indeed it can take absolutly any way of writing double-precision
floats, since it reads only what it writes.

and then, it's not laziness, rather ignorance, i never dealt yet with
using text files (i'm only at my second C program)

[Michel Rouzic]

Keith Thompson wrote:

The most sensible approach *if* you don't care about portability of
the file is probably to declare a struct type

struct foo {
double a;
double b;
double c;
double d;
int e;
}

and use fread/fwrite to read and write values of type struct foo
directly to the file (in binary mode, of course). Never refer to
"36"; always use "sizeof(str uct foo)" or "sizeof obj" where obj is of
type struct foo.

The code should be portable to other platforms, but the data file will
not be; it will only be usable on the system where it was created.
That's likely to be good enough. (If it isn't, use some portable
external representation of the data; plain text is a good choice.)

And, of course, choose more descripive names, than a, b, c, d, e, and
foo.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

thx, but it's quite off topic. I just want to test what's the size of
the file like, I already know how to write my data to it, or even read
it, my problem, is that I don't want to read a file if it's empty.