Programming in standard c

[jacob navia]

In my "Happy Christmas" message, I proposed a function to read
a file into a RAM buffer and return that buffer or NULL if
the file doesn't exist or some other error is found.

It is interesting to see that the answers to that message prove that
programming exclusively in standard C is completely impossible even
for a small and ridiculously simple program like the one I proposed.

1 I read the file contents in binary mode, what should allow me
to use ftell/fseek to determine the file size.

No objections to this were raised, except of course the obvious
one, if the "file" was some file associated with stdin, for
instance under some unix machine /dev/tty01 or similar...

I did not test for this since it is impossible in standard C:
isatty() is not in the standard.

2) There is NO portable way to determine which characters should be
ignored when transforming a binary file into a text file. One
reader (CB Falconer) proposed to open the file in binary mode
and then in text mode and compare the two buffers to see which
characters were missing... Well, that would be too expensive.

3) I used different values for errno defined by POSIX, but not by
the C standard, that defines only a few. Again, error handling
is not something important to be standardized, according to
the committee. errno is there but its usage is absolutely
not portable at all and goes immediately beyond what standard C
offers.

We hear again and again that this group is about standard C *"ONLY"*.
Could someone here then, tell me how this simple program could be
written in standard C?

This confirms my arguments about the need to improve the quality
of the standard library!

You can't do *anything* in just standard C.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[William Pursell]

On Dec 26, 11:25 pm, jacob navia <[e-mail]:

Eric Sosman wrote:

You wrote: "You can't do *anything* in just standard C."
Do you stand by that statement, or do you retreat from it?
If you stand by it, why are you here?

int main(void) { int n = printf("hello\n");}
How much is n?

No way to know since the error codes of printf
are NOT standardized. This means that I can only
know that n can be... *ANYTHING*. Maybe it wrote
some characters, then stopped, or whatever!

If you want to know what n is, then you check
it. Perhaps like so:

int main(void) {
int n = printf("hello\n");
printf(" n = %d\n", n );
return 0;}
That's really not so hard.

[dj3vande]

In article <1a**************************@cache4.tilbu1.nb.hom e.nl>,
Serve La <[e-mail]:

>
<dj******@csclub.uwaterloo.ca.invalidschreef in bericht
news:fl**********@rumours.uwaterloo.ca...

>Now what happens when somebody decides to log to the printer? Does it
still sound silly to be passing "LPT1" to filesize?

yes
if one decides that i;m sure there will be other unportable means to do it

So we've gone from having a non-portable way to do something that's
useful and meaningful on some systems, to having a non-portable way to
do something that's useful and meaningful on some systems.

Remind me again what the point of this exercise was?
dave

[Keith Thompson]

"Serve La" <[e-mail]:
[...]

How about let filesize("/var/adm/messages") and others UB?? On some
systems it could return correct values on others an error

fflush(stdin) is UB so i dont see a reason why filesize should have
every single filetype perfectly defined.
And here in clc people will have 1 more reason to tell others that
demons will fly out of their nose when they try filesize() on
/dev/random or something!

Presumably filesize("/dev/random") could behave sensibly without
invoking undefined behavior. The system could detect that it's a
"character special file" and return an error indication. An
implementation of filesize() might fall back to reading every byte of
the file in some cases, but only when necessary. There's nothing
special about filesize("/var/adm/messages"); it's just an ordinary
file whose size happens to change a lot.

The set of functions in the C standard library is not a carefully
engineered coherent interface. The choice of which functions to
include and which to exclude has been largely arbitrary. strdup() is
fairly useful and commonly provided, but it was excluded by the
standard committee back in 1989. gets() is positively dangerous, but
it was included because of historical precedent (and was finally
deprecated nearly 20 years later). strtok() has its own set of
problems. The time interfaced is incomplete (mktime() is the inverse
of localtime(), but no inverse for gmtime() is provided, and asctime()
unnecessarily invokes undefined behavior for out-of-range arguments).
And so forth.

I'll note that Ada, which was first standardized several years before
C was, does have a function to obtain the current size of a file (it
takes Ada's equivalent of a FILE*, not a file name). The designers of
Ada were as concerned with portability as the designers of C; Ada,
like C, runs on everything from small embedded systems to PCs to
mainframes and supercomputers.

A filesize() function, with all its limitations, *could* have been
included in the standard. It wasn't. If it had been, it might have
been useful in some contexts, but you'd have to be very careful in
using it. I suspect that a function that takes a FILE* would be more
useful than one that takes a file name; perhaps both could have been
provided.

But there's always POSIX, which *does* provide this functionality via
the stat() and fstat() functions. The fact is that most, perhaps all,
systems on which filesize() would make sense provide it via an
extension. Yes, it's inconvenient to have to use different interfaces
on different systems.

I have no strong opinion on whether a filesize() function *should*
have been in the C standard. The fact is that it isn't. The lack has
not been a serious problem, since the functionality is generally
available when needed. And adding it to the standard now would be a
long and slow process -- one that is not particularly helped (or hurt)
by talking about it here.

If you really want filesize() in the C standard, one way to start
making that happen is to write a proposal and post it to comp.std.c
for discussion.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
[...]
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[dj3vande]

In article <Lf******************************@bt.com>,
Richard Heathfield <rj*@see.sig.invalidwrote:

>Stephen Montgomery-Smith said:

> But perhaps we should see standard C as perhaps a tool to be embedded
into real C, rather than as an object with value in of itself.

How do you feel about s/rather than/as well/ - because I think that such a
change reflects reality rather more closely.

I think I prefer to think of it in terms of C-with-extensions being a
tool to be used to supplement standard C when necessary. This is
dual[1] to Stephen's statement, and I think it more closely captures
reality than putting them on equal footing; there is a lot of useful
code that can be written in standard C in between the bits that you
need extensions for, but I've yet to encounter a problem (though they
probably exist) that doesn't have nontrivial subproblems that can (and
often should) be solved entirely with standard C. It's also worth
noting that without a well-defined (but not excessively strict)
language to build on, the extensions would be a lot harder to use and a
lot less useful.

Certainly for my own part, I
know that my use of what you call "real C" (by which you appear to mean "C
+ non-ISO9899 libraries") is dwarfed by my use of ISO C. Most of the C
programs I write are ISO C programs. Only a very small proportion use
non-ISO9899 libraries.

If you replace "programs" with "translation units" or "lines of code",
this is also consistent with my experience. Most of the code I write
does interesting things with data (which rarely if ever needs anything
beyond standard C) and interfaces with the rest of the world by using
system-specific code (which has already been written, often but not
always for every platform I'm interested in) to talk to something at
the other end of a network connection.

(It is, however, far from unheard of for the system-specific parts to
be moved out of the program and to the ends of a unix pipeline, leaving
a completely standard C program (or several) running offline or in the
middle of a network-server-in-a-line-of-shell-code, instead of linking
it with the code that's doing the interesting work.)
dave

[1] 'dual' is used here in its sense as a technical mathematical term
meaning approximately "describes the same thing from a different
perspective"

[Gordon Burditt]

>On the other hand, I think that there was a call somewhere in this

>thread for fileinformation() which I think might be a very nice
addition. I guess that even filesize() might not be so bad if they
renamed it currentfilesize(). Then (at least) it would be obvious
that it only contains an estimate.

No, it should be named previousfilesize(), since that's what it
returns.

[Richard]

Mark McIntyre <ma**********@spamcop.netwrites:

On Sat, 29 Dec 2007 02:02:41 +0100, jacob navia wrote:

>Mark McIntyre wrote:

>>On Fri, 28 Dec 2007 01:31:12 +0100, jacob navia wrote:

Look at that:

AlphaServer DS10L Noname pc clone (my machine) 466Mhz CPU
EV6 2GHZ Dual core AMD

Remember, CISC vs RISC !

The RISC idea was to reduce the instructions and speed up the clock.

No. Suggest you read up on this.

Pedantic.

Reduced instruction sets lead to the dropping of many features which ate
up lots of clock cycles which were hardly ever used.

The main idea being to produce a small suite of very fast useful
instructions.

In addition, the reduction in complex instructions meant more room for
fast registers and resulting higher CPU operating speeds.

I suggest you read up on this.

[CBFalconer]

jacob navia wrote:

CJ wrote:

>CBFalconer wrote:

>>Remember that you can't count on the availability of more than
64 Kbytes of memory (although most systems provide more).

Many C programs will need to get by on a lot less memory than
this! I know that for Jacob every computer is a 32-bit Windows
box with a couple of gigabytes of RAM, but out there in the real
world C programs often control things like toasters or kettles,
where memory is severely limited.

Look "CJ" whoever you are:

You know NOTHING of where I have programmed, or what I am doing.
Versions of lcc-win run in DSPs with 80k of memory, and only
20 usable.

Does that mean you have finally uncovered the reason it won't run
on a 486 under W98?

You (like all the "regulars") repeat the same lies about me again
and again but that doesn't makes them true.

Please specify some toasters or kettles controlled by lcc-win32.
Please also specify the breeds of CPU chips that are compatible.

You feel like insulting someone? Pick up another target or (much
better) try to stop kissing ass ok?

I saw no insult in CJs message. I did in yours.

--
Merry Christmas, Happy Hanukah, Happy New Year
Joyeux Noel, Bonne Annee, Frohe Weihnachten
Chuck F (cbfalconer at maineline dot net)
<http://cbfalconer.home.att.net>

--
Posted via a free Usenet account from http://www.teranews.com

[Eric Sosman]

Keith Thompson wrote:

"Serve La" <[e-mail]:
[...]

>How about let filesize("/var/adm/messages") and others UB?? On some
systems it could return correct values on others an error

fflush(stdin) is UB so i dont see a reason why filesize should have
every single filetype perfectly defined.
And here in clc people will have 1 more reason to tell others that
demons will fly out of their nose when they try filesize() on
/dev/random or something!

Presumably filesize("/dev/random") could behave sensibly without
invoking undefined behavior. The system could detect that it's a
"character special file" and return an error indication. An
implementation of filesize() might fall back to reading every byte of
the file in some cases, but only when necessary. [...]

On all systems I know of that provide a "/dev/random"
file, reading all the bytes would take too long ...

(R.H.: Any update on the progress of your infinite
loop test?)

--
Eric Sosman
es*****@ieee-dot-org.invalid

[Richard Heathfield]

Eric Sosman said:

<snip>

(R.H.: Any update on the progress of your infinite
loop test?)

Alas, it seems that infinity is bigger than I at first realised. It would
be premature to give a percentage-completed report at this stage.

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Kelsey Bjarnason]

[snips]

On Fri, 28 Dec 2007 00:23:02 +0100, jacob navia wrote:

There is no point in discussing with somebody that doesn't want
to go into the arguments of the other side. All multi user
OSes provide a "stat" function. This function gives of
course meaningless results and since at least 25 years progammers
use those "meaningless" results. Go figure, they did not know
"user923005" and the virtues of openvms.

P.S. by the way, in the reference I gave to the openvms wizard
there is a COBOL program to get the exact file size and other
attributes of the file. It wasn't anything very difficult as you say.

If you're going to change contexts - move the goalposts - don't be
surprised to find that the results don't match the problem.

[Kelsey Bjarnason]

[snips]

On Fri, 28 Dec 2007 12:27:18 +0100, jacob navia wrote:

I am not surprised you say this either. Till now, there hasn't been
ANY system where there wasn't an operation to get the file size.
VAX/VMS included. And if there was one, it could ALWAYS do
1) Open the file
2) Read until EOF

to get the file size. Granted, it wouldn't be efficient in some weird
systems but so what? It would be possible.

But it would not necessarily be *meaningful* which you persist in
overlooking.

If you do this and calculate you need to allocate 17,403 bytes to read the
file, do you know that in the time it takes you to allocate the space,
rewind and commence reading, someone else has not changed the size of the
file?

In a single-user, single-tasking system, not a problem, presumably. In a
multi-user system where the file is shared between many users, it's
entirely possible your calculations aren't valid by the time you use them.

So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

[Kelsey Bjarnason]

[snips]

On Wed, 26 Dec 2007 23:12:30 +0100, jacob navia wrote:

>but for streams which
correspond to devices, rather than files, there's really no
alternative.

We could restrict this to normal files.

What is a "normal" file?

There are file systems supporting sparse files, which in most cases appear
to be "normal" files, except that while they're reported to be of one
size, they're actually another size entirely - they claim to be 2GB, but
only occupy 200K on disk, for example.

Other systems allow for compressed or partially compressed files, where
the actual size of the data bears little, if any, relation to the size of
the file on disk.

And on and on and on.

What's needed, then, is not a way to calculate size on disk, but "size to
be read" - actual number of bytes to be processed by your code, after any
applicable decompression, translation, etc, etc, etc - and further, a way
to calculate this such that the results are valid despite the potential
for other users or processes to modify the file.

How, exactly, do you figure on accomplishing this, hmm?

[Gordon Burditt]

>But it would not necessarily be *meaningful* which you persist in

>overlooking.

A similar problem occurs for people wanting "how many people are
using this file" functions, intending to use them in place of file
locking.

>If you do this and calculate you need to allocate 17,403 bytes to read the
file, do you know that in the time it takes you to allocate the space,
rewind and commence reading, someone else has not changed the size of the
file?

The size of the file can be changed before you have a chance to
even receive the result of the hypothetical filesize() function.
It doesn't have to be while you're doing a calculation with it.

>In a single-user, single-tasking system, not a problem, presumably. In a

If the file in question is the file to which stdout of this program
is redirected, it could be a problem anyway. Consider something
like "DIR log.txt" under MS-DOS. What size of the file log.txt
does it show? You can get the same problem with "ls -l log.txt"
under UNIX, without needing two processes accessing the file
simultaneously.

You can also run into problems with commands like
"md5sum * md5.checksums", which computes the MD5 checksum of a
bunch of files. What checksum is shown for the "md5.checksums"
file (which is included in the expanded wildcard).

>multi-user system where the file is shared between many users, it's
entirely possible your calculations aren't valid by the time you use them.

It is also entirely possible that the file size has been changed
by someone with MALICIOUS INTENT. Someone who insists that the
file size can't change is likely to code in a way that causes a
buffer overflow when it does change size. This can lead to arbitrary
code execution, which sometimes leads to the computer becoming part
of a botnet. Take a look at patches issued by Microsoft for Windows
and related programs. Many of them are related to buffer overflows
by maliciously malformed data of some kind.

>So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

[Serve Lau]

"Keith Thompson" <ks***@mib.orgschreef in bericht
news:87************@kvetch.smov.org...

A filesize() function, with all its limitations, *could* have been
included in the standard. It wasn't. If it had been, it might have
been useful in some contexts, but you'd have to be very careful in
using it. I suspect that a function that takes a FILE* would be more
useful than one that takes a file name; perhaps both could have been
provided.

Yes I think too that such a function could be standardized and where the
function doesnt make any sense it could return an error or be excluded.
I just gave the string argument as an example to get it all on one line ;)
I'd prefer a FILE * in the real interface too.

I have no strong opinion on whether a filesize() function *should*
have been in the C standard. The fact is that it isn't. The lack has
not been a serious problem, since the functionality is generally
available when needed. And adding it to the standard now would be a
long and slow process -- one that is not particularly helped (or hurt)
by talking about it here.

I agree, all systems I know where such a function could be useful have fstat
so its already possible to write portable code for your application domain.
No real need to standardize but it would be nice. I just found the
resistance to adding such a function in this thread strange

If you really want filesize() in the C standard, one way to start
making that happen is to write a proposal and post it to comp.std.c
for discussion.

Jacob seems to want it more judging by his use of capitals in his posts :P

[Gordon Burditt]

>We could restrict this to normal files.

>
What is a "normal" file?

There are file systems supporting sparse files, which in most cases appear
to be "normal" files, except that while they're reported to be of one
size, they're actually another size entirely - they claim to be 2GB, but
only occupy 200K on disk, for example.

But they take 2GB to read into memory, if that is the number of interest.

>Other systems allow for compressed or partially compressed files, where
the actual size of the data bears little, if any, relation to the size of
the file on disk.

But the uncompressed size is the relevant number for space to read it
into memory.

>And on and on and on.

What's needed, then, is not a way to calculate size on disk, but "size to
be read" - actual number of bytes to be processed by your code, after any
applicable decompression, translation, etc, etc, etc - and further, a way
to calculate this such that the results are valid despite the potential
for other users or processes to modify the file.

>How, exactly, do you figure on accomplishing this, hmm?

Mandatory file locking is one way to accomplish this, but it has
other problems. It opens the system up to denial-of-service attacks
by a program that locks lots of important system files to keep
administrators out, then proceeds to do something evil.

[Serve Lau]

<dj******@csclub.uwaterloo.ca.invalidschreef in bericht
news:fl**********@rumours.uwaterloo.ca...

In article <1a**************************@cache4.tilbu1.nb.hom e.nl>,
Serve La <[e-mail]:

>>
<dj******@csclub.uwaterloo.ca.invalidschreef in bericht
news:fl**********@rumours.uwaterloo.ca...

>>Now what happens when somebody decides to log to the printer? Does it
still sound silly to be passing "LPT1" to filesize?

yes
if one decides that i;m sure there will be other unportable means to do it

So we've gone from having a non-portable way to do something that's
useful and meaningful on some systems, to having a non-portable way to
do something that's useful and meaningful on some systems.

Remind me again what the point of this exercise was?

Portable code which checks a file size can be written then but a programmer
cant just expect to do something strange like check the size of "LPT1" and
expect the standard C function to work correctly. In fact writing
fopen("LPT1:", "r"); is already not portable anymore in the sense that your
code wont work on multiple systems. Same goes for most of the other system
dependant files mentioned.

>

dave

[jacob navia]

Kelsey Bjarnason wrote:

[snips]

On Fri, 28 Dec 2007 12:27:18 +0100, jacob navia wrote:

>I am not surprised you say this either. Till now, there hasn't been
ANY system where there wasn't an operation to get the file size.
VAX/VMS included. And if there was one, it could ALWAYS do
1) Open the file
2) Read until EOF

to get the file size. Granted, it wouldn't be efficient in some weird
systems but so what? It would be possible.

But it would not necessarily be *meaningful* which you persist in
overlooking.

If you do this and calculate you need to allocate 17,403 bytes to read the
file, do you know that in the time it takes you to allocate the space,
rewind and commence reading, someone else has not changed the size of the
file?

In a single-user, single-tasking system, not a problem, presumably. In a
multi-user system where the file is shared between many users, it's
entirely possible your calculations aren't valid by the time you use them.

So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Yes, this is 100% clear logic. I will apply it then.

1) You can't open a file with
fopen("name","a+")
since somebody else could grow the file after the file is positioned at
EOF, so you would overwrite his data.

2) ftell/fseek/ and in general all file primitives should be dropped
from the standard. If you write something somebody else could have
written at the same position. You should not write in a file.

Nice isn't it?

I have answered this thousand times but you still come back with the
same nonsense!

--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[jacob navia]

Serve Lau wrote:

>
Portable code which checks a file size can be written then but a
programmer cant just expect to do something strange like check the size
of "LPT1" and expect the standard C function to work correctly. In fact
writing
fopen("LPT1:", "r"); is already not portable anymore in the sense that
your code wont work on multiple systems. Same goes for most of the other
system dependant files mentioned.

Yes. We should ban fopen/fwrite/ and all file primitives.

This has been answered thousand times but this people come back again
and again with the same nonsense.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Serve Lau]

"William Pursell" <bi**********@gmail.comschreef in bericht
news:59**********************************@f52g2000 hsa.googlegroups.com...

If you want to know what n is, then you check
it. Perhaps like so:

int main(void) {
int n = printf("hello\n");
printf(" n = %d\n", n );
return 0;}

*waiting for the regulars to point out the obvious flaw in this code*

[Gordon Burditt]

>I agree, all systems I know where such a function could be useful have fstat

>so its already possible to write portable code for your application domain.
No real need to standardize but it would be nice. I just found the
resistance to adding such a function in this thread strange

The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

>If you really want filesize() in the C standard, one way to start
making that happen is to write a proposal and post it to comp.std.c
for discussion.

Jacob seems to want it more judging by his use of capitals in his posts :P

Does Jacob have a financial interest in botnets?

[Richard]

Kelsey Bjarnason <kb********@gmail.comwrites:

[snips]

On Fri, 28 Dec 2007 12:27:18 +0100, jacob navia wrote:

>I am not surprised you say this either. Till now, there hasn't been
ANY system where there wasn't an operation to get the file size.
VAX/VMS included. And if there was one, it could ALWAYS do
1) Open the file
2) Read until EOF

to get the file size. Granted, it wouldn't be efficient in some weird
systems but so what? It would be possible.

But it would not necessarily be *meaningful* which you persist in
overlooking.

If you do this and calculate you need to allocate 17,403 bytes to read the
file, do you know that in the time it takes you to allocate the space,
rewind and commence reading, someone else has not changed the size of the
file?

How ridiculous. The data he READS at the time was that size. That is ALL
he can account for. Clearly writing back needs other checks or some sort
of locking out of others (transaction management) IF the data
manipulations he performs on his buffer do indeed need to be written
back.

>
In a single-user, single-tasking system, not a problem, presumably. In a
multi-user system where the file is shared between many users, it's
entirely possible your calculations aren't valid by the time you use them.

So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

[Gordon Burditt]

>So, explain to us how this file size you've calculated means anything

>relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Yes, this is 100% clear logic. I will apply it then.

1) You can't open a file with
fopen("name","a+")
since somebody else could grow the file after the file is positioned at
EOF, so you would overwrite his data.

In some implementations, the file is opened with the POSIX O_APPEND
flag when you fopen with mode "a" or "a+", so *EVEN IF* someone
else grows the file, you will still add on to the end of the file.

>2) ftell/fseek/ and in general all file primitives should be dropped
from the standard. If you write something somebody else could have
written at the same position. You should not write in a file.

If you are working with databases, isn't the whole point that you should
see writes made by other programs?

>I have answered this thousand times but you still come back with the
same nonsense!

Use of filesize() to allocate a buffer, and then assuming the file size
has not changed encourages virus-friendly programming. You seem to be
advocating buffer overflows. The other problems you mention seem to be
limited to loss of data.

[Gordon Burditt]

>So, explain to us how this file size you've calculated means anything

>relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

Do you get paid by the buffer overflow?

[Serve Lau]

"Kelsey Bjarnason" <kb********@gmail.comschreef in bericht
news:ar************@spanky.localhost.net...

What is a "normal" file?

There are file systems supporting sparse files, which in most cases appear
to be "normal" files, except that while they're reported to be of one
size, they're actually another size entirely - they claim to be 2GB, but
only occupy 200K on disk, for example.

Other systems allow for compressed or partially compressed files, where
the actual size of the data bears little, if any, relation to the size of
the file on disk.

And what do you expect that fread will put into the buffer on such a
filesystem? The compressed or decompressed data?

[Army1987]

Serve Lau wrote:

Portable code which checks a file size can be written then but a programmer
cant just expect to do something strange like check the size of "LPT1" and
expect the standard C function to work correctly. In fact writing
fopen("LPT1:", "r"); is already not portable anymore in the sense that your
code wont work on multiple systems. Same goes for most of the other system
dependant files mentioned.

Yeah, reading from a printer is veeeeery non-portable...
--
Army1987 (Replace "NOSPAM" with "email")

[Serve Lau]

"Army1987" <ar******@NOSPAM.itschreef in bericht
news:fl**********@tdi.cu.mi.it...

Serve Lau wrote:

>Portable code which checks a file size can be written then but a
programmer
cant just expect to do something strange like check the size of "LPT1"
and
expect the standard C function to work correctly. In fact writing
fopen("LPT1:", "r"); is already not portable anymore in the sense that
your
code wont work on multiple systems. Same goes for most of the other
system
dependant files mentioned.

Yeah, reading from a printer is veeeeery non-portable...

Not sure what you mean with this, does fopen("LPT1", "r") return a valid
FILE * on Linux for instance?

[jacob navia]

Gordon Burditt wrote:

Use of filesize() to allocate a buffer, and then assuming the file size
has not changed encourages virus-friendly programming. You seem to be
advocating buffer overflows. The other problems you mention seem to be
limited to loss of data.

I allocate the buffer given by filesize, then I read exactly that with
fread.

If the file has grown I will ignore what goes beyond, and if the file
shrinks, I will have a short read and a too large buffer.

There is NO WAY I COULD HAVE A BUFFER OVERFLOW!

For the nth time:

You are telling just NONSENSE!
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Serve Lau]

"Gordon Burditt" <go****@hammy.burditt.orgschreef in bericht
news:13*************@corp.supernews.com...

>>So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read
the
entire file into a buffer.

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

So? Like Jacob has indeed said 1000 times already, by that same logic you
can't use fread or fgets anymore either.
Somebody else could've made the file smaller when you started fgets on the
last line.

[Serve Lau]

"user923005" <dc*****@connx.comschreef in bericht
news:89**********************************@s27g2000 hsb.googlegroups.com...
On Dec 28, 4:32 am, Chris Torek <[e-mail]:

In article <[e-mail]>

Malcolm McLean <[e-mail]:

/*
function to slurp in an ASCII file
Params: path - path to file
Returns: malloced string containing whole file
*/

I think we can improve this a great deal, with the result being
a function that is written entirely in Standard C and works in
every case in which it is possible for it to work, and -- by
calling a system-dependent function that the user is to supply,
but which may be replaced with a #define that simply returns 0
if desired -- is "reasonably efficient" as well.

[snip]

I work for a database company, and most of our customers are large
customers. (E.g. huge US company, large university, government of
country x, etc.)

It is not at all unusual for a single file to be 20-100 GB. Needless
to say, you would not want to put this file into memory even if you
could do it.
The files I work with are also being modified constantly (though there
are occasionally windows of inactivity for some of them).
I am quite sure that the goal of reliably reading these sorts of files
into memory and doing something useful with them is literally
infeasible (not impossible, but the cost would make it so stupid that
nobody would want to do it).

What is your point? This DBMS doesnt come with an API to update the
database? You have to use fread to get records? Surely not. The reasons for
not including a function like filesize in the standard are getting funnier
by the minute.

There are BTW more uses for a filesize function than just get the size to
malloc memory with. How about for your DBMS and you want to create an
administration tool where you can constantly monitor how big the files are.
Just an extra thingie you would be able to do in standard C.

This is really getting silly, a function like filesize could be put into the
standard and it would be useful. But as pointed out elsewhere in this thread
this discussion wont help to get the function into it.

[Walter Roberson]

In article <44**************************@cache6.tilbu1.nb.hom e.nl>,
Serve Lau <[e-mail]:

>
"William Pursell" <bi**********@gmail.comschreef in bericht
news:59**********************************@f52g200 0hsa.googlegroups.com...

>If you want to know what n is, then you check
it. Perhaps like so:

int main(void) {
int n = printf("hello\n");
printf(" n = %d\n", n );
return 0;}

>*waiting for the regulars to point out the obvious flaw in this code*

<stdio.hnot included?
--
"I was very young in those days, but I was also rather dim."
-- Christopher Priest

[=?UTF-8?q?Harald_van_D=C4=B3k?=]

On Sun, 30 Dec 2007 22:29:32 +0100, Serve Lau wrote:

"Army1987" <ar******@NOSPAM.itschreef in bericht
news:fl**********@tdi.cu.mi.it...

>Serve Lau wrote:

>>fopen("LPT1:", "r");

Yeah, reading from a printer is veeeeery non-portable...

Not sure what you mean with this, does fopen("LPT1", "r") return a valid
FILE * on Linux for instance?

If a file named LPT1 exists, sure. It's just another name.

But on those systems where LPT1 is a printer device, you should probably
be writing to it, not reading from it.

[Serve Lau]

"Walter Roberson" <ro******@ibd.nrc-cnrc.gc.caschreef in bericht
news:fl**********@canopus.cc.umanitoba.ca...

In article <44**************************@cache6.tilbu1.nb.hom e.nl>,
Serve Lau <[e-mail]:

>>
"William Pursell" <bi**********@gmail.comschreef in bericht
news:59**********************************@f52g20 00hsa.googlegroups.com...

>>If you want to know what n is, then you check
it. Perhaps like so:

int main(void) {
int n = printf("hello\n");
printf(" n = %d\n", n );
return 0;}

>>*waiting for the regulars to point out the obvious flaw in this code*

<stdio.hnot included?

We have a winner! You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the toaster
XML configurable over the internet and the developers could not get the size
of the xml files and no network connection with standard C so they opted for
embedded java. You can burn patterns in your bread with it!

[jacob navia]

Gordon Burditt wrote:

>>So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

Do you get paid by the buffer overflow?

There is no buffer overflow as I told you in another message.

I repeat it now since you went silent instead of acknowledging it.

1) I call filesize
2) I allocate a buffer with the result+1.
3) I read with fread into that buffer.

If the file is bigger, the fread argument will ensure I read ONLY
what I allocated. I ignore the rest. If the file is smaller I get
a short read and the buffer is too big.

THERE IS NO POSSIBILITY OF A BUFFER OVERFLOW.

I suppose you will stay silent now, instead of acknowledging your
error.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Serve Lau]

"Harald van Dijk" <tr*****@gmail.comschreef in bericht
news:b3***************************@cache4.tilbu1.n b.home.nl...

On Sun, 30 Dec 2007 22:29:32 +0100, Serve Lau wrote:

>"Army1987" <ar******@NOSPAM.itschreef in bericht
news:fl**********@tdi.cu.mi.it...

>>Serve Lau wrote:
fopen("LPT1:", "r");

Yeah, reading from a printer is veeeeery non-portable...

Not sure what you mean with this, does fopen("LPT1", "r") return a valid
FILE * on Linux for instance?

If a file named LPT1 exists, sure. It's just another name.

But on those systems where LPT1 is a printer device, you should probably
be writing to it, not reading from it.

usenet is funny lol, I'm sure you are smarter irl as you make it out be
right now

But ok, let me rephrase!

If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize) and you use
fopen("LPT1") will you expect the program to actually open the printer on
Linux?

Is it clear now what I meant? rofl

[Keith Thompson]

go***********@burditt.org (Gordon Burditt) writes:

>>I agree, all systems I know where such a function could be useful have fstat
so its already possible to write portable code for your application domain.
No real need to standardize but it would be nice. I just found the
resistance to adding such a function in this thread strange

[The above was written by Serve Lau; Gordon, as always, snipped the
attribution line.]

The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

No, it's not useless.

Determine the *current* size of the file by some system-specific
method; let's say it's exactly 1 megabyte. Allocate buffer of exactly
1 megabyte, and attempt to read exactly 1 megabyte of data from the
file into the buffer. If the file has shrunk, you won't get the full
megabyte. If the file has grown, you won't get the entire current
contents of the file; you can then either realloc() the buffer, or
just use the data you've got. In neither case do you get a buffer
overrun.

Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.

The above was written by Keith Thompson <ks***@mib.org>. Gordon, if
you post a followup, please quote this paragraph. I do not give
permission to quote my words without attribution.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
[...]
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[=?UTF-8?q?Harald_van_D=C4=B3k?=]

On Sun, 30 Dec 2007 23:16:52 +0100, Serve Lau wrote:

"Harald van Dijk" <tr*****@gmail.comschreef in bericht
news:b3***************************@cache4.tilbu1.n b.home.nl...

>On Sun, 30 Dec 2007 22:29:32 +0100, Serve Lau wrote:

>>"Army1987" <ar******@NOSPAM.itschreef in bericht
news:fl**********@tdi.cu.mi.it...
Serve Lau wrote:
fopen("LPT1:", "r");

Yeah, reading from a printer is veeeeery non-portable...

Not sure what you mean with this, does fopen("LPT1", "r") return a
valid FILE * on Linux for instance?

If a file named LPT1 exists, sure. It's just another name.

But on those systems where LPT1 is a printer device, you should
probably be writing to it, not reading from it.

usenet is funny lol, I'm sure you are smarter irl as you make it out be
right now

If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize)

That simply doesn't make sense, which is what I believe Army1987's point
was, and which I restated.

and you
use fopen("LPT1") will you expect the program to actually open the
printer on Linux?

I answered that already. LPT1 is just another file name. You might get
the printer if someone has set up a compatibility symlink or special
device for your application. You might get an ordinary file if you
previously wrote to it. Or it might simply not exist.

[jacob navia]

Keith Thompson wrote:

go***********@burditt.org (Gordon Burditt) writes:

>>I agree, all systems I know where such a function could be useful have fstat
so its already possible to write portable code for your application domain.
No real need to standardize but it would be nice. I just found the
resistance to adding such a function in this thread strange

[The above was written by Serve Lau; Gordon, as always, snipped the
attribution line.]

>The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

No, it's not useless.

Determine the *current* size of the file by some system-specific
method; let's say it's exactly 1 megabyte. Allocate buffer of exactly
1 megabyte, and attempt to read exactly 1 megabyte of data from the
file into the buffer. If the file has shrunk, you won't get the full
megabyte. If the file has grown, you won't get the entire current
contents of the file; you can then either realloc() the buffer, or
just use the data you've got. In neither case do you get a buffer
overrun.

Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.

The above was written by Keith Thompson <ks***@mib.org>. Gordon, if
you post a followup, please quote this paragraph. I do not give
permission to quote my words without attribution.

Why do you ignore what I am saying?

I posted an identical message in this thread proving I could not
possibly have a buffer overflow.

You say the same thing without quoting me.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Serve Lau]

"Harald van Dijk" <tr*****@gmail.comschreef in bericht
news:5a***************************@cache1.tilbu1.n b.home.nl...

>and you
use fopen("LPT1") will you expect the program to actually open the
printer on Linux?

I answered that already. LPT1 is just another file name. You might get
the printer if someone has set up a compatibility symlink or special
device for your application. You might get an ordinary file if you
previously wrote to it. Or it might simply not exist.

yeah play more games, move along please nothing to see here

[Ben Pfaff]

jacob navia <ja***@nospam.comwrites:

Why do you ignore what I am saying?

The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)
--
Ben Pfaff
http://benpfaff.org

[Richard Heathfield]

Serve Lau said:

<snip>

You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.

Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[jacob navia]

Ben Pfaff wrote:

jacob navia <ja***@nospam.comwrites:

>Why do you ignore what I am saying?

The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)

Of course I am not in his killfile when it comes to
attacking me.

My point is proved. The silence of those people speaks for itself.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Serve Lau]

"Richard Heathfield" <rj*@see.sig.invalidschreef in bericht
news:dO******************************@bt.com...

Serve Lau said:

<snip>

>You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.

Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

Java already faced them you mean because such an operation is included in
*tadam* the standard library :)

[Flash Gordon]

Ben Pfaff wrote, On 30/12/07 22:53:

jacob navia <ja***@nospam.comwrites:

>Why do you ignore what I am saying?

The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)

Or Jacob's message had not reached Keith, or it had but Keith has not
read it yet, or he just happened to respond to a message on a different
part of the thread.

Jacob, you should not assume people are always out to get you.
--
Flash Gordon

[Randy Howard]

On Sun, 30 Dec 2007 16:15:13 -0600, jacob navia wrote
(in article <fl**********@aioe.org>):

Gordon Burditt wrote:

>>>So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.
Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

Do you get paid by the buffer overflow?

There is no buffer overflow as I told you in another message.

I repeat it now since you went silent instead of acknowledging it.

1) I call filesize
2) I allocate a buffer with the result+1.
3) I read with fread into that buffer.

If the file is bigger, the fread argument will ensure I read ONLY
what I allocated. I ignore the rest. If the file is smaller I get
a short read and the buffer is too big.

THERE IS NO POSSIBILITY OF A BUFFER OVERFLOW.

Except you left out some steps. Like checking for and handling errors.
Note my caps lock key isn't suffering from intermittent sticking
problems as yours seems to be.
--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw

[Randy Howard]

On Sun, 30 Dec 2007 17:50:58 -0600, Serve Lau wrote
(in article <68***************************@cache5.tilbu1.nb.ho me.nl>):

>
"Richard Heathfield" <rj*@see.sig.invalidschreef in bericht
news:dO******************************@bt.com...

>Serve Lau said:

<snip>

>>You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.

Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

Java already faced them you mean because such an operation is included in
*tadam* the standard library :)

Enlighten as to how this Java function handles real time changes to
file size between the time you make the call and the time you use the
result.
--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw

[Randy Howard]

On Sun, 30 Dec 2007 15:51:06 -0600, Serve Lau wrote
(in article <1e***************************@cache4.tilbu1.nb.ho me.nl>):

I work for a database company, and most of our customers are large
customers. (E.g. huge US company, large university, government of
country x, etc.)

It is not at all unusual for a single file to be 20-100 GB. Needless
to say, you would not want to put this file into memory even if you
could do it.

Especially since odds are the underlying file system buffer cache will
do it for you anyway.

[snip]

What is your point? This DBMS doesnt come with an API to update the
database? You have to use fread to get records? Surely not. The reasons for
not including a function like filesize in the standard are getting funnier
by the minute.

They might be funny to someone that doesn't realize that C runs on a
lot of platforms other than those they he/she is familiar with. I'm
not laughing, however.

There are BTW more uses for a filesize function than just get the size to
malloc memory with. How about for your DBMS and you want to create an
administration tool where you can constantly monitor how big the files are.
Just an extra thingie you would be able to do in standard C.

So you could make sure that the normal I/O to the files happens slower
than it would otherwise. Yes, I could see how some DMBS admins might
think that's actually a good thing. I am laughing now, btw.

This is really getting silly, a function like filesize could be put into the
standard and it would be useful. But as pointed out elsewhere in this thread
this discussion wont help to get the function into it.

Or, you could write one to do what you desire, in far less time than
you have spent so far on this thread, which does whatever it is you
think is appropriate for your platform.

--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw

[Randy Howard]

On Sun, 30 Dec 2007 17:50:04 -0600, Flash Gordon wrote
(in article <5e************@news.flash-gordon.me.uk>):

Ben Pfaff wrote, On 30/12/07 22:53:

>jacob navia <ja***@nospam.comwrites:

>>Why do you ignore what I am saying?

The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)

Or Jacob's message had not reached Keith, or it had but Keith has not
read it yet, or he just happened to respond to a message on a different
part of the thread.

Or recognize that NNTP is not real-time sync'd all over the globe, or
that everyone that subscribers to a usenet group reads the contents of
every single posting.

Jacob, you should not assume people are always out to get you.

Even the truly paranoid have real enemies.


--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw

[jacob navia]

Randy Howard wrote:

On Sun, 30 Dec 2007 17:50:58 -0600, Serve Lau wrote
(in article <68***************************@cache5.tilbu1.nb.ho me.nl>):

>"Richard Heathfield" <rj*@see.sig.invalidschreef in bericht
news:dO******************************@bt.com...

>>Serve Lau said:

<snip>

You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.
Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

Java already faced them you mean because such an operation is included in
*tadam* the standard library :)

Enlighten as to how this Java function handles real time changes to
file size between the time you make the call and the time you use the
result.

It doesn't. This is up to the programmer.

And when I do

fseek(f,0,SEEK_END);
fread(...);

If the file grows and I am not reading at the end this
is not the language problem but the programmer's problem.

Should we discard fseek/fread too?

--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[jacob navia]

Serve Lau wrote:

>
"Richard Heathfield" <rj*@see.sig.invalidschreef in bericht
news:dO******************************@bt.com...

>Serve Lau said:

<snip>

>>You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.

Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

Java already faced them you mean because such an operation is included
in *tadam* the standard library :)

Not only that. There is a utility class to format the result
in human readable form:

public static java.lang.String format(long filesize)

Format the size of a file in human readable form. Anything less
than a kilobyte is presented in kilobytes to one decimal place. Anything
between a kilobyte and a megabyte is presented in kilobytes with to zero
decimal places. Anything greater than one megabyte is presented in
megabytes to two decimal places.

eg.

* format(512) -0.5 kb
* format(1024) -1.0 kb
* format(2048) -2 kb
* format(1024 * 400) -400 kb
* format(1024 * 1024) -1024 kb
* format(1024 * 1024 * 1.2) -1.20 Mb
* format(1024 * 1024 * 20) -20.00 Mb

Parameters:
filesize - The size of the file in bytes.
Returns:
The size in human readable form.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[Keith Thompson]

jacob navia <ja***@nospam.comwrites:

Keith Thompson wrote:

[...]

>Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.

[...]

>
Why do you ignore what I am saying?

I posted an identical message in this thread proving I could not
possibly have a buffer overflow.

You say the same thing without quoting me.

You're right.

In response to some other messages in this thread, no, you're not in
my (nonexistent) killfile, and yes, I think I had read your response.
I was refuting what Gordon had said; I probably should have credited
you for saying much the same thing first. I don't always pay complete
attention to who has said what.

Perhaps you could have saved your "My point is proved" remark until
after I had a chance to respond.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
[...]
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"