opaque style question

[Capstar]

Hi NG,

I've read some time ago in this NG, that when you're writing a library
it can be good practice to use an opaque type to point to your
structures if the user doesn't need to modify the structure himself.
This way the internal structure of the lib can be changed, and the lib
can be recompiled, but the application using the lib doesn't need to be
recompiled.

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use. I also read somewhere that it's bad
practice to make a typedef to a pointer in such a way that it is not
obvious from looking at the type, in this case lib_t, that it is a
pointer to whatever. So that would mean it would be best to use the
first typedef.
But if I use the first way, In the application I need to declare
lib_t *my_opaque_variable;
If I don't do it that way, I will get a compiler error because it
doesn't know the size of lib_t.
So that would mean the second typedef would be better, because the
compiler will always know the sizeof of a pointer.

So what is the usual way to do this?

Thanks in advance,
Mark

--
<<Remove the del for email>>

[Dan Pop]

In <c5**********@news.tudelft.nl> Capstar <sp***@deleg.homeip.net> writes:

I've read some time ago in this NG, that when you're writing a library
it can be good practice to use an opaque type to point to your
structures if the user doesn't need to modify the structure himself.
This way the internal structure of the lib can be changed, and the lib
can be recompiled, but the application using the lib doesn't need to be
recompiled.

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use. I also read somewhere that it's bad
practice to make a typedef to a pointer in such a way that it is not
obvious from looking at the type, in this case lib_t, that it is a
pointer to whatever. So that would mean it would be best to use the
first typedef.
But if I use the first way, In the application I need to declare
lib_t *my_opaque_variable;
If I don't do it that way, I will get a compiler error because it
doesn't know the size of lib_t.
So that would mean the second typedef would be better, because the
compiler will always know the sizeof of a pointer.

So what is the usual way to do this?

The standard C library uses the first approach. FILE stands for the
real thing, and you have to explicitly declare pointers to it.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de

[Irrwahn Grausewitz]

Capstar <sp***@deleg.homeip.net> wrote:
<snip>

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use. I also read somewhere that it's bad
practice to make a typedef to a pointer in such a way that it is not
obvious from looking at the type, in this case lib_t, that it is a
pointer to whatever. So that would mean it would be best to use the
first typedef.
Right.
But if I use the first way, In the application I need to declare
lib_t *my_opaque_variable;
Of course. Any problems with that?
If I don't do it that way, I will get a compiler error because it
doesn't know the size of lib_t.
Yes, for the obvious reason that it cannot tell the size of an
incomplete type until it's actually completed.
So that would mean the second typedef would be better, because the
compiler will always know the sizeof of a pointer.
Wrong conclusion. The compiler _always_ knows the size of a
pointer-to-whatever. What's *not* known to the compiler is the
size of an incomplete type, but that's not is changed at all by
typedef'ing a pointer to that type. Remember: typedef does
*NOT* introduce a new type, only an alias name for an arbitrary
type that already exists.
So what is the usual way to do this?

Stick with the first typedef (if any typedef at all) and use the
pointer notation. Just as you should always use FILE *, and not
any obscure typename alias for it.

HTH
Regards
--
Irrwahn Grausewitz (ir*******@freenet.de)
welcome to clc: http://www.ungerhu.com/jxh/clc.welcome.txt
clc faq-list : http://www.faqs.org/faqs/C-faq/faq/
clc OT guide : http://benpfaff.org/writings/clc/off-topic.html

[CBFalconer]

Capstar wrote:

I've read some time ago in this NG, that when you're writing a
library it can be good practice to use an opaque type to point
to your structures if the user doesn't need to modify the
structure himself. This way the internal structure of the lib
can be changed, and the lib can be recompiled, but the
application using the lib doesn't need to be recompiled.

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use. I also read somewhere that it's
bad practice to make a typedef to a pointer in such a way that
it is not obvious from looking at the type, in this case lib_t,
that it is a pointer to whatever. So that would mean it would
be best to use the first typedef.
But if I use the first way, In the application I need to declare
lib_t *my_opaque_variable;
If I don't do it that way, I will get a compiler error because
it doesn't know the size of lib_t.
So that would mean the second typedef would be better, because
the compiler will always know the sizeof of a pointer.

So what is the usual way to do this?

The objective is to hide the structure. That means you can never
declare a struct libstruct in the user code, because the size
isn't known. However you can declare a pointer type, because that
size is known. Now you can't malloc any memory for that pointer,
because the size isn't known. That means the library has to have
a routine to do the mallocing and return the pointer. All the
user code can do is save the pointer and return it to various
routines in the library, which know what to do with it. If the
mallocing for the pointer involves mallocing subfields, the
disposal has also to be a library routine.

--
fix (vb.): 1. to paper over, obscure, hide from public view; 2.
to work around, in a way that produces unintended consequences
that are worse than the original problem. Usage: "Windows ME
fixes many of the shortcomings of Windows 98 SE". - Hutchison

[Jarno A Wuolijoki]

On Thu, 15 Apr 2004, Capstar wrote:

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use.

The difference between them is that the second form allows you to
switch from pointers to handles without changing the interface. The
downside is that the programmer needs to be aware that it's either
pointer or an integer type. (*)
Unless you're into paranoid defensive programming I don't think
that's too useful so go with the first.

*)
int libtcmp(lib_t *a, lib_t *b) { return a==b; }
vs
int libtcmp(lib_t a, lib_t b) { return a==b; }

Can you tell whether the latter piece is correct or not without
knowing what type lib_t is?

[E. Robert Tisdale]

Capstar wrote:

I've read some time ago in this NG that,
when you're writing a library, it can be good practice
to use an opaque type to point to your structures
if the user doesn't need to modify the structure himself.
This way, the internal structure of the lib can be changed
and the lib can be recompiled
but the application using the lib doesn't need to be recompiled.

But then there are two ways to define the opaque type:

#typedef struct libstruct lib_t;
Yes! Always.
or

#typedef struct libstruct* lib_t;
No! Never.
I'm not sure which one to use. I also read somewhere that
it's bad practice to make a typedef to a pointer
in such a way that it is not obvious from looking at the type
(lib_t in this case) that it is a pointer to whatever.
So that would mean it would be best to use the first typedef.
But if I use the first way,
in the application, I need to declare lib_t* my_opaque_variable;
If I don't do it that way, I will get a compiler error
because it doesn't know the size of lib_t.
So that would mean the second typedef would be better
because the compiler will always know the sizeof of a pointer.

So what is the usual way to do this?

There is no way
to deny a determined hacker direct access to data members.
The only thing you can do is to help prevent application programmers
from *accidently* referencing data members directly.
This can be accomplished by substituting a *public* type definition
with the same *size* and *alignment* as the *private* definition
which does not reveal the private data member names:
#ifdef NCL_REVEAL
/* Reveal type definitions after applications are thoroughly tested. */

typedef struct { /* subvector class definition */
/* private: */
ncl_dhandle H;
ncl_offset O;
ncl_extent N;
ncl_stride S;
} ncl_dsubvector;
/* A subvector does not own the array storage that it references. */
/* It does not allocate any array storage when it is constructed */
/* nor does it deallocate any array storage when it is destroyed. */

#else /* NCL_REVEAL */
/* Conceal type definitions until applications are thoroughly tested. */

#define NCL_DSUBVECTOR_SIZE NCL_SUBVECTOR_SIZE
typedef int ncl_dhidden_subvector[NCL_DSUBVECTOR_SIZE/sizeof(int)];

typedef struct { /* subvector class definition */
/* private: */
ncl_dhidden_subvector V;
} ncl_dsubvector;
/* A subvector does not own the array storage that it references. */
/* It does not allocate any array storage when it is constructed */
/* nor does it deallocate any array storage when it is destroyed. */

#endif /* NCL_REVEAL */
The problem with passing pointers to opaque objects is that
all library functions which must access the private data members
must be compiled externally and linked into the application program.
Light-weight operations cannot be implemented
as inline functions or C preprocessor macros without including
the private type definition in the application program.

Take a look at "The ANSI C Numerical Class Library"

http://www.netwood.net/~edwin/svmtl/

if this is of any interest to you.

[CBFalconer]

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

The objective is to hide the structure. That means you can never
declare a struct libstruct in the user code, because the size
isn't known. However you can declare a pointer type, because that
size is known. Now you can't malloc any memory for that pointer,
because the size isn't known. That means the library has to have
a routine to do the mallocing and return the pointer. All the
user code can do is save the pointer and return it to various
routines in the library, which know what to do with it. If the
mallocing for the pointer involves mallocing subfields, the
disposal has also to be a library routine.

You have accurately described FILE and the way it is handled by the
<stdio.h> part of the standard C library. Yet, for some reason, the
C standard requires the size of FILE to be available to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type, as long as
there is *nothing* useful user code can do with an object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[Keith Thompson]

Capstar <sp***@deleg.homeip.net> writes:
[...]

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

[...]

Quibble: typedef is not a preprocessor directive; there's no '#'.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
Schroedinger does Shakespeare: "To be *and* not to be"

[CBFalconer]

Jarno A Wuolijoki wrote:

On Thu, 15 Apr 2004, Capstar wrote:

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

I'm not sure which one to use.

The difference between them is that the second form allows you to
switch from pointers to handles without changing the interface. The
downside is that the programmer needs to be aware that it's either
pointer or an integer type. (*)
Unless you're into paranoid defensive programming I don't think
that's too useful so go with the first.

*)
int libtcmp(lib_t *a, lib_t *b) { return a==b; }
vs
int libtcmp(lib_t a, lib_t b) { return a==b; }

Can you tell whether the latter piece is correct or not without
knowing what type lib_t is?

You shouldn't be programming this operation unless you are
programming the library. That is the point of keeping the type to
yourself. So you don't need to know.

--
fix (vb.): 1. to paper over, obscure, hide from public view; 2.
to work around, in a way that produces unintended consequences
that are worse than the original problem. Usage: "Windows ME
fixes many of the shortcomings of Windows 98 SE". - Hutchison

[Capstar]

Keith Thompson wrote:

Capstar <sp***@deleg.homeip.net> writes:
[...]

But then there are two ways to define the opque type:
#typedef struct libstruct lib_t;
or
#typedef struct libstruct * lib_t;

[...]

Quibble: typedef is not a preprocessor directive; there's no '#'.

d'oh! what was I thinking...

--
<<Remove the del for email>>

[Capstar]

Thanks for all your quick replies.
I think I will stick to
typedef struct libstruct lib_t;

and declare
lib_t *my_opaque_variable;

I know I won't be able to use inline functions or macros for lightweight
functions, but I do have the ability to modify the structure without the
need to recompile to application.

Mark

--
<<Remove the del for email>>

[Peter Nilsson]

"CBFalconer" <cb********@yahoo.com> wrote in message
news:40***************@yahoo.com...

Dan Pop wrote:

... for some reason, the C standard requires the size of FILE to be
available to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type,
as long as there is *nothing* useful user code can do with an
object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

What would preclude an implementation of putc as a (non-trivial) macro if
FILE were an incomplete type?

Consider...

struct __FILE {
int foo;
...
};

typedef struct __file FILE;

#define __F(f) ((struct __FILE *) (f))

#define putc(c,f) (__F(f)->foo ? ...)

This would probably be too tedious for implementors to bother with, and it
seems to me that about the only thing the language would gain from an
incomplete FILE type is that some implementations could naturally diagnose
code like...

FILE x = *stdin;

--
Peter

[Jarno A Wuolijoki]

On Fri, 16 Apr 2004, CBFalconer wrote:

int libtcmp(lib_t *a, lib_t *b) { return a==b; }
vs
int libtcmp(lib_t a, lib_t b) { return a==b; }

Can you tell whether the latter piece is correct or not without
knowing what type lib_t is?

You shouldn't be programming this operation unless you are
programming the library. That is the point of keeping the type to
yourself. So you don't need to know.

Oh well.. I think requiring bloating the interface with extra
macro/function just to know whether two variables refer to the very
same entity underlines the point even better;)

[CBFalconer]

Peter Nilsson wrote:

"CBFalconer" <cb********@yahoo.com> wrote in message

Dan Pop wrote:

... for some reason, the C standard requires the size of FILE
to be available to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type,
as long as there is *nothing* useful user code can do with an
object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

What would preclude an implementation of putc as a (non-trivial)
macro if FILE were an incomplete type?

Consider...

struct __FILE {
int foo;
...
};

That is neither an opaque type nor an incomplete type.

--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[Dan Pop]

In <40***************@yahoo.com> CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

The objective is to hide the structure. That means you can never
declare a struct libstruct in the user code, because the size
isn't known. However you can declare a pointer type, because that
size is known. Now you can't malloc any memory for that pointer,
because the size isn't known. That means the library has to have
a routine to do the mallocing and return the pointer. All the
user code can do is save the pointer and return it to various
routines in the library, which know what to do with it. If the
mallocing for the pointer involves mallocing subfields, the
disposal has also to be a library routine.

You have accurately described FILE and the way it is handled by the
<stdio.h> part of the standard C library. Yet, for some reason, the
C standard requires the size of FILE to be available to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type, as long as
there is *nothing* useful user code can do with an object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

Engage your brain, Chuck. These things are written by the implementor
and there is no way to hide the actual defintion of FILE from him ;-)

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de

[CBFalconer]

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

The objective is to hide the structure. That means you can never
declare a struct libstruct in the user code, because the size
isn't known. However you can declare a pointer type, because that
size is known. Now you can't malloc any memory for that pointer,
because the size isn't known. That means the library has to have
a routine to do the mallocing and return the pointer. All the
user code can do is save the pointer and return it to various
routines in the library, which know what to do with it. If the
mallocing for the pointer involves mallocing subfields, the
disposal has also to be a library routine.

You have accurately described FILE and the way it is handled by
the <stdio.h> part of the standard C library. Yet, for some
reason, the C standard requires the size of FILE to be available
to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type, as
long as there is *nothing* useful user code can do with an
object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

Engage your brain, Chuck. These things are written by the
implementor and there is no way to hide the actual defintion of
FILE from him ;-)

Time to engage yours. :-) The macro is expanded by the user in
his own little world. How does that expansion get at the various
fields of the FILE structure if that structure is hidden? Are you
recommending the implementor evaluate offsetof(...) for every
field in FILE and insert corresponding absolute values into his
macros? Along with the appropriate casts. Sounds like a
maintenance nightmare to me.

To avoid all this the implementor publishes the actual FILE
structure in stdio.h and makes no fuss about it. He uses those
fields (which are in his namespace) in putc and getc macros, and
again makes no fuss about it. Joe Q. User uses them and never
sees them. Until he gets some knowledge and goes looking. Then
he finds he can use those funny names with preceding '_'s to get
at interesting things and make trouble.

Now look at something that has a truly opaque type, such as my
hashlib package. hashlib.h has the statement:

/* opaque incomplete object */
typedef struct hshtag hshtbl;

and "hshtbl *" is the type returned by hshinit(). Every function
call into the library has an initial parameter of type "hshtbl
*". The user never sees the completed definition, and has no C
language means of accessing any field of a hshtbl. If I were
doing it again I would move the * into the typedef and obscure the
fact that it is a pointer. As long as I don't alter the .h file I
can do anything I want in the implementation (.c) portion.

However I can never publish any macros that deal with a hshtbl.
Luckily all operations on the table are sufficiently complex that
the overhead of a function call will never be noticed.

--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[Michael Wojcik]

In article <c5***********@sunnews.cern.ch>, Da*****@cern.ch (Dan Pop) writes:

In <40***************@yahoo.com> CBFalconer <cb********@yahoo.com> writes:

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

Engage your brain, Chuck. These things are written by the implementor
and there is no way to hide the actual defintion of FILE from him ;-)

But if FILE is an incomplete type in a given translation unit, then
putc in that t.u. can't be a function-like macro that expands into
code which refers to members of the FILE type. So if the implementor
wants to provide a macro version of putc, it must be accompanied by a
complete FILE type.

Still, it's odd that the Standard would *require* that FILE be an
object type, rather than simply allowing it. If it were legal for a
user program to copy FILE objects (say, for the purpose of saving and
restoring state), then it might make sense; but C90 7.9.3 states that
"a copy of a FILE object may not necessarily serve in place of the
original".

So while I agree with Chuck that there are reasons to allow FILE to
be an object type, I also agree with Dan that an implementation
ought to be allowed to make it an incomplete type, since there
doesn't appear to be any valid mandated use for FILE as an object
type in the standard.

But I may be missing something...

--
Michael Wojcik mi************@microfocus.com

This book uses the modern technology to explain the phenomemon in the world
of Japanese animation. If you love anime so much, you'd better read this.
After you read it, you may agree that is destroying the dream of the child.
Needs Chinese viewing system. -- The Goodboy Scientific

[Joe Wright]

Dan Pop wrote:

[ snip ]

I've never understood why FILE can't be an incomplete type, as long as
there is *nothing* useful user code can do with an object of type FILE.

I agree there is no member of the FILE structure that is interesting
to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

...might not work.

--
Joe Wright mailto:jo********@comcast.net
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[CBFalconer]

Joe Wright wrote:

Dan Pop wrote:

[ snip ]

I've never understood why FILE can't be an incomplete type, as
long as there is *nothing* useful user code can do with an
object of type FILE.

I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
although they won't do you much good.

--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!

[Irrwahn Grausewitz]

Joe Wright <jo********@comcast.net> wrote:
<snip>

I agree there is no member of the FILE structure that is interesting
to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

How so?

Regards
--
Irrwahn Grausewitz (ir*******@freenet.de)
welcome to clc: http://www.ungerhu.com/jxh/clc.welcome.txt
clc faq-list : http://www.faqs.org/faqs/C-faq/faq/
clc OT guide : http://benpfaff.org/writings/clc/off-topic.html

[Joe Wright]

CBFalconer wrote:

Joe Wright wrote:

Dan Pop wrote:

[ snip ]

I've never understood why FILE can't be an incomplete type, as
long as there is *nothing* useful user code can do with an
object of type FILE.

I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
although they won't do you much good.

As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

--
Joe Wright mailto:jo********@comcast.net
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[Joe Wright]

Irrwahn Grausewitz wrote:

Joe Wright <jo********@comcast.net> wrote:
<snip>

I agree there is no member of the FILE structure that is interesting
to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

How so?

As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

--
Joe Wright mailto:jo********@comcast.net
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[Irrwahn Grausewitz]

Joe Wright <jo********@comcast.net> wrote:

Irrwahn Grausewitz wrote:

Joe Wright <jo********@comcast.net> wrote:

[...] if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

How so?

As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

The respective values of stdin and stdout. Nothing prevents
an implementation to declare stdin et al. as extern FILE *.

Regards
--
Irrwahn Grausewitz (ir*******@freenet.de)
welcome to clc: http://www.ungerhu.com/jxh/clc.welcome.txt
clc faq-list : http://www.faqs.org/faqs/C-faq/faq/
clc OT guide : http://benpfaff.org/writings/clc/off-topic.html

[CBFalconer]

Joe Wright wrote:

CBFalconer wrote:

Joe Wright wrote:

Dan Pop wrote:

[ snip ]

I've never understood why FILE can't be an incomplete type, as
long as there is *nothing* useful user code can do with an
object of type FILE.

I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
although they won't do you much good.

As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

That's the point. FILE is _not_ incomplete on any system known to
me, in order to allow putc and getc macros to function. Thus you
can write that FILE in = *stdin with impunity on those systems,
and get no errors. If you then try to pass &in to various
functions, peculiar things are going to happen.

I guess a fairly harmless test of this would be:

#include <stdio.h>

/* proving FILE is not incomplete on this system */
int main(void)
{
printf("sizeof(FILE) = %d\n", (int)sizeof(FILE));
return 0;
}

--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!

[Arthur J. O'Dwyer]

On Sat, 17 Apr 2004, CBFalconer wrote:

Joe Wright wrote:

CBFalconer wrote:

Joe Wright wrote:
Dan Pop wrote:
>
> I've never understood why FILE can't be an incomplete type, as
> long as there is *nothing* useful user code can do with an
> object of type FILE.
[Agreed.]
I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.
[Of course it would work.]
and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
[All systems, because FILE is defined to be an object type.]
although they won't do you much good.
As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

That's the point. FILE is _not_ incomplete on any system known to
me, in order to allow putc and getc macros to function. Thus you
can write that FILE in = *stdin with impunity on those systems,
and get no errors.

I think most participants in this thread aren't understanding each
other. FILE is not incomplete on any system because *the Standard
says it isn't*! This has nothing to do with the working or not of
putc and getc macros. It's perfectly possible to write macros that
work with incomplete types, for example:

#define FIRST_ELEM(a) (a[0])

where 'a' can be declared with type 'int[]'; or with pointers to
incomplete types, for example:

#define MY_GETC(fp) (fgetc(fp))

where 'fp' is defined with object type 'MY_FILE *' and 'MY_FILE'
is incomplete.
If you then try to pass &in to various
functions, peculiar things are going to happen.
Probably. That's not surprising at all; the value of &in, while
it has the right type (FILE *), was not derived from a call to
'fopen', nor is it equal to 'stdin', 'stdout', 'stderr', or any
other pre-defined 'FILE *' object. So it's basically a garbage
value. Garbage in, garbage out.
I guess a fairly harmless test of this would be:

#include <stdio.h>

/* proving FILE is not incomplete on this system */
int main(void)
{
printf("sizeof(FILE) = %d\n", (int)sizeof(FILE));
return 0;
}

You really could have removed the first and last four lines of that
program, and you still would have had a proof that 'FILE' is not an
incomplete type. The Standard *says* it's not.

-Arthur

[Barry Schwarz]

On Sat, 17 Apr 2004 19:26:05 GMT, CBFalconer <cb********@yahoo.com>
wrote:

Joe Wright wrote:

CBFalconer wrote:

Joe Wright wrote:
Dan Pop wrote:

[ snip ]

> I've never understood why FILE can't be an incomplete type, as
> long as there is *nothing* useful user code can do with an
> object of type FILE.

I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
although they won't do you much good.
As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

That's the point. FILE is _not_ incomplete on any system known to
me, in order to allow putc and getc macros to function. Thus you
can write that FILE in = *stdin with impunity on those systems,

Is there any reason that FILE could not be the typedef for an array.
In such a case, the assignment would be illegal.
and get no errors. If you then try to pass &in to various
functions, peculiar things are going to happen.

I guess a fairly harmless test of this would be:

#include <stdio.h>

/* proving FILE is not incomplete on this system */
int main(void)
{
printf("sizeof(FILE) = %d\n", (int)sizeof(FILE));
return 0;
}

<<Remove the del for email>>

[Joe Wright]

CBFalconer wrote:

Joe Wright wrote:

CBFalconer wrote:

Joe Wright wrote:

Dan Pop wrote:

[ snip ]
>I've never understood why FILE can't be an incomplete type, as
>long as there is *nothing* useful user code can do with an
>object of type FILE.

I agree there is no member of the FILE structure that is
interesting to the C programmer. But if FILE were incomplete..

FILE *in = stdin, *out = stdout;

..might not work.

and why not? The thing that won't work is:

FILE in = *stdin;

but you can write such statements with impunity in most systems,
although they won't do you much good.

As both stdin and stdout have type 'pointer to FILE' and FILE is
incomplete, what values would be assigned to in and out?

That's the point. FILE is _not_ incomplete on any system known to
me, in order to allow putc and getc macros to function. Thus you
can write that FILE in = *stdin with impunity on those systems,
and get no errors. If you then try to pass &in to various
functions, peculiar things are going to happen.

I guess a fairly harmless test of this would be:

#include <stdio.h>

/* proving FILE is not incomplete on this system */
int main(void)
{
printf("sizeof(FILE) = %d\n", (int)sizeof(FILE));
return 0;
}

We may be going sideways here. I was responding to Dan's conjecture
on whether FILE could/should be incomplete so as to be opaque. While
you can declare a pointer to incomlete struct type, it would be
problematic to assign meaningful values to such pointers.

I'm perfectly happy that FILE is not incomplete. The only time I
might even look at it is on Sunday afternoon of a really dismal day
on which I am terminally bored. And then I can't really make sense
of it. It might as well be opaque.

--
Joe Wright mailto:jo********@comcast.net
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[CBFalconer]

"Arthur J. O'Dwyer" wrote:

.... snip ...
You really could have removed the first and last four lines of
that program, and you still would have had a proof that 'FILE'
is not an incomplete type. The Standard *says* it's not.

After all the fooferaw and the above I actually went and read the
appropriate bits of N869. You are right. All we can say is that
nothing within FILE is defined, or that everything within FILE is
implementation defined.

This proves it is a terrible example of opaque objects. :-) But
be of good cheer, I gave a better example from my hashlib package
earlier. Having built that, I claim to know what it is.

--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!

[Arthur J. O'Dwyer]

On Sat, 17 Apr 2004, Joe Wright wrote:

We may be going sideways here. I was responding to Dan's conjecture
on whether FILE could/should be incomplete so as to be opaque. While
you can declare a pointer to incomlete struct type, it would be
problematic to assign meaningful values to such pointers.

No, it wouldn't. Look, what do you do with C's objects of type
'FILE *' at the moment? You can:

- Assign predefined values to them: fp = stdin;
- Assign new values to them: fp = fopen(...);
- Pass them to functions: fclose(fp);
- Pass their addresses to functions: my_open(&fp);
- Compare them for equality: if (fp == stdin) ...
- Compare them to predefined values: if (fp == NULL) ...

What other meaningful operations on file descriptors can you think
of?* Note that all of the above operations could be done whether
FILE were an opaque type or not. As it happens, FILE is *not* an
opaque type, but I agree with Dan Pop here --- there's no practical
reason for the Standard to say it mustn't be. It looks like just
one of C's "quirks."

-Arthur
[*] - I know the phrase "file descriptor" usually means something
else on Unixoid systems, but if I said "file-describer," I'd get
responses correcting my English. ;-) Standard C, as far as I know,
doesn't use the term "file descriptor" to mean either an integer
*or* a [pointer to a] FILE object.

[Keith Thompson]

CBFalconer <cb********@yahoo.com> writes:
[...]

After all the fooferaw and the above I actually went and read the
appropriate bits of N869. You are right. All we can say is that
nothing within FILE is defined, or that everything within FILE is
implementation defined.

It's defined by the implementation, but it's not
"implementation-defined" (i.e., the implementation isn't required to
document it).

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
Schroedinger does Shakespeare: "To be *and* not to be"

[Dan Pop]

In <40***************@yahoo.com> CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:
CBFalconer <cb********@yahoo.com> writes:

> The objective is to hide the structure. That means you can never
> declare a struct libstruct in the user code, because the size
> isn't known. However you can declare a pointer type, because that
> size is known. Now you can't malloc any memory for that pointer,
> because the size isn't known. That means the library has to have
> a routine to do the mallocing and return the pointer. All the
> user code can do is save the pointer and return it to various
> routines in the library, which know what to do with it. If the
> mallocing for the pointer involves mallocing subfields, the
> disposal has also to be a library routine.

You have accurately described FILE and the way it is handled by
the <stdio.h> part of the standard C library. Yet, for some
reason, the C standard requires the size of FILE to be available
to user code:

2 The types declared are size_t (described in 7.17);

FILE

which is an object type...
^^^^^^^^^^^^^^

I've never understood why FILE can't be an incomplete type, as
long as there is *nothing* useful user code can do with an
object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.
Engage your brain, Chuck. These things are written by the
implementor and there is no way to hide the actual defintion of
FILE from him ;-)

Time to engage yours. :-)

It was already engaged.
The macro is expanded by the user in
his own little world. How does that expansion get at the various
fields of the FILE structure if that structure is hidden? Are you
recommending the implementor evaluate offsetof(...) for every
field in FILE and insert corresponding absolute values into his
macros? Along with the appropriate casts. Sounds like a
maintenance nightmare to me.

A simple cast is enough for the job. A conversion between pointer to
incomplete type and pointer to the real thing. Something along the
lines:

struct __FILE { /* definition of the real thing */ };
typedef struct __FOO FILE; /* struct __FOO never defined */

Now, FILE is an incomplete type, but a pointer to FILE can be converted
to a pointer to __FILE and vice versa.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de

[Dan Pop]

In <Cc********************@comcast.com> Joe Wright <jo********@comcast.net> writes:

We may be going sideways here. I was responding to Dan's conjecture
on whether FILE could/should be incomplete so as to be opaque. While
you can declare a pointer to incomlete struct type, it would be
problematic to assign meaningful values to such pointers.

Not at all. You take a pointer to a complete struct type (the real thing)
and convert it to a pointer to an incomplete struct type. You do the
opposite conversion when actually trying to use the value stored in the
pointer to incomplete struct type.

But it needn't be an incomplete struct in the first place. Consider

typedef void FILE;

The *only* reason a conforming implementation cannot do that is because
the standard wants an object type for FILE. For diagnosing purposes,
such a definition for FILE would not be optimal (you could pass any object
or incomplete pointer type where a FILE pointer is expected), but
the entire <stdio.h> specification could be implemented without any
problems with FILE defined like this.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de

[CBFalconer]

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:

CBFalconer <cb********@yahoo.com> writes:
Dan Pop wrote:
> CBFalconer <cb********@yahoo.com> writes:
>
>> The objective is to hide the structure. That means you can never
>> declare a struct libstruct in the user code, because the size
>> isn't known. However you can declare a pointer type, because that
>> size is known. Now you can't malloc any memory for that pointer,
>> because the size isn't known. That means the library has to have
>> a routine to do the mallocing and return the pointer. All the
>> user code can do is save the pointer and return it to various
>> routines in the library, which know what to do with it. If the
>> mallocing for the pointer involves mallocing subfields, the
>> disposal has also to be a library routine.
>
> You have accurately described FILE and the way it is handled by
> the <stdio.h> part of the standard C library. Yet, for some
> reason, the C standard requires the size of FILE to be available
> to user code:
>
> 2 The types declared are size_t (described in 7.17);
>
> FILE
>
> which is an object type...
> ^^^^^^^^^^^^^^
>
> I've never understood why FILE can't be an incomplete type, as
> long as there is *nothing* useful user code can do with an
> object of type FILE.

In order to allow putc and getc to be implemented as macros, and
put/get things from a buffer embedded in the FILE structure, and
such like.

Engage your brain, Chuck. These things are written by the
implementor and there is no way to hide the actual defintion of
FILE from him ;-)

Time to engage yours. :-)

It was already engaged.

The macro is expanded by the user in
his own little world. How does that expansion get at the various
fields of the FILE structure if that structure is hidden? Are you
recommending the implementor evaluate offsetof(...) for every
field in FILE and insert corresponding absolute values into his
macros? Along with the appropriate casts. Sounds like a
maintenance nightmare to me.

A simple cast is enough for the job. A conversion between pointer
to incomplete type and pointer to the real thing. Something along
the lines:

struct __FILE { /* definition of the real thing */ };
typedef struct __FOO FILE; /* struct __FOO never defined */

Now, FILE is an incomplete type, but a pointer to FILE can be
converted to a pointer to __FILE and vice versa.

How does that prevent the user from writing something of the form:

fp->fieldname = whatever;

where fp is a FILE* secured from fopen? Maybe that thing is,
strictly speaking, an incomplete type (although I have my doubts),
but it is not opaque, which is the subject of this thread. The
purpose of all these gyrations is to protect various entities from
prying.

--
"I'm a war president. I make decisions here in the Oval Office
in foreign policy matters with war on my mind." - Bush.
"Churchill and Bush can both be considered wartime leaders, just
as Secretariat and Mr Ed were both horses." - James Rhodes.

[Arthur J. O'Dwyer]

On Mon, 19 Apr 2004, CBFalconer wrote:

Dan Pop wrote:

>> Dan Pop wrote:
>>> I've never understood why FILE can't be an incomplete type, as
>>> long as there is *nothing* useful user code can do with an
>>> object of type FILE.
A simple cast is enough for the job. A conversion between pointer
to incomplete type and pointer to the real thing. Something along
the lines:

struct __FILE { /* definition of the real thing */ };
typedef struct __FOO FILE; /* struct __FOO never defined */

Now, FILE is an incomplete type, but a pointer to FILE can be
converted to a pointer to __FILE and vice versa.

How does that prevent the user from writing something of the form:

fp->fieldname = whatever;

where fp is a FILE* secured from fopen?

Let's see. FILE is a typedef for struct __FOO, and struct __FOO
is never defined in the user's translation unit. Thus, it has no
'fieldname' available; thus, constraint violation or syntax error
or something. Invalid program, anyway.
The user *could* follow the lead of 'getc'/'putc' and write

((struct __FILE *)fp)->fieldname = whatever;

but now not only is he *obviously* *intentionally* breaking the
rules, he's introducing a big ugly cast in his code, which is a
tipoff that something dubious is happening.
Maybe that thing is,
strictly speaking, an incomplete type (although I have my doubts),
but it is not opaque, which is the subject of this thread. The
purpose of all these gyrations is to protect various entities from
prying.

If "various entities" can read the <stdio.h> header and relevant
implementation specs, and apply offsetof and pointer arithmetic, they
can get around any opaqueness at all. What exactly are you trying
to prevent?

(Example: I can "get around" anything at all by applying an expression
of the form

*(char *)my_address = my_num;

I don't think you'd consider this a big hole in opaqueness. So what
*do* you consider a problem?)

-Arthur

[E. Robert Tisdale]

CBFalconer wrote:

How does that prevent the user from writing something of the form:

fp->fieldname = whatever;

where fp is a FILE* secured from fopen?
Maybe that thing is, strictly speaking, an incomplete type
(although I have my doubts) but it is not opaque,
which is the subject of this thread.
The purpose of all these gyrations
is to protect various entities from prying.

No! You are confused.

The purpose of an opaque type
is to help prevent the application programmers from
*accidental* direct references.
It is *not* encryption.
A dedicated hacker can usually determine the private definition
of an opaque data type by doing a few simple tests.

[CBFalconer]

"Arthur J. O'Dwyer" wrote:

.... snip ...
If "various entities" can read the <stdio.h> header and relevant
implementation specs, and apply offsetof and pointer arithmetic,
they can get around any opaqueness at all. What exactly are you
trying to prevent?

(Example: I can "get around" anything at all by applying an
expression of the form

*(char *)my_address = my_num;

I don't think you'd consider this a big hole in opaqueness. So
what *do* you consider a problem?)

I don't consider FILE to be opaque in the first place. Obscure,
yes. Things are properly opaque when the user can't get at
anything by reading the header; i.e. he requires the source code
in addition. Then the source can be freely modified without
affecting the header, and the maximum effect on the user code is a
need for relinking.

--
"I'm a war president. I make decisions here in the Oval Office
in foreign policy matters with war on my mind." - Bush.
"Churchill and Bush can both be considered wartime leaders, just
as Secretariat and Mr Ed were both horses." - James Rhodes.

[Dan Pop]

In <40***************@yahoo.com> CBFalconer <cb********@yahoo.com> writes:

"Arthur J. O'Dwyer" wrote:

... snip ...

If "various entities" can read the <stdio.h> header and relevant
implementation specs, and apply offsetof and pointer arithmetic,
they can get around any opaqueness at all. What exactly are you
trying to prevent?

(Example: I can "get around" anything at all by applying an
expression of the form

*(char *)my_address = my_num;

I don't think you'd consider this a big hole in opaqueness. So
what *do* you consider a problem?)

I don't consider FILE to be opaque in the first place. Obscure,
yes. Things are properly opaque when the user can't get at
anything by reading the header;

Who said the user can read the header? Chapter and verse, please.

Anyway, *your* definition of type opacity is at odds with the common one:
a type whose definition is not documented (C header files don't count as
documentation). FILE is the canonical C example of opaque type. As usual
C helps people to avoid shooting themselves in the foot but doesn't even
try to prevent them from doing that.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de

[CBFalconer]

Dan Pop wrote:

.... snip ...
Who said the user can read the header? Chapter and verse, please.
They do. They also do other things that are not considered
kosher. How many compilers do you know that do not use physical
header files?

Anyway, *your* definition of type opacity is at odds with the
common one: a type whose definition is not documented (C header
files don't count as documentation). FILE is the canonical C
example of opaque type. As usual C helps people to avoid
shooting themselves in the foot but doesn't even try to prevent
them from doing that.

Since we disagree on the definition, there is no point in arguing
about the details. I can create things that satisfy my
definition.

--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!

[Dan Pop]

In <40***************@yahoo.com> CBFalconer <cb********@yahoo.com> writes:

Dan Pop wrote:

... snip ...

Who said the user can read the header? Chapter and verse, please.

They do. They also do other things that are not considered
kosher. How many compilers do you know that do not use physical
header files?

They need not be in human readable format (ever heard of precompiled
headers?). Furthermore, the definition of __FILE can be built into
the compiler, rather than provided in <stdio.h>. It's up to the
implementor. __FILE is in the implementation name space, therefore
its definition can be always in scope, regardless of what headers
are included.

Anyway, *your* definition of type opacity is at odds with the
common one: a type whose definition is not documented (C header
files don't count as documentation). FILE is the canonical C
example of opaque type. As usual C helps people to avoid
shooting themselves in the foot but doesn't even try to prevent
them from doing that.

Since we disagree on the definition, there is no point in arguing
about the details. I can create things that satisfy my
definition.

Only if you destroy the source code after compiling it ;-)

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de