Calling a function when the number of parameters isn't known till runtime

[John Friedland]

[apologies if anyone thinks this is off-topic. I originally
cross-posted to comp.lang.asm.x 86, but I'm now re-trying just c.l.c
after discovering that c.l.a.x86 is moderated]

My problem: I need to call (from C code) an arbitrary C library
function, but I don't know until runtime what the function name is,
how many parameters are required, and what the parameters are. I can
use dlopen/whatever to convert the function name into a pointer to
that function, but actually calling it, with the right number of
parameters, isn't easy.

As far as I can see, there are only two solutions:

1) This one is portable. If you know in advance that no function will
require more than, say, 2 parameters, then you can do something simple
like:

switch(nparams) {
case 0: (*user_func)(); break;
case 1: (*user_func)(P1 ); break;
case 2: (*user_func)(P1 , P2); break;
}

2) If you don't know the maximum number of parameters, then there
seems to be no way to do this, short of writing assembler code. I
guess this would look something like this:

- the C code pushes all the parameters onto a local stack
- the C code calls an assembler routine, passing in the function
address, the local stack address, and maybe the number of parameters
- the assembler routine sets up a stack frame and does an indirect
call to the user's C code
- the assembler routine clears up the stack frame and returns

Any thoughts? Have I missed anything/does this make sense? If I have
to go for (2), I'll start with Linux/gcc/x86 and move on to other
systems if necessary. Does anyone know of any web resources that could
help me do this? I've got no idea how to do this at the moment.

Thanks -

John

[Ancient_Hacker]

John Friedland wrote:

[apologies if anyone thinks this is off-topic. I originally
cross-posted to comp.lang.asm.x 86, but I'm now re-trying just c.l.c
after discovering that c.l.a.x86 is moderated]

My problem: I need to call (from C code) an arbitrary C library
function, but I don't know until runtime what the function name is,
how many parameters are required, and what the parameters are. I can
use dlopen/whatever to convert the function name into a pointer to
that function, but actually calling it, with the right number of
parameters, isn't easy.

As far as I can see, there are only two solutions:

1) This one is portable. If you know in advance that no function will
require more than, say, 2 parameters, then you can do something simple
like:

switch(nparams) {
case 0: (*user_func)(); break;
case 1: (*user_func)(P1 ); break;
case 2: (*user_func)(P1 , P2); break;
}

Not good enough, as the parameters might be of different sizes.

You're in luck if you're calling Windows Win32 API functions, as almost
all (All?) parameters are 32-bits. But in general they won't be
something like this might work:

#define pushbyte(b) __asm{ push byte ptr b }
#define pushword(b) __asm{ push word ptr b }

#define popoff(c) __asm{ add sp,c }

void CallFunc( FuncPtr TheFunctionAddr ess, char * ParamInfo; unsigned
long int Params[] )
{

for( i = 0; i < strlen( ParamInfo ); i++ ) { int Len; Len = 0;
switch( ParamInfo[i] ) {
case 'b': pushbyte( Params[i] ); Len+=1;; break;
case 'w': pushword( Params[i] ); Len +=2;break;
case 'l: pushlong( Params[i] ); Len +=4;break;
case 'a: pushaddr( Params[i] ); Len += 4; break;
case default: printf("bad param descriptor: !! %c", ParamInfo[i] );
}
*TheFunctionAdd ress();
popoff( Len );
}

You'll probably have to step thru this code several times to watch the
pushing and popping action until you get it all just right.

[Eric Sosman]

John Friedland wrote On 07/11/06 10:54,:

[apologies if anyone thinks this is off-topic. I originally
cross-posted to comp.lang.asm.x 86, but I'm now re-trying just c.l.c
after discovering that c.l.a.x86 is moderated]

My problem: I need to call (from C code) an arbitrary C library
function, but I don't know until runtime what the function name is,
how many parameters are required, and what the parameters are. I can
use dlopen/whatever to convert the function name into a pointer to
that function, but actually calling it, with the right number of
parameters, isn't easy.

A function pointer makes it easy to handle the unknown
name, but handling the unknown "signature" is another matter.
A function call in C is static in the sense that the code to
gather the arguments and retrieve the returned value is built
at compile time and cannot be changed at run time. Some
functions can accept variable numbers and even variable types
of arguments, but any particular call to a function has only
a fixed number of arguments of known types.

As far as I can see, there are only two solutions:

1) This one is portable. If you know in advance that no function will
require more than, say, 2 parameters, then you can do something simple
like:

switch(nparams) {
case 0: (*user_func)(); break;
case 1: (*user_func)(P1 ); break;
case 2: (*user_func)(P1 , P2); break;
}

This will work (after a small correction). Note, though,
that the argument types must be known; you cannot extend this
approach to "anonymous" arguments. You could use a fancier
dispatching method than simply the argument count, but you'll
still need to write different calls for different signatures.

The small correction is that you need to convert the
function pointer to match the actual type of the function
when you call it, and a function's type includes information
about its argument list. You'd need something like

void (*user_func)(vo id) = ...;
...
switch (nparams) {
case 0: (*user_func)(); break;
case 1: (*(void (*)(int))user_f unc)(P1); break;
case 2: (*(void (*)(int,int))us er_func)(P1,P2) ; break;
...

This is a case where a few typedefs are a distinct aid
to readability. Using them (and dropping the unnecessary
albeit harmless dereference operator), you'd have

typedef void (*Fptr0)(void);
typedef void (*Fptr1)(int);
typedef void (*Fptr2)(int, int);
Fptr0 user_func = ...;
...
switch (nparams) {
case 0: user_func(); break;
case 1: ((Fptr1)user_fu nc)(P1); break;
case 2: ((Fptr2)user_fu nc)(P1, P2); break;
...

2) If you don't know the maximum number of parameters, then there
seems to be no way to do this, short of writing assembler code. I
guess this would look something like this:

- the C code pushes all the parameters onto a local stack
- the C code calls an assembler routine, passing in the function
address, the local stack address, and maybe the number of parameters
- the assembler routine sets up a stack frame and does an indirect
call to the user's C code
- the assembler routine clears up the stack frame and returns

... for suitable values of "something like." The mechanisms
for passing arguments to functions are platform-specific. Some
platforms use a stack that's popped by the caller, others use a
stack popped by the callee. Some pass their arguments in CPU
registers instead of on a stack, to the extent possible. Some
use different strategies for different argument types: integers
and pointers in the A0,A1,... registers and floating-point
values in F0,F1,... Some use different strategies when calling
variadic functions than when calling functions with fixed-length
parameter lists. Some call struct-valued functions differently
than int-valued functions. In short, this approach requires an
intimate knowledge of the details of subroutine linkage on the
various platforms you want to support.

Any thoughts? Have I missed anything/does this make sense? If I have
to go for (2), I'll start with Linux/gcc/x86 and move on to other
systems if necessary. Does anyone know of any web resources that could
help me do this? I've got no idea how to do this at the moment.

Approach #1 is practical, if you can live with a predetermined
set of "signatures " and the set is not too large. Approach #2 is
flexible, but requires a lot of work that will need to be re-done
each time the code encounters a new machine (or sometimes, even
a new compiler). Personally, I'd stick with #1 and resort to #2
only in the very direst circumstances; it will turn out to be far
more expensive.

In fact, before resorting to #2 I'd take a step back and
ponder for a while. You're looking for a way to build a function
call at run-time, something that really can't be done in C (even
#1 is just choosing from among a bunch of pre-built calls). You
are presumably doing this not for the sheer enjoyment, but in
pursuit of a solution to a wider problem. Perhaps it's time to
re-examine the wider problem and see whether there's a more C-
friendly way to approach it, or even whether it ought to be
tackled with a different language altogether.

--
Er*********@sun .com

[John Friedland]

On 11 Jul 2006 08:33:23 -0700, "Ancient_Hacker " <gr**@comcast.n et>
wrote:

>switch(nparams ) {
case 0: (*user_func)(); break;
case 1: (*user_func)(P1 ); break;
case 2: (*user_func)(P1 , P2); break;
}

Not good enough, as the parameters might be of different sizes.

Forgot to say - I know that all Pn are the same integer type. I don't
know what 'user_func' does, but I can specify the integer type for Pn.
But, even so, I don't like this solution.

>something like this might work:

#define pushbyte(b) __asm{ push byte ptr b }
#define pushword(b) __asm{ push word ptr b }

#define popoff(c) __asm{ add sp,c }

void CallFunc( FuncPtr TheFunctionAddr ess, char * ParamInfo; unsigned
long int Params[] )
{

for( i = 0; i < strlen( ParamInfo ); i++ ) { int Len; Len = 0;
switch( ParamInfo[i] ) {
case 'b': pushbyte( Params[i] ); Len+=1;; break;
case 'w': pushword( Params[i] ); Len +=2;break;
case 'l: pushlong( Params[i] ); Len +=4;break;
case 'a: pushaddr( Params[i] ); Len += 4; break;
case default: printf("bad param descriptor: !! %c", ParamInfo[i] );
}
*TheFunctionAd dress();
popoff( Len );
}

Wow. Have you ever tried this? I'll give it a go with gcc -S and see
if it makes sense.

Thanks -

John

[John Friedland]

On Tue, 11 Jul 2006 11:51:35 -0400, Eric Sosman <Er*********@su n.com>
wrote:

The small correction is that you need to convert the
function pointer to match the actual type of the function
when you call it, and a function's type includes information
about its argument list. You'd need something like

void (*user_func)(vo id) = ...;
...
switch (nparams) {
case 0: (*user_func)(); break;
case 1: (*(void (*)(int))user_f unc)(P1); break;
case 2: (*(void (*)(int,int))us er_func)(P1,P2) ; break;
...

I was hoping that declaring user_func as 'void (*user_func)()' would
get around this, but I have to admit that I don't fully understand the
implications of an empty parameter list in a declaration.

>Personally, I'd stick with #1 and resort to #2
only in the very direst circumstances; it will turn out to be far
more expensive.

Yes, I agree that #2 is a nightmare, but my option is to tell users
that they can only write these functions with a maximum of N
parameters. What do I make N? Most of the time, it's probably going to
be 3 or 4 max, but then someone will come along and say that it
doesn't work with 500 or 1000 parameters, and I'll have to add more of
the calls and distribute new code. Easier or harder than writing a bit
of assembler for each new supported platform?

>Perhaps it's time to
re-examine the wider problem and see whether there's a more C-
friendly way to approach it, or even whether it ought to be
tackled with a different language altogether.

I'm surprised that it's not more common. I have what's essentially a
scripting language, which is implemented in C and C++. The user can
call 'external' functions (which must be written in C), so the user's
(script) names the library and the function, and provides the
parameters. I process the script at runtime and I have to call the
user's C library code. The lack of this sort of dynamic calling seems
to be a significant oversight in C. While googling I did find that
this was apparently considered while the language was being developed,
but it seems that it got too complicated and the overhead was too
much.

John

[Ancient_Hacker]

another way, guaranteed to work, if you can accept a slight delay each
time a new previously unseen function has to be called:

WriteOutGlueCod e( "/tmp/t%s.c");
system( "cc /tmp/t.c" );
h = dlopen( "/tmp/t.o" );

.... where WriteOutGlue is left as an exercise to the reader. And you
pass the parameters in and out thru a struct or (ugh) global
variables. And you insert the function name ni the file names of
course if you need more than one function at a time.

[Flash Gordon]

John Friedland wrote:

[apologies if anyone thinks this is off-topic. I originally
cross-posted to comp.lang.asm.x 86, but I'm now re-trying just c.l.c
after discovering that c.l.a.x86 is moderated]

Even moderated groups give responses eventually ;-)

However, cross posting between groups covering different languages is
rarely a good thing.

My problem: I need to call (from C code) an arbitrary C library
function, but I don't know until runtime what the function name is,
how many parameters are required, and what the parameters are.

If it could be literally any function, then standard C can't handle it.
With standard C you could do a look up in to an array of function
pointers, or a switch to call the appropriate function etc, but all
these methods require you to enumerate the possibilities in your C
source files.

I can
use dlopen/whatever to convert the function name into a pointer to
that function,

Well, if this is the *nix dlopen function, asking for advice on a *nix
group such as comp.unix.progr ammer might be a good idea.

but actually calling it, with the right number of
parameters, isn't easy.

In fact, standard C provides no mechanism for doing this.

As far as I can see, there are only two solutions:

1) This one is portable. If you know in advance that no function will
require more than, say, 2 parameters, then you can do something simple
like:

switch(nparams) {
case 0: (*user_func)(); break;
case 1: (*user_func)(P1 ); break;
case 2: (*user_func)(P1 , P2); break;
}

Yes, that is what you would have to do in standard C.

2) If you don't know the maximum number of parameters, then there
seems to be no way to do this, short of writing assembler code. I
guess this would look something like this:

- the C code pushes all the parameters onto a local stack
- the C code calls an assembler routine, passing in the function
address, the local stack address, and maybe the number of parameters
- the assembler routine sets up a stack frame and does an indirect
call to the user's C code
- the assembler routine clears up the stack frame and returns

Any thoughts? Have I missed anything/does this make sense? If I have
to go for (2), I'll start with Linux/gcc/x86 and move on to other
systems if necessary. Does anyone know of any web resources that could
help me do this? I've got no idea how to do this at the moment.

I think you will have to use techniques beyond what standard C has to
offer. Try comp.unix.progr ammer and the Linux and/or gcc groups to see
if they can provide any non-C-standard methods, if not either rethink
your problem or try the assembler approach. We can't help here since we
only deal with standard C and this is not possible in standard C (I'm
not complaining about the question being asked, since asking if
something can be done in standard C is not unreasonable for this group).
--
Flash Gordon, living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidelines and intro:
http://clc-wiki.net/wiki/Intro_to_clc

[John Friedland]

On Tue, 11 Jul 2006 17:29:02 +0100, John Friedland <jd*@nospam.org >
wrote:

>I'm surprised that it's not more common. I have what's essentially a
scripting language, which is implemented in C and C++. The user can
call 'external' functions (which must be written in C), so the user's
(script) names the library and the function, and provides the
parameters. I process the script at runtime and I have to call the
user's C library code.

After writing this (that's the great thing about writing down your
problems), it occurred to me that lots of people must have had exactly
the same problem. So, I looked up the Python/ctypes sources to see how
they do it. It turns out that they use Red Hat's libffi; from the
README:

>Some programs may not know at the time of compilation what arguments
are to be passed to a function. For instance, an interpreter may be
told at run-time about the number and types of arguments used to call
a given function. Libffi can be used in such programs to provide a
bridge from the interpreter program to compiled code.

libffi uses assembler code, and supports a number of different
platforms. It's currently part of gcc, and the best place to get
stand-alone sources may be via Python. Links, for anyone who comes
this way in the future:

http://sourceforge.net/project/showf...group_id=71702
http://sourceware.org/libffi/

John

[Eric Sosman]

John Friedland wrote On 07/11/06 12:29,:

On Tue, 11 Jul 2006 11:51:35 -0400, Eric Sosman <Er*********@su n.com>
wrote:

> The small correction is that you need to convert the
function pointer to match the actual type of the function
when you call it, and a function's type includes information
about its argument list. You'd need something like

void (*user_func)(vo id) = ...;
...
switch (nparams) {
case 0: (*user_func)(); break;
case 1: (*(void (*)(int))user_f unc)(P1); break;
case 2: (*(void (*)(int,int))us er_func)(P1,P2) ; break;
...

I was hoping that declaring user_func as 'void (*user_func)()' would
get around this, but I have to admit that I don't fully understand the
implications of an empty parameter list in a declaration.

The empty parameter list means "This function takes a
fixed number of arguments of fixed types, but we don't know
how many or of what kinds." That description (as far as it
goes) covers each of your usages individually, but I'm not
at all clear whether it covers them all collectively. A
true language lawyer might be able to settle the issue, but
instead of waiting for the pettifogging to settle down I'd
just opt for the squeaky-clean approach. It might not be
strictly necessary, but it's certainly prudent.

>>Personally, I'd stick with #1 and resort to #2
only in the very direst circumstances; it will turn out to be far
more expensive.

Yes, I agree that #2 is a nightmare, but my option is to tell users
that they can only write these functions with a maximum of N
parameters. What do I make N? Most of the time, it's probably going to
be 3 or 4 max, but then someone will come along and say that it
doesn't work with 500 or 1000 parameters, and I'll have to add more of
the calls and distribute new code. Easier or harder than writing a bit
of assembler for each new supported platform?

Hold your horses! Do I understand that you get to
prescribe the allowable function signatures? If so, your
problem simply vanishes: Tell the writers of the functions
that they'll receive one argument, namely, a pointer to
the start of an array containing the "real" arguments:

void (*user_func)(in t *) = ...;
...
int *args = malloc(nparams * sizeof *args);
if (args == NULL) ...
for (i = 0; i < nparams; ++i)
args[i] = ...;
user_func (args);
free (args);

If you like, you can "decorate" this pattern by passing
an argument count, a few `const' qualifiers, and/or some
sort of indications of different argument types. But as
long as you get to dictate the function signature, you might
as well make it easy on yourself ...

>>Perhaps it's time to
re-examine the wider problem and see whether there's a more C-
friendly way to approach it, or even whether it ought to be
tackled with a different language altogether.

I'm surprised that it's not more common. I have what's essentially a
scripting language, which is implemented in C and C++. The user can
call 'external' functions (which must be written in C), so the user's
(script) names the library and the function, and provides the
parameters. I process the script at runtime and I have to call the
user's C library code. The lack of this sort of dynamic calling seems
to be a significant oversight in C. While googling I did find that
this was apparently considered while the language was being developed,
but it seems that it got too complicated and the overhead was too
much.

No one language can be alle Dinge to tout le monde.

--
Er*********@sun .com

[Roberto Waltman]

John Friedland wrote:

>...I have what's essentially a
scripting language, which is implemented in C and C++. The user can
call 'external' functions (which must be written in C), so the user's
(script) names the library and the function, and provides the
parameters. I process the script at runtime and I have to call the
user's C library code. The lack of this sort of dynamic calling seems
to be a significant oversight in C.

Can you use a calling convention other than the trivial one shown so
far?

If so, you could call all functions with a single parameter: an array
of tagged structs, or a parameter count and a tagged struct array. (Or
an TLV array, or a TTLV array)

You could supply a few utility functions to help the called functions
decode and validate the params, to circumvent the "this is too
complicated" complaints that are sure to come.

Some quick ad-lib coding:
#define MAX_PARAMS ???

enum tags
{
IS_NOTHING,
IS_A_CHAR,
IS_A_INT,
...
IS_A_POINT3D,
...
IS_A_HEFTY_THIN G
...
};

union param_val
{
char char_val;
int int_val;
...
struct point3d point3d_val;
...
struct hefty_thing *hefty_ptr;
...
};

struct tagged_param
{
enum tags tag;
union param_val val;
};

struct tagged_param param_array[MAX_PARAMS];

void some_func()
{
param_array[0].tag = IS_A_INT;
param_array[0].val.int_val = 5;

param_array[1].tag = IS_A_POINT3D;
param_array[1].val... = ...

...

param_array[5].tag = ... ;
param_array[5].val... = ... ;

param_array[6].tag = IS_NOTHING; /* belt and suspenders */

user_function_w ith_5_params(pa ram_array);
}
Roberto Waltman

[ Please reply to the group,
return address is invalid ]