Static executable with shared modules

[Rickard Lind]

Is there any way to build the python executable statically and
still be able to load modules built as shared libraries?

I'm trying to run python scripts on a stripped down FreeBSD (4.9)
machine which has no shared system libraries so I want to link it
statically against libc et al, but it would be nice to still be
able to load modules which were built as shared libraries. In
particular I have a library for which I've generated a wrapper
with swig which I'd like to import.

I've googled up and down but can't find anyone who has tried this
particular combination. Just adding a -static to the Makefile
seems to remove the ability to load shared libraries altogether
as I get a "ImportError: Service unavailable" exception.

/r

[Martin v. Löwis]

Rickard Lind wrote:

Is there any way to build the python executable statically and
still be able to load modules built as shared libraries?
I'm not what "build statically" means; if you talking about
building a statically linked interpreter binary - then no,
this is not possible. At a minimum, you need to link with -ldl,
or else you cannot perform dlopen(3).
I'm trying to run python scripts on a stripped down FreeBSD (4.9)
machine which has no shared system libraries so I want to link it
statically against libc et al, but it would be nice to still be
able to load modules which were built as shared libraries.
Does that system support shared libraries? What is the API for
loading shared libraries, and finding a symbol in a dynamically-loaded
shared library, on that system?
In
particular I have a library for which I've generated a wrapper
with swig which I'd like to import.

If shared libraries are not supported, you could link the swig
module statically as well.

Regards,
Martin

[Rickard Lind]

Martin v. Löwis wrote:

I'm not what "build statically" means; if you talking about
building a statically linked interpreter binary - then no,
this is not possible. At a minimum, you need to link with -ldl,
or else you cannot perform dlopen(3).

I'll be more specific: when I build python 2.3.4 on FreeBSD 4.9,
the interpreter binary is linked against three shared libraries:
libutil.so.3, libm.so.2 and libc_r.so.4. Now, these libraries are
not present on the TARGET system (which is distinct from the build
system, but based on the same version of FreeBSD) so I add "-static"
to LDFLAGS. This produces an interpreter that runs on the target
system (no dependency on shared libc etc) but it also cannot load
modules compiled as shared libraries. Man page for dlopen(3) says
it is located in libc (and consquently there seems to be no libdl),
and anyway I'd expect to get a link error if the dl* functions were
not present. What I DO get is an ImportError exception.

At present I see no other option than to link the modules into the
interpreter which is very inconvenient since I'll have to rebuild
the every time a module changes :-(

/r

[Martin v. Löwis]

Rickard Lind wrote:

I'll be more specific: when I build python 2.3.4 on FreeBSD 4.9,
the interpreter binary is linked against three shared libraries:
libutil.so.3, libm.so.2 and libc_r.so.4. Now, these libraries are
not present on the TARGET system (which is distinct from the build
system, but based on the same version of FreeBSD) so I add "-static"
to LDFLAGS. This produces an interpreter that runs on the target
system (no dependency on shared libc etc) but it also cannot load
modules compiled as shared libraries. Man page for dlopen(3) says
it is located in libc (and consquently there seems to be no libdl),
and anyway I'd expect to get a link error if the dl* functions were
not present. What I DO get is an ImportError exception.
Most likely, the static libc.a does not provide dlopen (contrary
to what the man page says). Python's configure detects that you
don't have dlopen (when linking with -static), and concludes that
dynamic loading of modules is not supported on your system.
Then, during import, it checks built-in modules, module.py, module.pyc,
and finds neither - it then reports an import error.

To confirm this theory, have a look at pyconfig.h, and check
whether HAVE_DLOPEN and/or HAVE_DYNAMIC_LOADING are defined;
I expect that neither is defined.
At present I see no other option than to link the modules into the
interpreter which is very inconvenient since I'll have to rebuild
the every time a module changes :-(

The other option, clearly, is to move the shared libraries to the
target system along with the interpreter binary (assuming the target
system supports shared libraries).

If the target system does not have a dlopen, and the static libc.a
does not have dlopen, either, there is nothing you can do except
to use a different operating system.

Regards,
Martin