How to test python extension modules during 'make check' / 'make distcheck'?

[Mark Asbach]

Hi pythonians,

I'm one of the maintainers of an open source image processing toolkit
(OpenCV) and responsible for parts of the autotools setup. The package
mainly consists of four shared libraries but is accompanied by a python
package containing some pure python code and of course extension modules
for the four libraries.

Now during the last month we were preparing a major release which means
testing, fixing, testing, fixing, ... in the first degree. Typical
functionality of the shared libraries is verified during 'make check'
and 'make distcheck' by binaries that are linked against the libraries
(straight forward) and are listed in the 'TESTS' automake primary.

Unfortunately, many problems with the python wrappers arose from time to
time. Currently we have to build and install before we can run any
python-based test routines. When trying to integrate python module
testing into the automake setup, there are some problems that I couldn't
find a solution for:

a) the extension modules are built in different (other) subdirectories -
so they are not in the local path where python could find them

b) the libraries and extension modules are built with libtool and may
have rpaths compiled in (is this problematic)?

c) a different version of our wrappers might be installed on the testing
machine, somewhere in python/site-packages. How can I make sure that
python only finds my 'new' local generated modules?

Since the project is targeted at C/C++ developers in the first degree
and the python wrappers are just an add-on, there is no chance to
migrate it away from automake to someting else (like distutils, etc.).

Did someone solve a similar problem?

Thanks in advance,

Mark

P.S.: I'll post a similar message to the automake mailing list, but
probably python developers are a minority there ...

--
Dipl.-Ing. Mark Asbach Tel [phone]
Institute of Communications Engineering Fax [phone]
RWTH Aachen University, Germany http://www.ient.rwth-aachen.de

[Leo Kislov]

Mark Asbach wrote:

Hi pythonians,

I'm one of the maintainers of an open source image processing toolkit
(OpenCV) and responsible for parts of the autotools setup. The package
mainly consists of four shared libraries but is accompanied by a python
package containing some pure python code and of course extension modules
for the four libraries.

Now during the last month we were preparing a major release which means
testing, fixing, testing, fixing, ... in the first degree. Typical
functionality of the shared libraries is verified during 'make check'
and 'make distcheck' by binaries that are linked against the libraries
(straight forward) and are listed in the 'TESTS' automake primary.

Unfortunately, many problems with the python wrappers arose from time to
time. Currently we have to build and install before we can run any
python-based test routines. When trying to integrate python module
testing into the automake setup, there are some problems that I couldn't
find a solution for:

a) the extension modules are built in different (other) subdirectories -
so they are not in the local path where python could find them

As I understand it's not python that cannot find them but dynamic
linker. On ELF UNIX systems you can set LD_LIBRARY_PATH to help linker
find dependencies, on Windows -- PATH. If you need details, you can
find them in dynamic linker manuals.

b) the libraries and extension modules are built with libtool and may
have rpaths compiled in (is this problematic)?

libtools seems to have some knobs to cope with rpath:
http://sourceware.org/ml/bug-glibc/2.../msg00058.html

c) a different version of our wrappers might be installed on the testing
machine, somewhere in python/site-packages. How can I make sure that
python only finds my 'new' local generated modules?

Set PYTHONPATH to the directory where locally generated modules are
located. They will be found before site packages.

-- Leo