M2Crypto 0.18 - new version, same old build bugs.

[John Nagle]

Back in March, I posted this:

Hit that with OpenSSL. Red Hat took elliptical curve cryptography
out of Fedora 6 for patent reasons. With that missing, M2Crypto won't
build. It ought to; the implementor of M2Crypto thought of that, because
it's an optional feature. But M2Crypto uses SWIG. SWIG doesn't normally
process nested include files. OpenSSL had changed their configuration
approach to use nested include files. So SWIG didn't see the #define that
said to turn off elliptical curve crypto support. This resulted in
compile errors in the 24,000 lines of code that come out of SWIG.

That was for M2Crypto 0.17.

It's still broken in M2Crypto 0.18.

And there's no RPM or Windows binary.

Nobody actually uses this stuff, do they?

John Nagle

[Paul Boddie]

On 20 Sep, 22:12, John Nagle <[e-mail]:

>
That was for M2Crypto 0.17.

It's still broken in M2Crypto 0.18.

And there's no RPM or Windows binary.

Nobody actually uses this stuff, do they?

I can find Ubuntu packages for 0.16 and 0.17:

http://packages.ubuntu.com/feisty/libs/python-m2crypto
http://packages.ubuntu.com/gutsy/libs/python-m2crypto

So, clearly some people are managing to build the library, even if it
isn't the absolute latest version. You might want to take a look at
the source packages and the patches the distribution people apply,
although it doesn't look like they are actually doing much patching.
The source packages suggest that they use SWIG 1.3.24 to build
M2Crypto - is that what you're using?

Paul

[John Nagle]

Paul Boddie wrote:

On 20 Sep, 22:12, John Nagle <[e-mail]:

> That was for M2Crypto 0.17.

It's still broken in M2Crypto 0.18.

And there's no RPM or Windows binary.

Nobody actually uses this stuff, do they?

I can find Ubuntu packages for 0.16 and 0.17:

http://packages.ubuntu.com/feisty/libs/python-m2crypto
http://packages.ubuntu.com/gutsy/libs/python-m2crypto

So, clearly some people are managing to build the library, even if it
isn't the absolute latest version. You might want to take a look at
the source packages and the patches the distribution people apply,
although it doesn't look like they are actually doing much patching.
The source packages suggest that they use SWIG 1.3.24 to build
M2Crypto - is that what you're using?

Paul

I have builds of 0.17 of M2Crypto for both Linux and Windows. I want
the new bug fixes. The 0.17 version mishandles SSL certs that specify
more than one domain name, and that's supposedly fixed in 0.18. But
there are enough changes that just copying over the relevant file
(Checker.py) isn't sufficient.

John Nagle

[Heikki Toivonen]

John Nagle wrote:

Back in March, I posted this:

> Hit that with OpenSSL. Red Hat took elliptical curve cryptography
out of Fedora 6 for patent reasons. With that missing, M2Crypto won't
build. It ought to; the implementor of M2Crypto thought of that, because
it's an optional feature. But M2Crypto uses SWIG. SWIG doesn't normally
process nested include files. OpenSSL had changed their configuration
approach to use nested include files. So SWIG didn't see the #define
that
said to turn off elliptical curve crypto support. This resulted in
compile errors in the 24,000 lines of code that come out of SWIG.

That was for M2Crypto 0.17.

It's still broken in M2Crypto 0.18.

This was reported as
https://bugzilla.osafoundation.org/show_bug.cgi?id=9404 and fixed, at
least according to the person who reported the bug. I'd be curious to
know why the fix does not work for you...

As an alternative, does it work if you add line 130 to setup.py:

'-DOPENSSL_NO_EC', # Uncomment if you have
trouble compiling without EC support

--
Heikki Toivonen

[John Nagle]

Heikki Toivonen wrote:

John Nagle wrote:

>>Back in March, I posted this:

>> Hit that with OpenSSL. Red Hat took elliptical curve cryptography
out of Fedora 6 for patent reasons. With that missing, M2Crypto won't
build. It ought to; the implementor of M2Crypto thought of that, because
it's an optional feature. But M2Crypto uses SWIG. SWIG doesn't normally
process nested include files. OpenSSL had changed their configuration
approach to use nested include files. So SWIG didn't see the #define
that
said to turn off elliptical curve crypto support. This resulted in
compile errors in the 24,000 lines of code that come out of SWIG.

That was for M2Crypto 0.17.

It's still broken in M2Crypto 0.18.

This was reported as
https://bugzilla.osafoundation.org/show_bug.cgi?id=9404 and fixed, at
least according to the person who reported the bug. I'd be curious to
know why the fix does not work for you...

As an alternative, does it work if you add line 130 to setup.py:

'-DOPENSSL_NO_EC', # Uncomment if you have
trouble compiling without EC support

OK, here's the build failing:

python setup.py build
running build
running build_py
running build_ext
building 'M2Crypto.__m2crypto' extension
swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c
swig -python -I/usr/local/include/python2.5 -I/usr/include -o
SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i
/usr/include/openssl/opensslconf.h:27: Error: CPP #error ""This openssl-devel
package does not work your architecture?"". Use the -cpperraswarn option to
continue swig processing.
error: command 'swig' failed with exit status 1

OK, let's check the requirements, from
http://chandlerproject.org/bin/view/...ts/MeTooCrypto
M2Crypto 0.18:
* Python 2.3 or newer
o m2urllib2 requires Python 2.4 or newer
* OpenSSL 0.9.7 or newer
o Some optional new features will require OpenSSL 0.9.8 or newer
* SWIG 1.3.24 or newer

python --version
Python 2.5 [OK here, 2.5 >= 2.4]

swig -version
SWIG Version 1.3.31 [OK here; 1.3.31 >= 1.3.24]
Compiled with i386-redhat-linux-g++ [i386-redhat-linux-gnu]

openssl
OpenSSLversion
OpenSSL 0.9.8a 11 Oct 2005 [OK here; 0.9.8a =0.9.7.]

Tried the suggested patch to "setup.py":
diff setup.py.orig setup.py
130c130
< #'-D__i386__', # Uncomment for early OpenSSL
0.9.7 versions
---

'-D__i386__', # Uncomment for early OpenSSL

0.9.7 versions

No change in error message. That gets us

-bash-3.1$ python setup.py build
running build
running build_py
running build_ext
building 'M2Crypto.__m2crypto' extension
swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c
swig -python -I/usr/local/include/python2.5 -I/usr/include -o
SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i
/usr/include/openssl/opensslconf.h:27: Error: CPP #error ""This openssl-devel
package does not work your architecture?"". Use the -cpperraswarn option to
continue swig processing.
error: command 'swig' failed with exit status 1

But notice that the -D entry didn't appear on the SWIG command line.
Neither did the "-includeall". The "swig_opts" values around line
129 aren't actually being used. I think that's left over from the code intended
to allow builds with Python 2.3 and earlier. The "self.swig_opts" up at
line 53 of "setup.py" seems to be controlling. The patch was to
the obsolete code.

This is on Fedora Core 5, x86, 32 bit.

John Nagle

[Heikki Toivonen]

John Nagle wrote:

But notice that the -D entry didn't appear on the SWIG command line.
Neither did the "-includeall". The "swig_opts" values around line
129 aren't actually being used. I think that's left over from the code
intended
to allow builds with Python 2.3 and earlier. The "self.swig_opts" up at
line 53 of "setup.py" seems to be controlling. The patch was to
the obsolete code.

Aha! Good find. I reopened
https://bugzilla.osafoundation.org/show_bug.cgi?id=9404 and attached a
patch that should address this for real this time. At least -includeall
appears in my Ubuntu Dapper Drake environment. Could you give it a go
and let me know how it works?

--
Heikki Toivonen

[John Nagle]

Heikki Toivonen wrote:

John Nagle wrote:

>>But notice that the -D entry didn't appear on the SWIG command line.
Neither did the "-includeall". The "swig_opts" values around line
129 aren't actually being used. I think that's left over from the code
intended
to allow builds with Python 2.3 and earlier. The "self.swig_opts" up at
line 53 of "setup.py" seems to be controlling. The patch was to
the obsolete code.

Aha! Good find. I reopened
https://bugzilla.osafoundation.org/show_bug.cgi?id=9404 and attached a
patch that should address this for real this time. At least -includeall
appears in my Ubuntu Dapper Drake environment. Could you give it a go
and let me know how it works?

That's progress, but the build still doesn't work:

$ python setup.py build
running build
running build_py
running build_ext
building 'M2Crypto.__m2crypto' extension
swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c
swig -python -I/usr/local/include/python2.5 -I/usr/include -includeall
-D__i386__ -o SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i
/usr/include/openssl/opensslconf.h:13: Error: Unable to find 'opensslconf-i386.h'
error: command 'swig' failed with exit status 1

Now that SWIG is being invoked with "-includeall", it has to have
all the include paths defined. (Without "includeall", includes
are deferred until the C compile stage.) Note that the SWIG command line
specifies "/usr/include", but not "/usr/include/openssl".
Within M2Crypto's SWIG files, there's

_ec.i:%include <openssl/opensslconf.h>

which brings in "opensslconf.h" from /usr/include/openssl.
But that file has

#if defined(__i386__)
#include "opensslconf-i386.h"
#elif defined(__ia64__)
....

Since "opensslconf-i386.h" lives in /usr/include/openssl,
and that directory isn't mentioned on the SWIG command line,
the #include of "opensslconf-i386.h" fails.

And, no, adding

self.swig_opts.append('-DOPENSSL_NO_EC')
# Uncomment if you can't build with EC disabled

does not help.

As a test, I tried adding "-I/usr/include/openssl" to the SWIG command
line. Building then gets further, but during a C compile, we get

SWIG/_m2crypto_wrap.c:2529:18: error: _lib.h: No such file or directory

and the build goes downhill from there, with many compile errors in the
GCC phase. The gcc call

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
-fPIC -I/usr/local/include/python2.5 -c SWIG/_m2crypto_wrap.c -o
build/temp.linux-i686-2.5/SWIG/_m2crypto_wrap.o -DTHREADING

is in the directory below SWIG, and doesn't include SWIG, so it's clear
why gcc couldn't find the file.

So those fixes were not enough. Include file management in the M2Crypto build
clearly needs some work.

John Nagle

[John Nagle]

John Nagle wrote:

Heikki Toivonen wrote:

That's progress, but the build still doesn't work:

....

during a C compile, we get

SWIG/_m2crypto_wrap.c:2529:18: error: _lib.h: No such file or directory

and the build goes downhill from there, with many compile errors in the
GCC phase. The gcc call

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python2.5 -c SWIG/_m2crypto_wrap.c -o build/temp.linux-i686-2.5/SWIG/_m2crypto_wrap.o -DTHREADING

is in the directory below SWIG, and doesn't include SWIG, so it's clear
why gcc couldn't find the file.

Any sign of a fix yet?

John Nagle