More M2Crypto issues

[John Nagle]

I've been running M2Crypto successfully using Python 2.4 on Windows 2000,
and now I'm trying to get it to work on Python 2.3.4 on Linux.

Attempting to initialize a context results in

Traceback (most recent call last):
File "/www/htdocs/sitetruth.com/cgi/ratingdetails.cgi", line 46, in ?
DetailsPageBuilder.detailspage(kdbfile,ktemplatefi le,url) # check and
display domain or URL as web page
File "./sitetruth/DetailsPageBuilder.py", line 70, in detailspage
sitecert = InfoSSL2.Certificate(siteinfo, kverifylocations, verbose)
File "./sitetruth/InfoSSL2.py", line 147, in __init__
self.ctx = createsslcontext(trustedcafile, verbose) # Generate general SSL
context
File "./sitetruth/InfoSSL2.py", line 40, in createsslcontext
ctx = SSL.Context('sslv3') # Create context with SSL params
File "/home/sitetruth/lib/python/M2Crypto/SSL/Context.py", line 43, in __init__
map()[long(self.ctx)] = self
ValueError: invalid literal for long(): _480e1008_p_SSL_CTX

which, when I look at the code and try some test cases, seems
legitimate. The cacheing code is trying to convert a reference to an
object (a C object, in fact) into a "long". Python 2.4 on Windows
will do that. Python 2.3.4 on Linux converts it to a string first,
gets "_480e1008_p_SSL_CTX", and then tries to convert that to an
integer, which fails.

M2Crypto is supposed to work with Python 2.3, so this should work.

John Nagle

[John Nagle]

Upgraded to Python 2.5 on the Linux system, rebuild M2Crypto,
and it stil fails, but the last line of the error message changed to:

ValueError: invalid literal for long() with base 10: '_40f91a08_p_SSL_CTX'

Others have encountered this problem, from a Google search.

Is "long" supposed to operate on C objects?

John Nagle

John Nagle wrote:

I've been running M2Crypto successfully using Python 2.4 on Windows 2000,
and now I'm trying to get it to work on Python 2.3.4 on Linux.

Attempting to initialize a context results in

Traceback (most recent call last):
File "/www/htdocs/sitetruth.com/cgi/ratingdetails.cgi", line 46, in ?
DetailsPageBuilder.detailspage(kdbfile,ktemplatefi le,url) # check
and display domain or URL as web page
File "./sitetruth/DetailsPageBuilder.py", line 70, in detailspage
sitecert = InfoSSL2.Certificate(siteinfo, kverifylocations, verbose)
File "./sitetruth/InfoSSL2.py", line 147, in __init__
self.ctx = createsslcontext(trustedcafile, verbose) # Generate
general SSL context
File "./sitetruth/InfoSSL2.py", line 40, in createsslcontext
ctx = SSL.Context('sslv3') # Create
context with SSL params
File "/home/sitetruth/lib/python/M2Crypto/SSL/Context.py", line 43, in
__init__
map()[long(self.ctx)] = self
ValueError: invalid literal for long(): _480e1008_p_SSL_CTX

which, when I look at the code and try some test cases, seems
legitimate. The cacheing code is trying to convert a reference to an
object (a C object, in fact) into a "long". Python 2.4 on Windows
will do that. Python 2.3.4 on Linux converts it to a string first,
gets "_480e1008_p_SSL_CTX", and then tries to convert that to an
integer, which fails.

M2Crypto is supposed to work with Python 2.3, so this should work.

John Nagle

[Gabriel Genellina]

At Thursday 18/1/2007 04:41, John Nagle wrote:

I've been running M2Crypto successfully using Python 2.4 on Windows 2000,
and now I'm trying to get it to work on Python 2.3.4 on Linux.

Attempting to initialize a context results in

Traceback (most recent call last):
File "/www/htdocs/sitetruth.com/cgi/ratingdetails.cgi", line 46, in ?
DetailsPageBuilder.detailspage(kdbfile,ktemplatefi le,url) # check and
display domain or URL as web page
File "./sitetruth/DetailsPageBuilder.py", line 70, in detailspage
sitecert = InfoSSL2.Certificate(siteinfo, kverifylocations, verbose)
File "./sitetruth/InfoSSL2.py", line 147, in __init__
self.ctx = createsslcontext(trustedcafile, verbose) #
Generate general SSL
context
File "./sitetruth/InfoSSL2.py", line 40, in createsslcontext
ctx =
SSL.Context('sslv3')
# Create context with SSL params
File "/home/sitetruth/lib/python/M2Crypto/SSL/Context.py", line
43, in __init__
map()[long(self.ctx)] = self
ValueError: invalid literal for long(): _480e1008_p_SSL_CTX

On a previous version of M2Crypto that line said: map()[self.ctx] =
self, and that failed too ("unhashable object", I think).
I changed the class _ctxmap (the map() above returns an instance of
it) to use str(key) in the 3 places it was used. (That would be
equivalent to use str(self.ctx) everywhere, instead of long(self.ctx)
as your traceback is showing). All I can say is that no error
happened afterwards, but I don't know if that broke something. (and I
didn't care at the moment, I only needed a Zope instance running on
Windows with SSL for less than a week - and if it were "cheating SSL"
it was OK for me then)

which, when I look at the code and try some test cases, seems
legitimate. The cacheing code is trying to convert a reference to an
object (a C object, in fact) into a "long". Python 2.4 on Windows
will do that. Python 2.3.4 on Linux converts it to a string first,
gets "_480e1008_p_SSL_CTX", and then tries to convert that to an
integer, which fails.

So using str() appears, at least on the surface, to be reasonable.
But someone with more intimate knowledge of the library should
confirm that. I don't even understand what's the point for the
_ctxmap singleton - it's the same thing as a global dictionary, isn't it?
--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

[John Nagle]

Gabriel Genellina wrote:

At Thursday 18/1/2007 04:41, John Nagle wrote:

On a previous version of M2Crypto that line said: map()[self.ctx] =
self, and that failed too ("unhashable object", I think).
I changed the class _ctxmap (the map() above returns an instance of it)
to use str(key) in the 3 places it was used. (That would be equivalent
to use str(self.ctx) everywhere, instead of long(self.ctx) as your
traceback is showing). All I can say is that no error happened
afterwards, but I don't know if that broke something.
So using str() appears, at least on the surface, to be reasonable. But
someone with more intimate knowledge of the library should confirm that.
I don't even understand what's the point for the _ctxmap singleton -
it's the same thing as a global dictionary, isn't it?

I played around with using "str" too, but I was worried about
Python and SWIG version issues. I'm not sure you can use "str"
on those objects on all versions and platforms. I think that
SWIG is generating the implementations of __str__ and __long__.

Incidentally, note in that area that if you never explicitly
call "close" on a Context, it will never be released. Or so
it looks from the code.

Actually, at the moment I'm having an M2Crypto problem related
to a SWIG/OpenSSL conflict. Older versions of OpenSSL have an
include file that needs __i386__ defined, which is something GCC
does based on what platform you're on. SWIG uses CPP, but
doesn't set the platform defines, so the SWIG phase of the
M2Crypto build fails. I'm currently trying to get the shared
host where that build took place upgraded to a later version of
OpenSSL, but that requires a server restart, so it may take
a few days. I'm doing something that requires M2Crypto to
run on a range of machines, which turns out to be rather harder
than expected.

All this stuff is in the area that Guido was unhappy about
in his "M2Crypto Woes" article.

http://www.artima.com/weblogs/viewpost.jsp?thread=95863

The worst problems there have been fixed, but we're not out
of the woods yet. As Guido wrote, using SWIG does complicate
things.

I'm currently getting good results on Windows 2000 with
Python 2.4 using M2Crypto 0.17; right now, the problems are
on the Linux side.

John Nagle

[Heikki Toivonen]

John Nagle wrote:

I've been running M2Crypto successfully using Python 2.4 on Windows 2000,
and now I'm trying to get it to work on Python 2.3.4 on Linux.

Attempting to initialize a context results in

Traceback (most recent call last):

[...]

map()[long(self.ctx)] = self
ValueError: invalid literal for long(): _480e1008_p_SSL_CTX

This is almost certainly because of SWIG that is too old. The minimum
required is SWIG 1.3.24. If you cannot upgrade SWIG, the alternative
would be to play around with these values to fit your version of SWIG.

I'm adding this to the FAQ.

--
Heikki Toivonen

[Heikki Toivonen]

John Nagle wrote:

Actually, at the moment I'm having an M2Crypto problem related
to a SWIG/OpenSSL conflict. Older versions of OpenSSL have an
include file that needs __i386__ defined, which is something GCC
does based on what platform you're on. SWIG uses CPP, but
doesn't set the platform defines, so the SWIG phase of the
M2Crypto build fails. I'm currently trying to get the shared
host where that build took place upgraded to a later version of
OpenSSL, but that requires a server restart, so it may take
a few days. I'm doing something that requires M2Crypto to
run on a range of machines, which turns out to be rather harder
than expected.

Which version of OpenSSL is that?

--
Heikki Toivonen

[John Nagle]

Heikki Toivonen wrote:

John Nagle wrote:

> Actually, at the moment I'm having an M2Crypto problem related
to a SWIG/OpenSSL conflict. Older versions of OpenSSL have an
include file that needs __i386__ defined, which is something GCC
does based on what platform you're on. SWIG uses CPP, but
doesn't set the platform defines, so the SWIG phase of the
M2Crypto build fails. I'm currently trying to get the shared
host where that build took place upgraded to a later version of
OpenSSL, but that requires a server restart, so it may take
a few days. I'm doing something that requires M2Crypto to
run on a range of machines, which turns out to be rather harder
than expected.

Which version of OpenSSL is that?

M2Crypto 0.17 requirements:
Python 2.3 or newer
o m2urllib2 requires Python 2.4 or newer
OpenSSL 0.9.7 or newer
o Some optional new features will require OpenSSL 0.9.8 or newer
SWIG 1.3.24 or newer

Server in use:
Python version: "Python 2.5 (r25:51908, Jan 18 2007, 10:46:37)"
OpenSSL version: "OpenSSL 0.9.7a Feb 19 2003"
SWIG version: "SWIG Version 1.3.31"
GCC version: "[GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2"
Computer: Athlon 686 rackmount server.

The actual build failure is:

running build_ext
building 'M2Crypto.__m2crypto' extension
swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c
swig -python -I/usr/include -o SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i
/usr/include/openssl/opensslconf.h:27: Error: CPP #error ""This openssl-devel
package does not work your architecture?"". Use the -cpperraswarn option to
continue swig processing.
error: command 'swig' failed with exit status 1

The problem is that "opensslconf.h" expects the compiler to define __i386__
when running the C preprocessor, and SWIG doesn't do that. Compare
"opensslconf.h" in different versions of OpenSSL; it's changed considerably.

John Nagle

[Heikki Toivonen]

John Nagle wrote:

OpenSSL version: "OpenSSL 0.9.7a Feb 19 2003"

Hmm, I've never actually used that old OpenSSL myself, just assumed from
the original author's notes that anything from 0.9.7 onward worked.
Guess not. I am thinking of changing the requirements to state which one
works... I think the oldest that I have personally tested may have been
0.9.7d.

--
Heikki Toivonen

[John Nagle]

Heikki Toivonen wrote:

John Nagle wrote:

> OpenSSL version: "OpenSSL 0.9.7a Feb 19 2003"

Hmm, I've never actually used that old OpenSSL myself, just assumed from
the original author's notes that anything from 0.9.7 onward worked.
Guess not. I am thinking of changing the requirements to state which one
works... I think the oldest that I have personally tested may have been
0.9.7d.

I think older versions were built with an older OpenSSL and an older
SWIG. The SWIG preprocessor has some limitations which can cause
errors to be reported during the SWIG preprocessor phase.

This problem in SWIG has hit other projects. See

http://blog.gmane.org/gmane.comp.pro...month=20051201

where the Common LISP people had a similar problem.

John Nagle