Fatal Python error: deallocating None

[George Sakkis]

It's the first time in the three years I've been using python that a
program crashes without a nice traceback to give me a clue of what
might be wrong. After searching a little, it seems it's one of those
hard to get down to cases, so I've no idea where to look for. Briefly
what I do is, select some rows from one table, do some number crunching
and insert or update rows in another table. The C extensions I'm using
are numarray and MySQLdb (indirectly, through Sqlalchemy).

What makes the problem worse is that it's not deterministic; I can
restart it from (a little before) the point of crash and it doesn't
happen again at the same point, but it might happen further down. Now,
I wouldn't mind restarting it manually every time since the crashes are
not all that frequent; problem is, it's supposed to be a long-running
process that will probably take days to finish normally, so I want it
to run overnight too. Any hints ?

Thanks,
George

[Gabriel Genellina]

At Wednesday 25/10/2006 21:28, George Sakkis wrote:

>It's the first time in the three years I've been using python that a
program crashes without a nice traceback to give me a clue of what
might be wrong. After searching a little, it seems it's one of those
hard to get down to cases, so I've no idea where to look for. Briefly
what I do is, select some rows from one table, do some number crunching
and insert or update rows in another table. The C extensions I'm using
are numarray and MySQLdb (indirectly, through Sqlalchemy).

What makes the problem worse is that it's not deterministic; I can
restart it from (a little before) the point of crash and it doesn't
happen again at the same point, but it might happen further down. Now,
I wouldn't mind restarting it manually every time since the crashes are
not all that frequent; problem is, it's supposed to be a long-running
process that will probably take days to finish normally, so I want it
to run overnight too. Any hints ?

Mmm, it appears that one of these C extensions isn't managing the ref
count correctly - perhaps there is a return Py_None without a
previous Py_INCREF? If None were returned in certain functions to
indicate failure or something exceptional - and not a regular
condition - that would explain that it doesn't fail very often.
Unfortunately I don't know how to debug this - except by carefully
inspecting the C code :(
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

[Fredrik Lundh]

Gabriel Genellina wrote:

Mmm, it appears that one of these C extensions isn't managing the ref
count correctly - perhaps there is a return Py_None without a previous
Py_INCREF? If None were returned in certain functions to indicate
failure or something exceptional - and not a regular condition - that
would explain that it doesn't fail very often.
Unfortunately I don't know how to debug this - except by carefully
inspecting the C code :(

inserting

print sys.getrefcount(None)

at strategic locations in your code will usually help you identify the
culprit.

(last time I stumbled upon this was in an earlier release of "sqlite",
which leaked one Py_None every time you connected to the database. our
application contained a separate task that created it's own database
connection every 15:th minute or so, to calculate some statistics. and
a fresh Python interpreter contains 500-600 references to Py_None...)

</F>