Hello everybody,
Probably, this is being too demanding for Python, but it may be
useful to unimport modules to work with dynamic code (though not the
best, one example is [2]). In fact, it is supposed to be possible[1],
but I have detected it usually leaks memory.
When unimported in Linux, the simple C module attached (has no
functions, just the structure) leaks two memory pages
To see the test you can just put the attached files in a directory and:
python setupmod1.py install --install-lib .
python testMemory.py
Its output follows. First, the memory used before import. Second memory
used after the import. Third the number of references to that object is
checked before using del. Finally the memory used after unimporting.
-- Testing mod1 --
Mem. used: 1242 (gc: 0 )
Mem. used: 1244 (gc: 0 )
Check refs (should be = 2): 2
Mem. used: 1244 (gc: 0 )
-- Testing bigModule --
Mem. used: 1244 (gc: 0 )
Mem. used: 2686 (gc: 0 )
Check refs (should be = 2): 2
Mem. used: 1244 (gc: 0 )
-- Testing random --
Mem. used: 1244 (gc: 0 )
Mem. used: 1256 (gc: 0 )
Check refs (should be = 2): 2
Mem. used: 1256 (gc: 57 )
Unimporting attached "bigmodule.py" there are no memory leaks.
Unimporting python's random module it leaks some pages, but garbage
collector admit it can not free them.
If a module with the same name that the unimported module is imported,
the pages are reused. While, if this same module is loaded with another
name, they are not freed, and the program grows with each module
imported/unimported
Is this a Python bug? A Linux bug? Am i missing some way of freeing that
memory used by C module?
[1] http://mail.python.org/pipermail/pyt...ay/002669.html
[2] http://ax5.com/antonio/orangesnns
PD. If I attach the files the message does not reach to the list :-?
Code follows:
bigModule.py--------------------------------
def funcion_de_prueba(a,b):
print a,b
big_list = []
for i in xrange(1000):
big_list.append(1000 * 'biiiig')
--------------------------------------------
setupmod1.py--------------------------------
from distutils.core import setup, Extension
module1 = Extension('mod1',
sources = ['srcmod1.c'])
setup (name = 'OrangeSNNStmp',
version = '1.0',
description = 'Function that calls a trained NN',
ext_modules = [module1])
--------------------------------------------
srcmod1.c-----------------------------------
#include <Python.h>
static PyMethodDef orangeSnnsTmpMethods[] = {
{NULL, NULL, 0, NULL} /* Sentinel */
};
PyMODINIT_FUNC
initmod1(void)
{
(void) Py_InitModule("mod1", orangeSnnsTmpMethods);
}
--------------------------------------------
testMemory.py-------------------------------
#
# Test memory comsumption importing and unimporting modules
#
import gc, os, sys
def printMemoryUse():
rgc = gc.collect()
f = open("/proc/" + str(os.getpid()) + "/statm")
totalMemorySize = f.readline().split()[0]
f.close()
print "Mem. used:", totalMemorySize, "(gc:", rgc, ")"
def testImport(moduleName):
print "-- Testing", moduleName, "--"
printMemoryUse()
module = __import__(moduleName, globals(), locals())
printMemoryUse()
del sys.modules[moduleName]
print "Check refs (should be = 2):", sys.getrefcount(module)
del module
printMemoryUse()
testImport("mod1")
testImport("bigModule")
testImport("random")
--------------------------------------------
--
Regards,
Antonio Arauzo Azofra
