howto load and unload a module

[Guy Robinson]

Hello,

I have a directory of python scripts that all (should) contain a number of
attributes and methods of the same name.

I need to import each module, test for these items and unload the module. I have
2 questions.

1.. How do unload an imported module?
2.. how do I test for the existance of a method in a module without running it?

TIA,

Guy

[Peter Hansen]

Guy Robinson wrote:

I have a directory of python scripts that all (should) contain a number
of attributes and methods of the same name.

I need to import each module, test for these items and unload the
module. I have 2 questions.

1.. How do unload an imported module?
Why would you want to? Doing what you describe doesn't require that you
"unload" a module, unless that means something more to you than, say,
merely releasing the memory used by it (which is likely insignificant to
you).
2.. how do I test for the existance of a method in a module without
running it?

The object bound to the name used in the import statement is, well, an
object, so you can use the usual tests:

import mymodule
try:
mymodule.myfunction
except AttributeError:
print 'myfunction does not exist'

or use getattr(), or some of the introspection features available in the
"inspect" module.

-Peter

[Fuzzyman]

Answer to 2 - ``hasattr(module, name)``

[Guy Robinson]

Why would you want to? Doing what you describe doesn't require that you
"unload" a module, unless that means something more to you than, say,
merely releasing the memory used by it (which is likely insignificant to
you).

Hi Peter,

I have an application with Python embedded. I'm parsing a script directory to
build a dictionary of script names with descriptions of what the scripts etc
extracted from each script. The user then selects one of these scripts to
execute by the embedded python.

Some of these scripts could potentially be quite large. Also the list of scripts
could be quite large. So the main reason for unloading modules is to save memory.

Regards,

Guy

[Peter Hansen]

Guy Robinson wrote:

Some of these scripts could potentially be quite large. Also the list of
scripts could be quite large. So the main reason for unloading modules
is to save memory.

Unless you're talking megabytes of bytecode (do a "wc *.pyc" on the
compiled files and see) it's probably not worth the bother.

Still, assuming the modules are pure Python, and don't do strange things
like inject references to themselves or their data into other places
(i.e. other modules, including sys or builtins), it should be possible
to unload them simply by deleting all references to them, *including*
manually removing them from sys.modules.

How do you plan to import them? Using the import statement, or
__import__, or some other means? How you do it will determine exactly
what steps are required to free them up.

Note also that this won't necessarily release any memory back to the
operating system, and it won't necessarily unload any extension modules
or other shared libraries that are loaded. The whole concept of
"unloading" a module is pretty much undefined in Python, so whatever you
can get is the best you can expect...

-Peter

[John Machin]

Peter Hansen wrote:

Guy Robinson wrote:

I have a directory of python scripts that all (should) contain a
number of attributes and methods of the same name.

I need to import each module, test for these items and unload the
module. I have 2 questions. [snip] 2.. how do I test for the existance of a method in a module without
running it?

What the OP is calling a 'method' is more usually called a 'function'
when it is defined at module level rather than class level.


The object bound to the name used in the import statement is, well, an
object, so you can use the usual tests:

import mymodule
try:
mymodule.myfunction
except AttributeError:
print 'myfunction does not exist'

or use getattr(), or some of the introspection features available in the
"inspect" module.

Ummm ... doesn't appear to scale well for multiple modules and multiple
attributes & functions. Try something like this (mostly tested):

modules = ['foomod', 'barmod', 'brentstr', 'zotmod']
attrs = ['att1', 'att2', 'att3', 'MyString']
funcs = ['fun1', 'fun2', 'fun3']
# the above could even be read from file(s)
for modname in modules:
try:
mod = __import__(modname)
except ImportError:
print "module", modname, "not found"
continue
for attrname in attrs:
try:
attr = getattr(mod, attrname)
except AttributeError:
print "module %s has no attribute named %s" % \
(modname, attrname)
continue
# check that attr is NOT a function (maybe)
for funcname in funcs:
pass
# similar to above but check that it IS a function
BTW, question for the OP: what on earth is the use-case for this? Bulk
checking of scripts written by students?

Cheers,
John

[Peter Hansen]

John Machin wrote:

Peter Hansen wrote: [sample code] Ummm ... doesn't appear to scale well for multiple modules and multiple
attributes & functions.
It certainly wouldn't! :-) I was posting mainly to elicit more
information, since clearly you wouldn't get far hardcoding all the names
you were interested in. (It's hard to judge a poster's level of
expertise in Python without any example code from him. That makes it
too likely to go way above the head of the poster, and possibly provide
a much more complex solution than he really needs.)
Try something like this (mostly tested):

modules = ['foomod', 'barmod', 'brentstr', 'zotmod']
attrs = ['att1', 'att2', 'att3', 'MyString']
funcs = ['fun1', 'fun2', 'fun3']
# the above could even be read from file(s)
for modname in modules:
try:
mod = __import__(modname)
except ImportError:
print "module", modname, "not found"
continue
for attrname in attrs:
try:
attr = getattr(mod, attrname)
except AttributeError:
print "module %s has no attribute named %s" % \
(modname, attrname)
continue
# check that attr is NOT a function (maybe)
for funcname in funcs:
pass
# similar to above but check that it IS a function

Of course, one could simply hand the man a complete answer... ;-)

-Peter

[Guy Robinson]

BTW, question for the OP: what on earth is the use-case for this? Bulk
checking of scripts written by students?

Cheers,
John
I've embedded python in an application which has a .NET API. So users can write
scripts in python that access the .NET API. Because of the way the API works
running scripts is a 2 stage process. First you select a script from a list then
run the selected script. All scripts must therefore share a common calling
function name and I wanted to test this function existed.

I will have no idea the name or how many or how complicated the scripts will be
so it seemed a good idea to try and free up memory from the scripts that won't
be run.
(It's hard to judge a poster's level of expertise in Python without any

example >code from him.

I'm not a professional programmer so my terminology is probably confusing.It
looks like I shouldn't worry about memory issues.

Thanks for your help Peter and John,

Guy