468,284 Members | 1,511 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,284 developers. It's quick & easy.

Finding a module's sub modules at runtime

[If this is documented somewhere, please just point me there. I googled on
the terms that made sense to me, and didn't find anything.]

So, I have:

ModTest
__init__.py
AModule.py
BModule.py
CModule.py

All works fine. However, when I import ModTest, I would like it to discover
and store the names of the modules beneath it, and construct a list, say
mod_list, that I can access later to find the names of the sub-modules in
this module. Kind of setting __all__ at run time, I guess (yes, I'm aware
of the case caveats).

I figured __init__.py coudl take its own __path__ and walk the directory to
find all .py files other than __init__.py, but that seemed hackish. Is
there an "official" way to do this? Or a better way?

To give "context:" all the modules will have classes that have the same
name, same methods etc. One of the modules will be picked depending on
which implementation is needed.

Thanks!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer
http://www.eeinternet.com
PGP Key: http://pgp.mit.edu/ *ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

Mar 28 '07 #1
8 4440
On Mar 28, 2:44 pm, "Joshua J. Kugler" <jos...@eeinternet.comwrote:
[If this is documented somewhere, please just point me there. I googled on
the terms that made sense to me, and didn't find anything.]

So, I have:

ModTest
__init__.py
AModule.py
BModule.py
CModule.py

All works fine. However, when I import ModTest, I would like it to discover
and store the names of the modules beneath it, and construct a list, say
mod_list, that I can access later to find the names of the sub-modules in
this module. Kind of setting __all__ at run time, I guess (yes, I'm aware
of the case caveats).

I figured __init__.py coudl take its own __path__ and walk the directory to
find all .py files other than __init__.py, but that seemed hackish. Is
there an "official" way to do this? Or a better way?

To give "context:" all the modules will have classes that have the same
name, same methods etc. One of the modules will be picked depending on
which implementation is needed.

Thanks!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmerhttp://www.eeinternet.com
PGP Key:http://pgp.mit.edu/ ID 0xDB26D7CE

--
Posted via a free Usenet account fromhttp://www.teranews.com
I think you need to research how to create documentation. When I
import a module/package, I can then type help(moduleName) and it'll
give me the module or package's contents.

http://mail.python.org/pipermail/pyt...ry/192069.html

You may be able to figure out how to do this just be studying the
"help" module itself.

Mike

Mar 28 '07 #2
On Wednesday 28 March 2007 12:04, ky******@gmail.com wrote:
>All works fine. However, when I import ModTest, I would like it to
discover and store the names of the modules beneath it, and construct a
list, say mod_list, that I can access later to find the names of the
sub-modules in
this module. Kind of setting __all__ at run time, I guess (yes, I'm
aware of the case caveats).

I figured __init__.py coudl take its own __path__ and walk the directory
to
find all .py files other than __init__.py, but that seemed hackish. Is
there an "official" way to do this? Or a better way?

To give "context:" all the modules will have classes that have the same
name, same methods etc. One of the modules will be picked depending on
which implementation is needed.

I think you need to research how to create documentation. When I
import a module/package, I can then type help(moduleName) and it'll
give me the module or package's contents.

http://mail.python.org/pipermail/pyt...ry/192069.html

You may be able to figure out how to do this just be studying the
"help" module itself.

Mike
Well, it seems the "help" module is a built in, and has no .py file
anywhere. The epydoc package uses the imports.file_modules(dirname)
function, which just walks the directory tree. Thanks for the
pointer...just confirmed that I have to do something I wanted to avoid.
But I guess if I use Python's os module and the __path__ string, it should
still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer
http://www.eeinternet.com
PGP Key: http://pgp.mit.edu/ *ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

Mar 28 '07 #3
Joshua J. Kugler <jo****@eeinternet.comwrote:
still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.
Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?
Alex
Mar 29 '07 #4
On Thursday 29 March 2007 07:33, Alex Martelli wrote:
Joshua J. Kugler <jo****@eeinternet.comwrote:
>still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.

Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?
Hmm...right: dynamic language, runtime binding. It would go out and find
whether or not Module1.Module2 existed only when import was called. Sorry,
my bad. I am assuming, however, that the help function walks the directory
tree, or else I would not get output like this:
>>import ModTest
help(ModTest)
Help on package ModTest:

NAME
ModTest

FILE
/usr/local/lib/python2.4/site-packages/ModTest/__init__.py

PACKAGE CONTENTS
AModule
BModule
CModule

Thanks for the clarification.

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer
http://www.eeinternet.com
PGP Key: http://pgp.mit.edu/ *ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

Mar 29 '07 #5
Joshua J. Kugler <jo****@eeinternet.comwrote:
On Thursday 29 March 2007 07:33, Alex Martelli wrote:
Joshua J. Kugler <jo****@eeinternet.comwrote:
still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.
Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?

Hmm...right: dynamic language, runtime binding. It would go out and find
whether or not Module1.Module2 existed only when import was called. Sorry,
my bad. I am assuming, however, that the help function walks the directory
tree, or else I would not get output like this:
>import ModTest
help(ModTest)
Help on package ModTest:

NAME
ModTest

FILE
/usr/local/lib/python2.4/site-packages/ModTest/__init__.py

PACKAGE CONTENTS
AModule
BModule
CModule

Thanks for the clarification.
Sure, pydoc (which help calls under the code) does that, with a nice mix
of inspect, os, and pkgutil.iter_modules calls. pkgutil.iter_modules
may in fact be most of what you need:
>>help(pkgutil.iter_modules)
Help on function iter_modules in module pkgutil:

iter_modules(path=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

You will still have to call it recursively, though, I believe.
Alex
Mar 30 '07 #6
On Thursday 29 March 2007 17:58, Alex Martelli wrote:
Sure, pydoc (which help calls under the code) does that, with a nice mix
of inspect, os, and pkgutil.iter_modules calls. pkgutil.iter_modules
may in fact be most of what you need:
>>>help(pkgutil.iter_modules)
Help on function iter_modules in module pkgutil:

iter_modules(path=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.
OK, that looks nice...but what version of Python is that?
http://docs.python.org/lib/module-pkgutil.html only shows one function (and
that's 2.5) and my python 2.4 installation is similarly lacking an
iter_modules() function for the pkgutil module. Is this a 2.6 thing?

Suppose I could just copy the code from here:
http://mail.python.org/pipermail/pyt...il/051452.html and
add it to my module. :)

Thanks for the pointer!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer
http://www.eeinternet.com
PGP Key: http://pgp.mit.edu/ *ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

Apr 3 '07 #7
Joshua J. Kugler wrote:
On Thursday 29 March 2007 17:58, Alex Martelli wrote:
>Sure, pydoc (which help calls under the code) does that, with a nice mix
of inspect, os, and pkgutil.iter_modules calls. pkgutil.iter_modules
may in fact be most of what you need:
>>>>help(pkgutil.iter_modules)
Help on function iter_modules in module pkgutil:

iter_modules(path=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

OK, that looks nice...but what version of Python is that?
http://docs.python.org/lib/module-pkgutil.html only shows one function (and
that's 2.5) and my python 2.4 installation is similarly lacking an
iter_modules() function for the pkgutil module. Is this a 2.6 thing?
No, 2.5. The documentation is not up to date. Read the source.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Apr 3 '07 #8
On Monday 02 April 2007 16:33, Robert Kern wrote:
>>>>>help(pkgutil.iter_modules)
Help on function iter_modules in module pkgutil:

iter_modules(path=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

OK, that looks nice...but what version of Python is that?
http://docs.python.org/lib/module-pkgutil.html only shows one function
(and that's 2.5) and my python 2.4 installation is similarly lacking an
iter_modules() function for the pkgutil module. Is this a 2.6 thing?

No, 2.5. The documentation is not up to date. Read the source.
Gotcha. Thanks...well, since we're using 2.4, that will have to wait. For
the archives, here is what I've come up with.

Contents of the __init__.py for a module.

import os
_myDir = __path__[0]

def mod_list():
"""
A quick hack that retrieves all the sub modules in a directory that has
an __init__.py file. I could use pkgutil.iter_modules, but that is
Python 2.5 only, and this should work with several versions of Python.
"""
modList = []
modHash = {}
isModule = False
for ii in os.walk(_myDir):
if ii[0] == _myDir:
for f in ii[2]:
# If there is no __init__ file, then the directory
# upon which mod_list() is operating is not a module
if f[0:8] == '__init__':
isModule = True
elif f[-3:] == '.py':
modHash[f[:-3]] = True
elif f[-4:] == '.pyc' or f[-4:] == '.pyo':
modHash[f[:-4]] = True
if isModule:
modList = modHash.keys()
modList.sort()
return(modList)
else:
return(None)

Hope that helps someone!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer
http://www.eeinternet.com
PGP Key: http://pgp.mit.edu/ *ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

Apr 3 '07 #9

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

7 posts views Thread by Jorgen Grahn | last post: by
3 posts views Thread by benoit laude | last post: by
13 posts views Thread by pb648174 | last post: by
6 posts views Thread by mwt | last post: by
10 posts views Thread by Jia Lu | last post: by
6 posts views Thread by SMcK | last post: by
1 post views Thread by Juan | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.