How to: get list of modules in a package

[Dan Perl]

Is there a direct way to get a list of all the modules in a package, other
than using an os.listdir() for the directory of the package?

To be more specific, I would like to inspect every module in a package and
check if it contains the definition of a certain class. At the same time, I
would like to be able to add modules to this package in a dynamic way, so
__init__.py would not be of much help because I do not want to change it for
every new module.

Thanks,

Dan

[Peter L Hansen]

Dan Perl wrote:

Is there a direct way to get a list of all the modules in a package, other
than using an os.listdir() for the directory of the package?

Given the dynamic nature of Python, I doubt that's easy to do.
I'm not sure even os.listdir(), for example, can be relied on,
given that you can import modules from zip files, or if the
import hook is used, even from remote locations or, presumably
(though I haven't seen it done yet), from "thin air" where
the module is generated dynamically.

-Peter

[Mike C. Fletcher]

Peter L Hansen wrote:
....

Given the dynamic nature of Python, I doubt that's easy to do.
I'm not sure even os.listdir(), for example, can be relied on,
given that you can import modules from zip files, or if the
import hook is used, even from remote locations or, presumably
(though I haven't seen it done yet), from "thin air" where
the module is generated dynamically.

I used to do this somewhere in mcf.vrml (old VRML97 processing
package). The idea was to generate a module to hold classes which were
also generated on-the-fly (from VRML97 prototypes). You could then
import the classes from the non-existent module as long as the module
itself (which roughly corresponded to a VRML97 file) was part of your
pickle. The package holding those modules was empty until such time as
you either generated one of those modules or loaded one from a pickle,
and there was just no reasonable way to know what would eventually show
up there.

Advances in Python (and my moving away from my perverse love of such
tricks ;) ) made that particular usage obsolete, but it definitely has
been done in the past...

Enjoy,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

[Dan Perl]

Thanks, Peter. So the fact I didn't see any solution was not due to my lack
of knowledge. I thought I should ask and maybe someone would point out an
elegant solution that was just beyond my own grasp of Python.

os.listdir() would not work in the general case, but it's good enough in my
limited case. The package is well-determined and it's always a directory.
Even if I will change that some day, there will always be another equivalent
for the os.listdir.

So thanks again, I can confidently go and implement a solution based on
os.listdir.

Dan

"Peter L Hansen" <pe***@engcorp.com> wrote in message
news:df********************@powergate.ca...

Dan Perl wrote:

Is there a direct way to get a list of all the modules in a package,
other than using an os.listdir() for the directory of the package?

Given the dynamic nature of Python, I doubt that's easy to do.
I'm not sure even os.listdir(), for example, can be relied on,
given that you can import modules from zip files, or if the
import hook is used, even from remote locations or, presumably
(though I haven't seen it done yet), from "thin air" where
the module is generated dynamically.

-Peter

[Dan Perl]

What a tease! "It definitely has been done in the past", so there must be a
way, but you're not telling us how you did that! :-)

Oh well, I have another solution (based on os.listdir) and even if it is not
the most elegant one that could exist, at least it's going to be clear to
understand.

Dan

"Mike C. Fletcher" <mc******@rogers.com> wrote in message
news:ma**************************************@pyth on.org...

Peter L Hansen wrote:
...

Given the dynamic nature of Python, I doubt that's easy to do.
I'm not sure even os.listdir(), for example, can be relied on,
given that you can import modules from zip files, or if the
import hook is used, even from remote locations or, presumably
(though I haven't seen it done yet), from "thin air" where
the module is generated dynamically.

I used to do this somewhere in mcf.vrml (old VRML97 processing package).
The idea was to generate a module to hold classes which were also
generated on-the-fly (from VRML97 prototypes). You could then import the
classes from the non-existent module as long as the module itself (which
roughly corresponded to a VRML97 file) was part of your pickle. The
package holding those modules was empty until such time as you either
generated one of those modules or loaded one from a pickle, and there was
just no reasonable way to know what would eventually show up there.

Advances in Python (and my moving away from my perverse love of such
tricks ;) ) made that particular usage obsolete, but it definitely has
been done in the past...

Enjoy,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

[Mike C. Fletcher]

Dan Perl wrote:

What a tease! "It definitely has been done in the past", so there must be a
way, but you're not telling us how you did that! :-)

Ah, apparently I wasn't clear enough in my quoted antecedent :) , I was
referring to Peter's comment that he'd not seen anyone produce modules
'from "thin air"', rather than to whether this could be solved in the
general case. I was backing him up in his assertion that there's no
*general* way to solve the problem, rather than attempting to contradict
him :) (those Canucks get dangerous when contradicted).

If you mean you wanted to know how to do the module-from-thin-air thing,
the code for mcf.vrml is available on sourceforge, so it's not like I'm
hiding it ;) , but I doubt you'd want to use that particular pattern
anymore :) .

Have fun,
Mike

Peter L Hansen wrote:
...

....

import hook is used, even from remote locations or, presumably
(though I haven't seen it done yet), from "thin air" where
the module is generated dynamically.

....
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

[Dan Perl]

"Mike C. Fletcher" <mc******@rogers.com> wrote in message
news:ma**************************************@pyth on.org...

Dan Perl wrote:

What a tease! "It definitely has been done in the past", so there must be
a way, but you're not telling us how you did that! :-)
Ah, apparently I wasn't clear enough in my quoted antecedent :) , I was
referring to Peter's comment that he'd not seen anyone produce modules
'from "thin air"', rather than to whether this could be solved in the
general case. I was backing him up in his assertion that there's no
*general* way to solve the problem, rather than attempting to contradict
him :) (those Canucks get dangerous when contradicted).

Got it now! And what do you mean "those" Canucks? From the email address
you seem to be Canadian too! Not to mention that I am Canadian!
If you mean you wanted to know how to do the module-from-thin-air thing,
the code for mcf.vrml is available on sourceforge, so it's not like I'm
hiding it ;) , but I doubt you'd want to use that particular pattern
anymore :) .
No, thanks, that's not what I wanted, and thanks for confirming Peter's
reply. I can move on with my own solution.

Dan
Have fun,
Mike

[Mike C. Fletcher]

Dan Perl wrote:
....

Got it now! And what do you mean "those" Canucks?
Igloo-dwellers from the North... very short tempers... I blame the
strong beer...
From the email address
you seem to be Canadian too!
Scurrilous rumours those. I was born in Montreal, so I'm obviously not
Canadian ;) :) .
Not to mention that I am Canadian!

Really, had no idea ;) :D ... guess I shouldn't contradict you then...

Have fun,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

[Peter L Hansen]

Mike C. Fletcher wrote:

referring to Peter's comment that he'd not seen anyone produce modules
'from "thin air"', rather than to whether this could be solved in the
general case. I was backing him up in his assertion that there's no
*general* way to solve the problem, rather than attempting to contradict
him :) (those Canucks get dangerous when contradicted).

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
No they don't!

-Peter

P.S.: See you at the PyGTA meeting in two weeks, Mike...