modules and generated code

[Nigel Rantor]

Hi all,

Python newbie here with what I hope is a blindingly obvious question
that I simply can't find the answer for in the documentation.

So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of placing my
own code under the same package hierarchy without all the code living in
the same directory structure.

Ideally I would like something like the following:

package_dir/
top_level_package/
generated_code_package/
implementation_code_package/

but have two distinct directories that hold them so that I can simply
delete the generated code and regenerate it without worrying that
anything got left behind.

So, I want something like:

generated_package_dir/
top_level_package/
generated_code_package/

implementation_package_dir/
top_level_package/
implementation_code_package/

Whilst I can create this structure, and add 'generated_package_dir' and
'implementation_package_dir' to PYTHONPATH the fact that both
directories contain 'top_level_package' seems to be causing clashes,
perhaps because there are multiple __init__.py files for
'top_level_package'?

I know that this is possible in Java, Perl and C++ so I am finding it
hard to believe I can't do the same in Python, I just think I'm too new
to know how.

I have spent most of this morning searching through all the docs I can
find, searching on USENET and the web to no avail.

Any help or pointers greatly appreciated.

Regards,

n

[Peter Otten]

Nigel Rantor wrote:

So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of placing my
own code under the same package hierarchy without all the code living in
the same directory structure.

http://docs.python.org/lib/module-pkgutil.html

Peter

[Nigel Rantor]

Peter Otten wrote:

Nigel Rantor wrote:

>So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of placing my
own code under the same package hierarchy without all the code living in
the same directory structure.

http://docs.python.org/lib/module-pkgutil.html

Ooh, thanks for that.

Yep, looks like that should work, but it doesn't. :-/

Do you have any idea whether other __init__.py scripts from the same
logical module will still be run in this case?

The generated code uses its init script to pull in other code.

Off, to tinker some more with this.

n

[Peter Otten]

Nigel Rantor wrote:

Peter Otten wrote:

>Nigel Rantor wrote:

>>So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of placing my
own code under the same package hierarchy without all the code living in
the same directory structure.

>http://docs.python.org/lib/module-pkgutil.html

Yep, looks like that should work, but it doesn't. :-/

Do you have any idea whether other __init__.py scripts from the same
logical module will still be run in this case?

I don't think it will.

The generated code uses its init script to pull in other code.

You could invoke it explicitly via

execfile("/path/to/generated/package/__init__.py")

in the static package/__init__.py.

Peter

[Nigel Rantor]

Peter Otten wrote:

Nigel Rantor wrote:

>Peter Otten wrote:

>>Nigel Rantor wrote:

>>>So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of placing my
own code under the same package hierarchy without all the code living in
the same directory structure.

>>http://docs.python.org/lib/module-pkgutil.html

>Yep, looks like that should work, but it doesn't. :-/

Do you have any idea whether other __init__.py scripts from the same
logical module will still be run in this case?

I don't think it will.

Yeah, I am getting that impression. Gah!

>The generated code uses its init script to pull in other code.

You could invoke it explicitly via

execfile("/path/to/generated/package/__init__.py")

in the static package/__init__.py.

Hmm, yes, that works. It's not pretty though, it seems to be finding the
file relative to the current directory, I suppose writing a bit of code
that figures out where this package is located and modifying it won't be
too hard.

And, at the risk of being flamed or sounding like a troll, this seems
like something that should be easy to do...other languages manage it
quite neatly. Up until this point I was really liking my exposure to
Python :-/

I wonder if there is any more magic that I'm missing, the thing is the
pkgutil method looks exactly like what I want, except for not executing
the additional __init__.py files in the added directories.

Thanks for the help so far Peter, if anyone has a prettier solution then
I'm all ears.

Cheers,

n

[J. Clifford Dyer]

Nigel Rantor wrote:

Peter Otten wrote:

>Nigel Rantor wrote:

>>Peter Otten wrote:
Nigel Rantor wrote:

>>>>So, if I have a tool that generates python code for me (in my case,
CORBA stubs/skels) in a particular package is there a way of
placing my
own code under the same package hierarchy without all the code
living in
the same directory structure.

>>>http://docs.python.org/lib/module-pkgutil.html

>>Yep, looks like that should work, but it doesn't. :-/

Do you have any idea whether other __init__.py scripts from the same
logical module will still be run in this case?

I don't think it will.

Yeah, I am getting that impression. Gah!

>>The generated code uses its init script to pull in other code.

You could invoke it explicitly via
execfile("/path/to/generated/package/__init__.py")
in the static package/__init__.py.

Hmm, yes, that works. It's not pretty though, it seems to be finding the
file relative to the current directory, I suppose writing a bit of code
that figures out where this package is located and modifying it won't be
too hard.

And, at the risk of being flamed or sounding like a troll, this seems
like something that should be easy to do...other languages manage it
quite neatly. Up until this point I was really liking my exposure to
Python :-/

I wonder if there is any more magic that I'm missing, the thing is the
pkgutil method looks exactly like what I want, except for not executing
the additional __init__.py files in the added directories.

Thanks for the help so far Peter, if anyone has a prettier solution then
I'm all ears.

Cheers,

n

Maybe I'm missing something obvious, but it sounds like you are
over-complicating the idea of inheritance. Do you just want to create a
subclass of the other class?

Cheers,
Cliff

[Nigel Rantor]

J. Clifford Dyer wrote:

>
Maybe I'm missing something obvious, but it sounds like you are
over-complicating the idea of inheritance. Do you just want to create a
subclass of the other class?

Nope, that isn't my problem.

I have an IDL file that is used to generate a set of stub and skeleton
code that is not human-modifiable.

Eventually I would like to have my IDL in source control and have a
setup script able to generate my stubs and skels and install them for me.

At the same time I want to produce code that uses this code but in the
same package.

In Java or Perl I can easily create a couple package/module like this:

package org.mine.package;

[...class definitions...]
and then somewhere else

package org.mine.otherpackage;

[...class definitions...]

These can be compiled into separate Jar files and just work.

Since the python is the final target though I don't want to put it all
in one directory because then I need to be clever when I regenerate the
generated code, I don't want old python modules lying around that are no
longer in the IDL. Blowing the entire generated directory away is the
best way of doing this, so I don't want my implementation code in there.

Basically, I want the same top-level package to have bits of code in
different directories, but because Python requires the __init__.py file
it only picks up the first one in PYTHONPATH.

I'm not sure if that makes sense, my brain is already toast from
meetings today.

n

[Fredrik Lundh]

Nigel Rantor wrote:

Basically, I want the same top-level package to have bits of code in
different directories, but because Python requires the __init__.py file
it only picks up the first one in PYTHONPATH.

would a single __init__.py function that does from-import-* on the
various implementation modules solve your problem?

</F>

[Gabriel Genellina]

At Wednesday 15/11/2006 14:33, Nigel Rantor wrote:

>I have an IDL file that is used to generate a set of stub and skeleton
code that is not human-modifiable.

Eventually I would like to have my IDL in source control and have a
setup script able to generate my stubs and skels and install them for me.

At the same time I want to produce code that uses this code but in the
same package.

[...]
Basically, I want the same top-level package to have bits of code in
different directories, but because Python requires the __init__.py file
it only picks up the first one in PYTHONPATH.

Put both directories in the same package. But this is the obvious
thing, so maybe I'm missing something.
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar