467,222 Members | 1,457 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,222 developers. It's quick & easy.

Usual practice: running/testing modules in a package

Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python -V
Python 2.5.2

~/python/testpackage$ echo $PYTHONPATH
..:/home/ale/python/libs:/home/ale/lib/python/:

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, fair enough. There's something about running with -m for running
modules:

~/python/testpackage$ python -m importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, maybe it can't see the __init__.py because it is in the current
directory. Go up one.

~/python$ python testpackage/importer.py
Traceback (most recent call last):
File "testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

No. What about:
~/python$ python -m testpackage.importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

That one is very puzzling. It knows that testpackage is a package
because I am invoking it with testpackage.importer, but still thinks
it isn't a package.

A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.

Aug 13 '08 #1
  • viewed: 1861
Share:
7 Replies
En Wed, 13 Aug 2008 04:57:32 -0300, alito <al******@gmail.comescribió:
Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

[...more attempts...]

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
I usually put the test code or the main entry point one level above the package in the development environment. This way, "import testpackage" works almost the same as it would in the production environment (where the package is installed, probably in site-packages)

--
Gabriel Genellina

Aug 18 '08 #2
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m” to do that.
So (in my humbled opinion), that is one error Python must fix.
Aug 22 '08 #3
En Fri, 22 Aug 2008 10:48:50 -0300, me*****@gmail.com <me*****@gmail.com>
escribió:
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
>A package is a library, meant to be imported by some other code. Your
main script (or the testing code) is a program, it uses (i.e. imports)
the library.

You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m” to do that.
So (in my humbled opinion), that is one error Python must fix.
A *module* or a *package*? The OP asked how to test a package, and a
package is clearly a library. In the final application, it will be
imported. So the test code should mimic the same environment, and import
the package.

The easiest way to import a package is to run a script from its container
directory: the current directory is already in sys.path so the package
directory is found directly without messing with sys.path, PYTHONPATH
variable and such. This is my suggested approach, but there are many other
alternatives. Anyway, the idea is to import the package to test it.

If you don't import the package and just run some script from inside it,
how would Python know that it belongs to a package? Looking for a
__init__.py in the same directory? what if __init__.py contains code? when
should it be executed?

--
Gabriel Genellina

Aug 22 '08 #4
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
what if __init__.py contains code?
Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.

Regards
Aug 22 '08 #5
En Fri, 22 Aug 2008 17:31:58 -0300, Medardo Rodriguez (Merchise Group) <me*****@gmail.comescribió:
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
>what if __init__.py contains code?

Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.
I think you misunderstood the comment.
Suppose this setup:

SomeDir/
SomePackage/
__init__.py
module1.py
module2.py
test.py

When someone executes test.py directly, currently Python doesn't know that it is contained inside a package, and some forms of relative imports don't work.
To make relative imports work, Python should be aware that test.py is contained inside a package. Looking for an __init__.py file at the same directory may reveal that there is a package - but that's not enough, because when a package is actually imported, its __init__.py is executed and a new module object is placed in sys.modules.
So, when test.py is run directly, what to do with the code in __init__.py? should it be executed, and when? I don't know the actual reasons, but this seems enough trouble to me to NOT automatically recognize a package unless someone actually imports it.

The application will import the package anyway, so why would the test code not do the same thing? I want to mimic the production environment as closely as possible in the testing environment. And the easiest way to do that is to "import SomePackage" in a script placed at SomeDir. The actual tests may reside in test.py, of course, but now Python *knows* that test.py lives inside a package and relative imports work fine now.

--
Gabriel Genellina

Aug 24 '08 #6
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
>
A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one?
I just ran into this problem trying to run a package from a local
directory. I agree creating another script just to import the local
package is a bad solution. Without another option, I would recommend
using a Makefile and creating a target to import the package and
kickoff the execution:

hello:
python -c 'from testpackage import config; config.hello()'

and then "make hello" to execute the command.

Aug 27 '08 #7
On Aug 26, 10:21*pm, Casey <casey.mcgi...@gmail.comwrote:
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
A wrapper on the level up works:
~/python$ cat importercaller.py
from testpackage import config
config.hello()
~/python$ python importercaller.py
hello
So, how do I run these modules without writing a wrapper script for
each one?

It looks like PEP 366 [http://www.python.org/dev/peps/pep-0366/] will
solve this issue.
Sep 3 '08 #8

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by grenness | last post: by
1 post views Thread by Rolfe | last post: by
3 posts views Thread by Alan Isaac | last post: by
39 posts views Thread by mike3 | last post: by
reply views Thread by Matthew Fitzgibbons | last post: by
1 post views Thread by Juan | last post: by
10 posts views Thread by Brendan Miller | last post: by
11 posts views Thread by r0g | last post: by
reply views Thread by Adict | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.