472,348 Members | 2,039 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,348 software developers and data experts.

Usual practice: running/testing modules in a package

Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python -V
Python 2.5.2

~/python/testpackage$ echo $PYTHONPATH
..:/home/ale/python/libs:/home/ale/lib/python/:

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, fair enough. There's something about running with -m for running
modules:

~/python/testpackage$ python -m importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, maybe it can't see the __init__.py because it is in the current
directory. Go up one.

~/python$ python testpackage/importer.py
Traceback (most recent call last):
File "testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

No. What about:
~/python$ python -m testpackage.importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

That one is very puzzling. It knows that testpackage is a package
because I am invoking it with testpackage.importer, but still thinks
it isn't a package.

A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.

Aug 13 '08 #1
7 2082
En Wed, 13 Aug 2008 04:57:32 -0300, alito <al******@gmail.comescribió:
Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

[...more attempts...]

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
I usually put the test code or the main entry point one level above the package in the development environment. This way, "import testpackage" works almost the same as it would in the production environment (where the package is installed, probably in site-packages)

--
Gabriel Genellina

Aug 18 '08 #2
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m” to do that.
So (in my humbled opinion), that is one error Python must fix.
Aug 22 '08 #3
En Fri, 22 Aug 2008 10:48:50 -0300, me*****@gmail.com <me*****@gmail.com>
escribió:
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
>A package is a library, meant to be imported by some other code. Your
main script (or the testing code) is a program, it uses (i.e. imports)
the library.

You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m” to do that.
So (in my humbled opinion), that is one error Python must fix.
A *module* or a *package*? The OP asked how to test a package, and a
package is clearly a library. In the final application, it will be
imported. So the test code should mimic the same environment, and import
the package.

The easiest way to import a package is to run a script from its container
directory: the current directory is already in sys.path so the package
directory is found directly without messing with sys.path, PYTHONPATH
variable and such. This is my suggested approach, but there are many other
alternatives. Anyway, the idea is to import the package to test it.

If you don't import the package and just run some script from inside it,
how would Python know that it belongs to a package? Looking for a
__init__.py in the same directory? what if __init__.py contains code? when
should it be executed?

--
Gabriel Genellina

Aug 22 '08 #4
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
what if __init__.py contains code?
Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.

Regards
Aug 22 '08 #5
En Fri, 22 Aug 2008 17:31:58 -0300, Medardo Rodriguez (Merchise Group) <me*****@gmail.comescribió:
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
>what if __init__.py contains code?

Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.
I think you misunderstood the comment.
Suppose this setup:

SomeDir/
SomePackage/
__init__.py
module1.py
module2.py
test.py

When someone executes test.py directly, currently Python doesn't know that it is contained inside a package, and some forms of relative imports don't work.
To make relative imports work, Python should be aware that test.py is contained inside a package. Looking for an __init__.py file at the same directory may reveal that there is a package - but that's not enough, because when a package is actually imported, its __init__.py is executed and a new module object is placed in sys.modules.
So, when test.py is run directly, what to do with the code in __init__.py? should it be executed, and when? I don't know the actual reasons, but this seems enough trouble to me to NOT automatically recognize a package unless someone actually imports it.

The application will import the package anyway, so why would the test code not do the same thing? I want to mimic the production environment as closely as possible in the testing environment. And the easiest way to do that is to "import SomePackage" in a script placed at SomeDir. The actual tests may reside in test.py, of course, but now Python *knows* that test.py lives inside a package and relative imports work fine now.

--
Gabriel Genellina

Aug 24 '08 #6
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
>
A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one?
I just ran into this problem trying to run a package from a local
directory. I agree creating another script just to import the local
package is a bad solution. Without another option, I would recommend
using a Makefile and creating a target to import the package and
kickoff the execution:

hello:
python -c 'from testpackage import config; config.hello()'

and then "make hello" to execute the command.

Aug 27 '08 #7
On Aug 26, 10:21*pm, Casey <casey.mcgi...@gmail.comwrote:
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
A wrapper on the level up works:
~/python$ cat importercaller.py
from testpackage import config
config.hello()
~/python$ python importercaller.py
hello
So, how do I run these modules without writing a wrapper script for
each one?

It looks like PEP 366 [http://www.python.org/dev/peps/pep-0366/] will
solve this issue.
Sep 3 '08 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: grenness | last post by:
Sorry for bothering you with a basic question. I have an account with www.webhost4life.com which includes PHP and MySQL on an IIS 6.0/Win2k3...
1
by: Rolfe | last post by:
Hi, I struggled, and got mod_python running on Apache/Win2k. Follow these instructions verbatim and you shouldn't have any trouble. These...
10
by: Noah | last post by:
I would like to package my main script and all the modules it imports into a single script that will run just as the collection would. It should...
3
by: Alan Isaac | last post by:
This is a simple question about actual practice. I just want to know how you (yes you) are approaching this problem. The problem: What is the...
39
by: mike3 | last post by:
Hi. I was writing a program in C++ that generates fractals. I got this weird bug though right now that's holding it up and was wondering if you...
0
by: Matthew Fitzgibbons | last post by:
I'm by no means a testing expert, but I'll take a crack at it. Casey McGinty wrote: I've never run into this. Rule of thumb: always separate...
1
by: Juan | last post by:
Hi I am programming a little script that makes use of a module I developed before. The utils are inside the directory src of the directory...
10
by: Brendan Miller | last post by:
What would heavy python unit testers say is the best framework? I've seen a few mentions that maybe the built in unittest framework isn't that...
11
by: r0g | last post by:
I'm collecting together a bunch of fairly random useful functions I have written over the years into a module. Generally speaking is it best to ...
0
better678
by: better678 | last post by:
Question: Discuss your understanding of the Java platform. Is the statement "Java is interpreted" correct? Answer: Java is an object-oriented...
0
by: teenabhardwaj | last post by:
How would one discover a valid source for learning news, comfort, and help for engineering designs? Covering through piles of books takes a lot of...
1
by: Kemmylinns12 | last post by:
Blockchain technology has emerged as a transformative force in the business world, offering unprecedented opportunities for innovation and...
0
by: Naresh1 | last post by:
What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge...
0
by: antdb | last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine In the overall architecture, a new "hyper-convergence" concept was...
2
by: Matthew3360 | last post by:
Hi, I have a python app that i want to be able to get variables from a php page on my webserver. My python app is on my computer. How would I make it...
0
by: AndyPSV | last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable...
0
by: Arjunsri | last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and...
0
by: Carina712 | last post by:
Setting background colors for Excel documents can help to improve the visual appeal of the document and make it easier to read and understand....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.