473,320 Members | 1,936 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Usual practice: running/testing modules in a package

Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python -V
Python 2.5.2

~/python/testpackage$ echo $PYTHONPATH
..:/home/ale/python/libs:/home/ale/lib/python/:

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, fair enough. There's something about running with -m for running
modules:

~/python/testpackage$ python -m importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

Ok, maybe it can't see the __init__.py because it is in the current
directory. Go up one.

~/python$ python testpackage/importer.py
Traceback (most recent call last):
File "testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

No. What about:
~/python$ python -m testpackage.importer
Traceback (most recent call last):
File "/usr/lib/python2.5/runpy.py", line 95, in run_module
filename, loader, alter_sys)
File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_code
mod_name, mod_fname, mod_loader)
File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
exec code in run_globals
File "/home/ale/python/testpackage/importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

That one is very puzzling. It knows that testpackage is a package
because I am invoking it with testpackage.importer, but still thinks
it isn't a package.

A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.

Aug 13 '08 #1
7 2164
En Wed, 13 Aug 2008 04:57:32 -0300, alito <al******@gmail.comescribió:
Hi all,

I am new to using packages to group my modules. I can't figure out
how to run a module that uses relative imports without writing a
wrapper that imports that module. Everything I try it complains that
I am attempting a relative import in a non-package.

eg
~/python/testpackage$ ls
config.py importer.py __init__.py

~/python/testpackage$ cat importer.py
from . import config
config.hello()

~/python/testpackage$ cat config.py
def hello():
print 'hello'

__init__.py is empty

~/python/testpackage$ python importer.py
Traceback (most recent call last):
File "importer.py", line 1, in <module>
from . import config
ValueError: Attempted relative import in non-package

[...more attempts...]

So, how do I run these modules without writing a wrapper script for
each one? My main use actually would be to get pylint to analyse
them, but it seems to use the python import system, so it fails
whenever python fails. If anyone can tell me how to get pychecker to
even be able to use a wrapper, it would also be very beneficial.
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
I usually put the test code or the main entry point one level above the package in the development environment. This way, "import testpackage" works almost the same as it would in the production environment (where the package is installed, probably in site-packages)

--
Gabriel Genellina

Aug 18 '08 #2
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m” to do that.
So (in my humbled opinion), that is one error Python must fix.
Aug 22 '08 #3
En Fri, 22 Aug 2008 10:48:50 -0300, me*****@gmail.com <me*****@gmail.com>
escribió:
On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.arwrote:
>A package is a library, meant to be imported by some other code. Your
main script (or the testing code) is a program, it uses (i.e. imports)
the library.

You are right that a module is a library and its main use is to be
imported from another module or program. But is a common practice in
Python to introduce testing code as part as the module itself. Even
there is an option “-m†to do that.
So (in my humbled opinion), that is one error Python must fix.
A *module* or a *package*? The OP asked how to test a package, and a
package is clearly a library. In the final application, it will be
imported. So the test code should mimic the same environment, and import
the package.

The easiest way to import a package is to run a script from its container
directory: the current directory is already in sys.path so the package
directory is found directly without messing with sys.path, PYTHONPATH
variable and such. This is my suggested approach, but there are many other
alternatives. Anyway, the idea is to import the package to test it.

If you don't import the package and just run some script from inside it,
how would Python know that it belongs to a package? Looking for a
__init__.py in the same directory? what if __init__.py contains code? when
should it be executed?

--
Gabriel Genellina

Aug 22 '08 #4
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
what if __init__.py contains code?
Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.

Regards
Aug 22 '08 #5
En Fri, 22 Aug 2008 17:31:58 -0300, Medardo Rodriguez (Merchise Group) <me*****@gmail.comescribió:
On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
<ga*******@yahoo.com.arwrote:
>what if __init__.py contains code?

Thats what I usually do to solve the "problem", but for my taste it's
better to write the test code of a module inside it.
The code I write in "__init__.py" is related to structures of
initializations, globals at package level.

if __name__ == "__main__":
pass # Here I test only concepts related to the entire package,
not to any module.
I think you misunderstood the comment.
Suppose this setup:

SomeDir/
SomePackage/
__init__.py
module1.py
module2.py
test.py

When someone executes test.py directly, currently Python doesn't know that it is contained inside a package, and some forms of relative imports don't work.
To make relative imports work, Python should be aware that test.py is contained inside a package. Looking for an __init__.py file at the same directory may reveal that there is a package - but that's not enough, because when a package is actually imported, its __init__.py is executed and a new module object is placed in sys.modules.
So, when test.py is run directly, what to do with the code in __init__.py? should it be executed, and when? I don't know the actual reasons, but this seems enough trouble to me to NOT automatically recognize a package unless someone actually imports it.

The application will import the package anyway, so why would the test code not do the same thing? I want to mimic the production environment as closely as possible in the testing environment. And the easiest way to do that is to "import SomePackage" in a script placed at SomeDir. The actual tests may reside in test.py, of course, but now Python *knows* that test.py lives inside a package and relative imports work fine now.

--
Gabriel Genellina

Aug 24 '08 #6
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
>
A wrapper on the level up works:

~/python$ cat importercaller.py
from testpackage import config
config.hello()

~/python$ python importercaller.py
hello

So, how do I run these modules without writing a wrapper script for
each one?
I just ran into this problem trying to run a package from a local
directory. I agree creating another script just to import the local
package is a bad solution. Without another option, I would recommend
using a Makefile and creating a target to import the package and
kickoff the execution:

hello:
python -c 'from testpackage import config; config.hello()'

and then "make hello" to execute the command.

Aug 27 '08 #7
On Aug 26, 10:21*pm, Casey <casey.mcgi...@gmail.comwrote:
On Aug 12, 9:57*pm, alito <alito...@gmail.comwrote:
A wrapper on the level up works:
~/python$ cat importercaller.py
from testpackage import config
config.hello()
~/python$ python importercaller.py
hello
So, how do I run these modules without writing a wrapper script for
each one?

It looks like PEP 366 [http://www.python.org/dev/peps/pep-0366/] will
solve this issue.
Sep 3 '08 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: grenness | last post by:
Sorry for bothering you with a basic question. I have an account with www.webhost4life.com which includes PHP and MySQL on an IIS 6.0/Win2k3 Server. I have succeeded in installing phpbb, and...
1
by: Rolfe | last post by:
Hi, I struggled, and got mod_python running on Apache/Win2k. Follow these instructions verbatim and you shouldn't have any trouble. These instructions are based on...
10
by: Noah | last post by:
I would like to package my main script and all the modules it imports into a single script that will run just as the collection would. It should not need to package standard Python lib modules --...
3
by: Alan Isaac | last post by:
This is a simple question about actual practice. I just want to know how you (yes you) are approaching this problem. The problem: What is the recommended packaging of demo scripts or test...
39
by: mike3 | last post by:
Hi. I was writing a program in C++ that generates fractals. I got this weird bug though right now that's holding it up and was wondering if you could help. Anyway, it seems that when this...
0
by: Matthew Fitzgibbons | last post by:
I'm by no means a testing expert, but I'll take a crack at it. Casey McGinty wrote: I've never run into this. Rule of thumb: always separate software from hardware. Write mock classes or...
1
by: Juan | last post by:
Hi I am programming a little script that makes use of a module I developed before. The utils are inside the directory src of the directory utils, and the package is nutum.utils. The script is in...
10
by: Brendan Miller | last post by:
What would heavy python unit testers say is the best framework? I've seen a few mentions that maybe the built in unittest framework isn't that great. I've heard a couple of good things about...
11
by: r0g | last post by:
I'm collecting together a bunch of fairly random useful functions I have written over the years into a module. Generally speaking is it best to a) Import all the other modules these functions...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.