473,395 Members | 1,968 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,395 software developers and data experts.

Code Management

Dear Reader,

I'm writing some modules in Python, and I'm also using unittests. I'm
wondering about some things:

1) Should I put my unittests in a subdirectory? Does the subdirectory
have to be a package?

2) Does the main folder /myproject have to be a package? Should I put
my modules directly under /myproject, or should I create a subfolder,
for example /myproject/modules

Does anyone have any "best practices" as to how to manage your code?

Thanks!
Nov 21 '07 #1
11 1336
Let's see:
1) Should I put my unittests in a subdirectory? Does the subdirectory
have to be a package?
Sure, to avoid cluttering put the tests in a folder called 'tests'
(and any input the tests require, like mock files, output files used
to test if the output is correct, etc etc, should be put in that
folder or in a subfolder of 'tests'.

I like this setup:

myproject
.../ package1
../../ tests
.../../../ mock_files
.../package2

and so on

If you want to be able to import the tests from python (say, 'from
myproject.tests import test_all'), then yes, you need to make the
tests folder a subpackage (just drop an empty __init__.py file)
2) Does the main folder /myproject have to be a package? Should I put
my modules directly under /myproject, or should I create a subfolder,
for example /myproject/modules
Of course... make /myproject a package. Now, if your modules are kinda
small you can just dump them in /myproject, but if what you are doing
is fairly complex, it's better to move each module to its own folder.
Does anyone have any "best practices" as to how to manage your code?
I want 'best practices' too... anyone?

Sergio
PS: Casey, neat idea with the 'alltests.py'. Btw, I think relative
imports can be helpful:
http://www.python.org/dev/peps/pep-0...ido-s-decision
Nov 21 '07 #2
Jens <j3****@gmail.comwrites:
On 21 Nov., 04:16, Jens <j3n...@gmail.comwrote:
On 21 Nov., 01:46, brzr...@gmail.com wrote:
dummy/
dummy_package/
__init__.py
moduleA.py
tests/
__init__.py
test.py
To avoid confusion, the directory that is the package should be named
as you want the imports to appear; e.g. if you want to 'import
foo.module_a', then name the directory containing 'module_a.py' as
'foo/'.

This often results in::

foo/
setup.py
foo/
__init__.py
module_a.py
test/
__init__.py
test_module_a.py

That is, the *project* directory (containing all the files) is named
'foo/'; the *package* directory (where all the implementation code is
found) is named 'foo/foo/', and the unit tests are found in the
directory 'foo/test/'.

That's normal, though if it confuses you you might want to rename the
project directory. I'm often doing development on multiple
version-control branches, so each project directory is named for the
branch it contains; within each of those, the same 'foo/' name is used
for the package directory.
I'm using Python 2.5.1. When I'm trying to call a function in
'moduleA' from 'test' it won't work unless I make the 'dummy'
folder a package as well. That's pretty weird. Does
'dummy_package' have to be in my pythonpath or something? How do I
reference moduleA from test?
You should install the package into a place where Python's import path
will find it. Read up on using the standard library 'distutils'
mechanism for this; it involves writing a 'setup.py' file to define
the parameters for installation of your package.
I would like to avoid making 'dummy' into a package as well.
Yes. The top-level directory is used for containing a number of files,
including 'setup.py', that should not be part of the installed
package.
Problem solved. I added 'dummy' to the PYTHONPATH. (Do I really have
to do that for every project I create?) Anyway, it works the way I'd
like it to now.
I hope the above makes it clearer what I prefer for this situation.

--
\ "True greatness is measured by how much freedom you give to |
`\ others, not by how much you can coerce others to do what you |
_o__) want." --Larry Wall |
Ben Finney
Nov 21 '07 #3
As a side note, I find much easier to drop a PTH file than messing
with pythonpath. If you are not familiar with PTH files, what I do is
this

1) Go to "C:\Program Files\Python25\Lib\site-packages" or whatever is
appropiate in your case.
2) Create a text file, name it something like "MyProjects.PTH" (note
the extension!)
3) in the file, just write the path of the folder that contains all
your projects (in my case, C:/docs/python)

The idea is to keep the main python installation separated from the
modules you are currently developing. Your python installation goes to
"program files/python" or "bin/python", and your personal projects go
somewhere else (usually inside your 'user' folder). This smooths many
things, like working with different versions of a package you are
developing.

On Nov 20, 2007 11:34 PM, Ben Finney <bi****************@benfinney.id.auwrote:
Jens <j3****@gmail.comwrites:
On 21 Nov., 04:16, Jens <j3n...@gmail.comwrote:
On 21 Nov., 01:46, brzr...@gmail.com wrote:
dummy/
dummy_package/
__init__.py
moduleA.py
tests/
__init__.py
test.py

To avoid confusion, the directory that is the package should be named
as you want the imports to appear; e.g. if you want to 'import
foo.module_a', then name the directory containing 'module_a.py' as
'foo/'.

This often results in::

foo/
setup.py
foo/
__init__.py
module_a.py
test/
__init__.py
test_module_a.py

That is, the *project* directory (containing all the files) is named
'foo/'; the *package* directory (where all the implementation code is
found) is named 'foo/foo/', and the unit tests are found in the
directory 'foo/test/'.

That's normal, though if it confuses you you might want to rename the
project directory. I'm often doing development on multiple
version-control branches, so each project directory is named for the
branch it contains; within each of those, the same 'foo/' name is used
for the package directory.
I'm using Python 2.5.1. When I'm trying to call a function in
'moduleA' from 'test' it won't work unless I make the 'dummy'
folder a package as well. That's pretty weird. Does
'dummy_package' have to be in my pythonpath or something? How do I
reference moduleA from test?

You should install the package into a place where Python's import path
will find it. Read up on using the standard library 'distutils'
mechanism for this; it involves writing a 'setup.py' file to define
the parameters for installation of your package.
I would like to avoid making 'dummy' into a package as well.

Yes. The top-level directory is used for containing a number of files,
including 'setup.py', that should not be part of the installed
package.
Problem solved. I added 'dummy' to the PYTHONPATH. (Do I really have
to do that for every project I create?) Anyway, it works the way I'd
like it to now.

I hope the above makes it clearer what I prefer for this situation.

--
\ "True greatness is measured by how much freedom you give to |
`\ others, not by how much you can coerce others to do what you |
_o__) want." --Larry Wall |
Ben Finney

--
http://mail.python.org/mailman/listinfo/python-list
Nov 21 '07 #4
On 21 Nov., 05:34, Ben Finney <bignose+hates-s...@benfinney.id.au>
wrote:
>
I hope the above makes it clearer what I prefer for this situation.
It does. You've been most helpful - thanks a lot! I'll bookmark this
thread and keep it under my pillow :-)
Nov 21 '07 #5
On Nov 21, 7:05 am, "Sergio Correia" <sergio.corr...@gmail.comwrote:
As a side note, I find much easier to drop a PTH file than messing
with pythonpath. If you are not familiar with PTH files, what I do is
this

1) Go to "C:\Program Files\Python25\Lib\site-packages" or whatever is
appropiate in your case.
2) Create a text file, name it something like "MyProjects.PTH" (note
the extension!)
3) in the file, just write the path of the folder that contains all
your projects (in my case, C:/docs/python)

The idea is to keep the main python installation separated from the
modules you are currently developing. Your python installation goes to
"program files/python" or "bin/python", and your personal projects go
somewhere else (usually inside your 'user' folder). This smooths many
things, like working with different versions of a package you are
developing.
Hi,

If I understand you correctly, you have the following directory
organisation
[Python installation]/site-packages/[MyProject.pth pointing to /home/
user/python-dev/]

And then you do your development in python-dev. But how do you manage
multiple development branches of the same program ?

My directory structure looks like this:
python-dev:
+ prog-branch1/
| + foo
| | + foo.py
| | + tests
| | + test_foo.py
| + bar
| + bar.py
| + tests
| + test_bar.py
+ prog-branch2/
+ foo
| + foo.py
| + tests
| + test_foo.py
+ bar
+ bar.py
+ tests
+ test_bar.py

bar/bar.py needs to import symbols from foo/foo.py . And bar/tests/
test_bar.py needs some symbols from both bar/bar.py and foo/foo.py

I don't understand how having python-dev in the .pth file solves the
problem.

In my case, I make all my imports relative to the root of my project:
bar/bar.py:
from foo.foo import some_foo

bar/tests/test_bar.py
from foo.foo import some_foo
from foo.bar import some_bar

The way I managed to make it work is by extending sys.path but I would
be happy to find a better solution:

bar/bar.py is actually:
import os, sys
sys.path.append( '..')

from foo.foo import some_foo

and bar/tests/test_bar.py is actually:
import os, sys
sys.path.append( os.path.join('..','..') )

from foo.foo import some_foo
from foo.foo import some_bar

What is not nice with this method is that every runnable test script
must extend unconditionally the sys.path . That creates some clutter.

Philippe
Nov 24 '07 #6
On 2007-11-24, BlueBird <ph**@freehackers.orgwrote:
On Nov 21, 7:05 am, "Sergio Correia" <sergio.corr...@gmail.comwrote:
And then you do your development in python-dev. But how do you manage
multiple development branches of the same program ?
If you are using SVN, you may want to check out 'combinator' by Divmod
(divmod.org). That tool manages SVN branches, and also switches Python when you
switch branches. Very handy.

Albert
Nov 26 '07 #7
Bluebird:

If you are using python 2.5, relative imports are no longer an issue:
http://docs.python.org/whatsnew/pep-328.html

That problem solved, what you sometimes want is to change the version
of your package. I just change the text in the PTH file, to point to
another version, and voilá (no need to move files around or to mess
with the code in the package itself). Probably you can write a script
that changes the PTH file from python itself.

Albert:

Thanks, that looks useful.
Nov 26 '07 #8
On Nov 20, 4:09 pm, Jens <j3n...@gmail.comwrote:
>
1) Should I put my unittests in a subdirectory? Does the subdirectory
have to be a package?
As others have suggested, this is a good way to organise your tests.
To avoid problems with the import path, look at nosetests [1]. This
allows you to run::

nosetests
main_package_name.subpackage1.test.test1:TestSomeC lass.test_func

to run the test_func function in the TestSomeClass test case.

Or alternatively::

nosetests main_package_name.subpackage1

to run all tests contained in the main_package_name.subpackage1
module.

This allows you to remove all of your 'if __name__ ==
"__main__":unittest.main()' code, as well as gets rid of the problem
with relative imports (which others solved using .pth files).

The only downside for me is that running nosetests takes at least half
a second (eg: import nose), compared to making the tests runnable
which can be _very_ quick. For test driven development it's nice to
have tests really really quick.

One gotcha; if you're converting from a 'if __name__ ==
"__main__":...' system; nosetests by default ignores files that are
executable (so you'll need to chmod -x main_package_name/subpackage1/
test/test1.py).
[1] nose: a discovery-based unittest extension - (http://
somethingaboutorange.com/mrl/projects/nose/).

Nov 28 '07 #9
On Nov 26, 5:07 pm, "Sergio Correia" <sergio.corr...@gmail.comwrote:
Bluebird:

If you are using python 2.5, relative imports are no longer an
issue:http://docs.python.org/whatsnew/pep-328.html
It does not solve my problem, or I missed something:

User@Phil_vaio /cygdrive/d/work/work/vy-dev/foo $ find . -name '*.py'
../bar/bar.py
../bar/tests/test_bar.py
../bar/tests/test_bar1.py
../bar/tests/__init__.py
../bar/__init__.py
../foo/foo.py
../foo/tests/__init__.py
../foo/__init__.py
../__init__.py

d:\work\work\vy-dev\foo>C:\Python25\python.exe bar/tests/test_bar.py
d:\work\work\vy-dev\foo
['d:\\work\\work\\vy-dev\\foo\\bar\\tests', 'C:\\Python25\\lib\\site-
packages\\h
gsvn-0.1.4-py2.5.egg', 'C:\\WINDOWS\\system32\\python25.zip', 'C:\
\Python25\\DLL
s', 'C:\\Python25\\lib', 'C:\\Python25\\lib\\plat-win', 'C:\\Python25\
\lib\\lib-
tk', 'C:\\Python25', 'C:\\Python25\\lib\\site-packages', 'C:\\Python25\
\lib\\sit
e-packages\\win32', 'C:\\Python25\\lib\\site-packages\\win32\\lib', 'C:
\\Python2
5\\lib\\site-packages\\Pythonwin', 'C:\\Python25\\lib\\site-packages\
\wx-2.8-msw
-ansi']
Traceback (most recent call last):
File "bar/tests/test_bar.py", line 8, in <module>
from ..bar import bar_f
ValueError: Attempted relative import in non-package

Dec 2 '07 #10
On Dec 2, 4:27 pm, BlueBird <p...@freehackers.orgwrote:
On Nov 26, 5:07 pm, "Sergio Correia" <sergio.corr...@gmail.comwrote:
Bluebird:
If you are using python 2.5, relative imports are no longer an
issue:http://docs.python.org/whatsnew/pep-328.html

It does not solve my problem, or I missed something:
Any idea what could be wrong in my code (in the previous message). Or
did I misunderstand relative import seriously ?

Philippe
Dec 7 '07 #11
BlueBird <ph**@freehackers.orgwrote:
On Dec 2, 4:27 pm, BlueBird <p...@freehackers.orgwrote:
>On Nov 26, 5:07 pm, "Sergio Correia" <sergio.corr...@gmail.comwrote:
>Bluebird:
If you are using python 2.5, relative imports are no longer an
issue:http://docs.python.org/whatsnew/pep-328.html

It does not solve my problem, or I missed something:

Any idea what could be wrong in my code (in the previous message). Or
did I misunderstand relative import seriously ?
Does this give you an idea?

0:tolot:/tmpmkdir t1
0:tolot:/tmptouch t1/__init__.py
0:tolot:/tmpecho print __name__ >t1/test.py
0:tolot:/tmppython t1/test.py
__main__
0:tolot:/tmppython -c "import t1.test"
t1.test

Marc
Dec 7 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Scott Abel | last post by:
For immediate release: The Rockley Group Content Management Workshop Series Coming to Atlanta, Seattle, Vancouver, Chicago, Washington, DC, Toronto, and Research Triangle Park Learn more:...
7
by: x muzuo | last post by:
Hi guys, I have got a prob of javascript form validation which just doesnt work with my ASP code. Can any one help me out please. Here is the code: {////<<head> <title>IIBO Submit Page</title>...
3
by: Sai Kit Tong | last post by:
I posted for help on legacy code interface 2 days ago. Probably I didn't make it clear in my original mail. I got a couple of answers but none of them address my issues directly (See attached...
1
by: andrewcw | last post by:
I have used System.Management in the past to extract and walk thru drives & check their type. I can also do it with COM's FileSystemObject. Here I am trying to just use the Management Object...
3
by: John Morgan | last post by:
I am using Visual Studio with code behind to build files within an application. The name of the project is bsecsSql. Everything works fine on the local computer when I use 'copy project' to copy...
7
by: scorpion53061 | last post by:
I get the following error with the class below it when installing on a NT/98/Me machine. Any suggestions would be helpful. Kind of in a bind. Suspect the issue revolves ...
1
by: Husam | last post by:
Hi EveryBody: Is there any one know how can I use the following code to make windows application project that can start and stop the windows32 service manually?and tell me what the languege this...
5
by: moondaddy | last post by:
I've seen how people have snippets of code stored over by the tool box and then can drag them into a code page. What do you call this and what's a reference on how to do this? Thanks. --...
0
by: Terry Olsen | last post by:
This code works, but I need some advice on how to do things "better". See the "For Each" loop below. I tried to use: 'If mo.Item(WMIProp(i)).GetType.ToString = "System.String" Then to...
7
by: Olegus | last post by:
Hello, in order to perform backup/restore MSSQL database using SMO, one needs to reference several namespaces in a backup class : using Microsoft.SqlServer.Management.Common; using...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.