473,385 Members | 2,028 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Package organization

I have a question about how to organize modules in a package. I think
there is something fundamental that I am missing. Say I'm creating a
package,
GreatPackage, which has three sub-packages, myUtilities, GoodStuff,
MoreGoodStuff.

The directories look like:

GreatPackage
|
|-- myUtilities
|
|-- GoodStuff
|
|-- MoreGoodStuff

What is the right way for modules in GoodStuff and MoreGoodStuff to
refer to modules etc... in myUtilities? I'd like to set PYTHONPATH to
just point to the directory containing GreatPackage. But, if put
something like:

from GreatPackage.myUtilities import aUtilityThingy

in a module in GoodStuff, then the whole package gets loaded,
including MoreGoodStuff. That's okay except when I'm developing and
MoreGoodStuff is broken.

What are guidelines for developing/deploying packages which contain
pacakges?

thanks,
-robert
Jul 18 '05 #1
4 4165
On Tue, 18 Nov 2003 16:38:15 -0800, Robert Ferrell wrote:
I have a question about how to organize modules in a package. I think
there is something fundamental that I am missing. Say I'm creating a
package,
GreatPackage, which has three sub-packages, myUtilities, GoodStuff,
MoreGoodStuff.

The directories look like:

GreatPackage
|
|-- myUtilities
|
|-- GoodStuff
|
|-- MoreGoodStuff

What is the right way for modules in GoodStuff and MoreGoodStuff to
refer to modules etc... in myUtilities?
There's a PEP that recommends that when importing things you refer to them
explicitly; i.e. inside GoodStuff you'd say

import GreatPackage.myUtilities

rather than just

import myUtilities

I've only benefitted from doing it that way myself in large apps that have
had many packages with sub packages. It keeps you safe from module name
clashes at different levels of the hierarchy (not that I think they're a
sensible thing to have). By name clashes I mean if you have modules
set up like this:

lib
shoe.lib
shoe.fruit

then you can't get at the top level lib module from inside shoe.fruit.
from GreatPackage.myUtilities import aUtilityThingy

in a module in GoodStuff, then the whole package gets loaded,
including MoreGoodStuff.
MoreGoodStuff won't get loaded if you import GreatPackage, unless
GreatPackage/__init__.py causes it to be loaded. Obviously, if something
in myUtilities imports something from MoreGoodStuff then python will
attempt to load MoreGoodStuff when importing myUtilities.

Also, because you're importing something from inside a module using the
"from" style imports you're risking running into circular reference
problems. In my experience from imports are best avoided, unless you're
importing a module from a package.
What are guidelines for developing/deploying packages which contain
pacakges?


For deploying, have you looked at distutils?

-- Graham
Jul 18 '05 #2
Graham Ashton <ga*****@cmedltd.com> wrote in message news:<pa****************************@cmedltd.com>. ..
On Tue, 18 Nov 2003 16:38:15 -0800, Robert Ferrell wrote:
I have a question about how to organize modules in a package. I think
there is something fundamental that I am missing. Say I'm creating a
package,
GreatPackage, which has three sub-packages, myUtilities, GoodStuff,
MoreGoodStuff.

The directories look like:

GreatPackage
|
|-- myUtilities
|
|-- GoodStuff
|
|-- MoreGoodStuff

What is the right way for modules in GoodStuff and MoreGoodStuff to
refer to modules etc... in myUtilities?
There's a PEP that recommends that when importing things you refer to them
explicitly; i.e. inside GoodStuff you'd say

import GreatPackage.myUtilities

rather than just

import myUtilities

I've only benefitted from doing it that way myself in large apps that have
had many packages with sub packages. It keeps you safe from module name
clashes at different levels of the hierarchy (not that I think they're a
sensible thing to have). By name clashes I mean if you have modules
set up like this:

lib
shoe.lib
shoe.fruit

then you can't get at the top level lib module from inside shoe.fruit.
from GreatPackage.myUtilities import aUtilityThingy

in a module in GoodStuff, then the whole package gets loaded,
including MoreGoodStuff.


MoreGoodStuff won't get loaded if you import GreatPackage, unless
GreatPackage/__init__.py causes it to be loaded. Obviously, if something
in myUtilities imports something from MoreGoodStuff then python will
attempt to load MoreGoodStuff when importing myUtilities.

Ah Ha! Somehow I'd convinced myself that GreatPackage/__init__.py had
to import all the submodules. So that's what I was doing. That's
obviously wrong. Now that I've corrected that mistake I see that what
I was having trouble with is not an issue.
Also, because you're importing something from inside a module using the
"from" style imports you're risking running into circular reference
problems. In my experience from imports are best avoided, unless you're
importing a module from a package.


I don't understand this comment. How does "from" style importing
increase the risk of circular imports?

Thanks for the clarification!
-robert
Jul 18 '05 #3
On Wed, 19 Nov 2003 14:23:33 -0800, Robert Ferrell wrote:
Graham Ashton <ga*****@cmedltd.com> wrote in message news:<pa****************************@cmedltd.com>. ..

Also, because you're importing something from inside a module using the
"from" style imports you're risking running into circular reference
problems. In my experience from imports are best avoided, unless you're
importing a module from a package.


I don't understand this comment. How does "from" style importing
increase the risk of circular imports?


Imagine you've got two modules, spam and eggs.

When you import a module it's top level statements get evaluated. Imagine
that spam imports something from eggs. The interpreter (whilst importing
spam) pauses to import eggs. What if eggs tries to import something from
spam?

You might think, ah, that's a circular import. But it isn't necessarily.
You see, when it starts importing spam the interpreter stores an empty
module object in sys.modules and fills in the details once it's
finished importing spam. Consequently when it's trying to import spam
again from inside eggs it finds a module reference in sys.modules for spam
and can carry on happily.

Things only break on you if eggs is trying to refer to something inside
spam that the original import of spam hasn't filled in yet.

I'll try and give you an example. Here's the spam module:

import eggs

def count_spam():
return 1

And the eggs module:

from spam import count_spam

def count_eggs_and_spam():
return count_spam() + 1

If your main script then tries to import spam it'll blow up:

% python
Python 2.3.2 (#1, Dec 8 2003, 10:49:17)
[GCC 3.3.2 20031022 (Gentoo Linux 3.3.2-r3, propolice)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import spam

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "spam.py", line 1, in ?
import eggs
File "eggs.py", line 1, in ?
from spam import count_spam
ImportError: cannot import name count_spam

It's because the spam module in sys.modules hasn't been filled in yet, as
the interpreter put off importing the contents of spam until the "import
eggs" statement returned. Replace the from import with a normal import and
the problem goes away. Here's a replacement eggs module:

% cat eggs.py
import spam

def count_eggs_and_spam():
return spam.count_spam() + 1

It's no longer an issue as the spam.count_spam() function doesn't get
evaluated until the count_eggs_and_spam() function is called, which is
long after both the modules have been fully imported.

Stay away from from, it's dangerous. And in my opinion you get less clear
code.

Incidentally, I learnt the nitty gritty of this from Martelli's
Python in a Nutshell. Recommended.

-- Graham
Jul 18 '05 #4
Robert Ferrell wrote:
Also, because you're importing something from inside a module using the
"from" style imports you're risking running into circular reference
problems. In my experience from imports are best avoided, unless you're
importing a module from a package.


I don't understand this comment. How does "from" style importing
increase the risk of circular imports?


python has no problem doing circular (or recursive) imports, but you're likely
to run into problems if you're doing recursive imports and you're referring to
stuff from one module on the *module-level* in another module. from-import
does exactly that.

for a diskussion, see the section on "recursive imports" on this page:

http://www.effbot.org/zone/import-confusion.htm

</F>


Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: Amardeep Verma | last post by:
Hi, I would like to determine the Country and Organization from a given IP. I have to write a JSP, which when given the IP address gives the country and the organization to which the IP belongs....
4
by: Thomas Lotze | last post by:
Hi, I've two questions concerning organizing and naming things when writing a Python package. - Naming of classes: I'm writing a library that reads PDF files. I have a data structure that...
6
by: twoeyedhuman1111 | last post by:
I was wondering, what group standardized c++?
1
by: Steven T. Hatton | last post by:
I think Danny was one cup of coffee shy of full consciousness when he wrote this, but the gist of it makes sens to me: "C++ Project Organization Guidelines Last updated May 26, 2005....
3
by: david.kao | last post by:
Hi All: I tried to uninstall a runtime security package from a user's machine. Based on document I have read, all I need to do is right click on MSI file, and choose uninstall from menu. After I...
11
by: fortepianissimo | last post by:
Say I have the following package organization in a system I'm developing: A |----B |----C |----D I have a module, say 'foo', that both package D and B require. What is the best practice in...
15
by: GS | last post by:
Hello, I am looking for any opensource package for "appointment system" for doctors, please let me know. This is just o take Name/Phone/time, incase if there is not any opensource, then is it...
3
by: someone | last post by:
Hi all, I am using Sql Server 2000. I need to create a very simple app which when clicked would fire the execution of a dts package on the sql server. The app itself would sit on the client...
3
by: sulekha | last post by:
Hi all, I was reading the book "Write Great Code vol 1" by Ryndall hyde in this book chapter 11 is named as Memory architecture & Organization. in this chapter there is a section named "Run time...
1
by: mypetslug | last post by:
Hi, I'm sorry if this has been asked before, but I can't seem to find an answer to this anywhere and so far, trial and error hasn't gotten me far either. Using python 2.4, I've created a...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.