473,595 Members | 2,638 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Module Structure/Import Design Problem

Hi,

I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.

My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.

An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...

Which would be used a bit like this:
>>type_a = any_type_instan ce.get_type("Ty peA")
special_typ e = type_a.get_type ("SpecialTypeA" )

Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?

Cheers

- Rafe
Nov 20 '08 #1
13 2278
Rafe <ra*******@gmai l.comwrites:
Hi,

I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.

My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.

An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...

Which would be used a bit like this:
>>>type_a = any_type_instan ce.get_type("Ty peA")
special_ty pe = type_a.get_type ("SpecialTypeA" )


Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?
It's not very clear what your problem is. I guess your factory
functions are defined in baselib.py whereas types.py and special.py
import baselib, therefore you don't know how to make the factory
function aware of the types defined in special.py and types.py.

You can use cyclic import in many cases.

Or (better IMHO) you can make types register themselves with the factory
function (in which case it would have some state so it would make more
sense to make it a factory object).

--
Arnaud
Nov 20 '08 #2
On Nov 20, 2:06*pm, Arnaud Delobelle <arno...@google mail.comwrote:
Rafe <rafesa...@gmai l.comwrites:
Hi,
I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.
My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.
An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
*- baselib.py * with *BaseClass
*- types.py * with *TypeA, ...
*- special.py * with *SpecialTypeA, ...
Which would be used a bit like this:
>>type_a = any_type_instan ce.get_type("Ty peA")
special_typ e = type_a.get_type ("SpecialTypeA" )
Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?

It's not very clear what your problem is. *I guess your factory
functions are defined in baselib.py whereas types.py and special.py
import baselib, therefore you don't know how to make the factory
function aware of the types defined in special.py and types.py.

You can use cyclic import in many cases.

Or (better IMHO) you can make types register themselves with the factory
function (in which case it would have some state so it would make more
sense to make it a factory object).

--
Arnaud
hi Arnaud,

You got my problem right, sorry it wasn't more clear.

Can you elaborate on what you mean by 'register' with the factory
function?

Also...holy cr@p, I got a clean import working! I swear I tried that
before with unhappy results. I'll carefully try this in my real code.

Is this the right way to impliment the imports?....

baselib.py
[1] class BaseClass(objec t):
[2] def factory(self):
[3] import typelib # <-- import inside function
[4] return typelib.TypeA()

typelib.py
[1] import baselib # <-- module level import
[2]
[3] class TypeA(baselib.B aseClass):
[4] def __init__(self):
[5] print "TypeA : __init__()"
>>import typelib
type = typelib.TypeA()
TypeA : __init__()
>>another_typ e = type.factory()
TypeA : __init__()
>>another_typ e
<typelib.Type A object at 0x00B45F10>
I am curious (not directed at Arnaud), why not have an 'include'-like
import for special cases in python (or do I not understand includes
either?)

Thanks!

- Rafe
Nov 20 '08 #3
Rafe wrote:
Hi,

I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.

My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.

An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...

Which would be used a bit like this:
>>>type_a = any_type_instan ce.get_type("Ty peA")
special_ty pe = type_a.get_type ("SpecialTypeA" )


Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?

I'm not an expert, I even don't fully understand your problem,
but having struggled with imports in the past,
I've a solution now, which seems to work quit well.

cheers,
Stef
Cheers

- Rafe
--
http://mail.python.org/mailman/listinfo/python-list
Nov 20 '08 #4
Rafe wrote:
Hi,

I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.

My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.

An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...

Which would be used a bit like this:
>>>type_a = any_type_instan ce.get_type("Ty peA")
special_ty pe = type_a.get_type ("SpecialTypeA" )


Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?
Well a simple way to do this is to observe that even when a base class's
method is inherited by an instance of a subclass, when the method is
called the type of "self" is the subclass. And you can call the
subclass's type to create an instance. Perhaps the following code would
make it more obvious:

$ cat baseclass.py
class Base(object):
def factory(self, arg):
return type(self)(arg)

sholden@lifeboy /c/Users/sholden/Projects/Python
$ cat subclass.py
from baseclass import Base

class sub(Base):
def __init__(self, arg):
print "Creating a sub with arg", arg

s = sub("Manual")

thing = s.factory("Auto ")
print type(thing)
sholden@lifeboy /c/Users/sholden/Projects/Python
$ python subclass.py
Creating a sub with arg Manual
Creating a sub with arg Auto
<class '__main__.sub'>

Hope this helps.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Nov 20 '08 #5
Stef Mientki wrote:
Rafe wrote:
>Hi,

I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.

My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.

An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...

Which would be used a bit like this:
>>>>type_a = any_type_instan ce.get_type("Ty peA")
special_typ e = type_a.get_type ("SpecialTypeA" )
>


Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?

I'm not an expert, I even don't fully understand your problem,
but having struggled with imports in the past,
I've a solution now, which seems to work quit well.
That's not very helpful, is it? Were you planning to keep the solution
secret?

regards
steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Nov 20 '08 #6
>>>
I'm not an expert, I even don't fully understand your problem,
but having struggled with imports in the past,
I've a solution now, which seems to work quit well.

That's not very helpful, is it? Were you planning to keep the solution
secret?
sorry slip of the keyboard ;-)
http://mientki.ruhosting.nl/data_www...importing.html
cheers,
Stef

Nov 20 '08 #7
On Nov 21, 1:39*am, Steve Holden <st...@holdenwe b.comwrote:
Rafe wrote:
Hi,
I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.
My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.
An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
*- baselib.py * with *BaseClass
*- types.py * with *TypeA, ...
*- special.py * with *SpecialTypeA, ...
Which would be used a bit like this:
>>type_a = any_type_instan ce.get_type("Ty peA")
special_typ e = type_a.get_type ("SpecialTypeA" )
Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?

Well a simple way to do this is to observe that even when a base class's
method is inherited by an instance of a subclass, when the method is
called the type of "self" is the subclass. And you can call the
subclass's type to create an instance. Perhaps the following *code would
make it more obvious:

$ cat baseclass.py
class Base(object):
* * def factory(self, arg):
* * * * return type(self)(arg)

sholden@lifeboy /c/Users/sholden/Projects/Python
$ cat subclass.py
from baseclass import Base

class sub(Base):
* def __init__(self, arg):
* * print "Creating a sub with arg", arg

s = sub("Manual")

thing = s.factory("Auto ")
print type(thing)

sholden@lifeboy /c/Users/sholden/Projects/Python
$ python subclass.py
Creating a sub with arg Manual
Creating a sub with arg Auto
<class '__main__.sub'>

Hope this helps.

regards
*Steve
--
Steve Holden * * * *+1 571 484 6266 * +1 800 494 3119
Holden Web LLC * * * * * * *http://www.holdenweb.com/
Hi Steve,

Correct me if I have this wrong, but the problem with your solution is
that it only creates a new instance of the same class, type(self),
while I need to return any number of different possibilities.

I thought about getting the module from self...
>>class base(object):
def factory(self, type):
module = sys.modules[self.__class__. __module__]
return getattr(module, type)
....but my baseclass is used from several modules so this would be
inaccurate for me (the factory method only uses my 'types' module, so
a hard import works)

I'm still wondering what Arnaud meant by "make types register
themselves with the factory function"

- Rafe
Nov 21 '08 #8
On Nov 21, 2:36*am, Stef Mientki <stef.mien...@g mail.comwrote:
I'm not an expert, I even don't fully understand your problem,
but having struggled with imports in the past,
I've a solution now, which seems to work quit well.
That's not very helpful, is it? Were you planning to keep the solution
secret?

sorry slip of the keyboard ;-)http://mientki.ruhosting.nl/data_www...importing.html
cheers,
Stef
I really don't understand what you are trying to accomplish in your
article.

I strongly disagree with this statement...
"A second demand is that every module should be able to act as a main
file by running it's main section."
....I am finding the best programs have only one entry point or
interface (though some libraries can be useful from outside the
package.) Being able to run any file in a package seems like it
creates a confusing user/developer experience. What kind of problem
makes this solution applicable?

Next, you say...
"...recursi ve searches for all subdirectories and adds them to the
Python Path."
....it seems like you add every module in your packages directly to the
sys.path. Doesn't this destroy the package name-spaces? For example, I
have a module called 'types' in my package if I add that to the python
path, 'import types' still returns the built-in 'types' module.
Wouldn't this collision be confusing? Regardless, isn't putting the
package in the right place enough? Please explain.
Cheers,

- Rafe
Nov 21 '08 #9
Rafe wrote:
On Nov 21, 1:39 am, Steve Holden <st...@holdenwe b.comwrote:
>Rafe wrote:
>>Hi,
I am in a situation where I feel I am being forced to abandon a clean
module structure in favor of a large single module. If anyone can save
my sanity here I would be forever grateful.
My problem is that classes in several modules share a common base
class which needs to implement a factory method to return instances of
these same classes.
An example to help illustrate what I mean:
Lets say I have the following modules with the listed classes:
- baselib.py with BaseClass
- types.py with TypeA, ...
- special.py with SpecialTypeA, ...
Which would be used a bit like this:
>type_a = any_type_instan ce.get_type("Ty peA")
>special_ty pe = type_a.get_type ("SpecialTypeA" )
Again, I can get around this by dumping everything in to one module,
but it muddies the organization of the package a bit. This seems like
a problem that would come up a lot. Are there any design paradigms I
can apply here?
Well a simple way to do this is to observe that even when a base class's
method is inherited by an instance of a subclass, when the method is
called the type of "self" is the subclass. And you can call the
subclass's type to create an instance. Perhaps the following code would
make it more obvious:

$ cat baseclass.py
class Base(object):
def factory(self, arg):
return type(self)(arg)

sholden@lifebo y /c/Users/sholden/Projects/Python
$ cat subclass.py
from baseclass import Base

class sub(Base):
def __init__(self, arg):
print "Creating a sub with arg", arg

s = sub("Manual")

thing = s.factory("Auto ")
print type(thing)

sholden@lifebo y /c/Users/sholden/Projects/Python
$ python subclass.py
Creating a sub with arg Manual
Creating a sub with arg Auto
<class '__main__.sub'>

Hope this helps.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Hi Steve,

Correct me if I have this wrong, but the problem with your solution is
that it only creates a new instance of the same class, type(self),
while I need to return any number of different possibilities.
In that case you need to pass the type of the new instance you require
as an argument to the factory method, I guess, or something similar. My
example has each subclass returning instances of the subclass that was
used to call the factory method, yes.

If A instances need to be able to get B instances and B instances need
to be able to create B instances then you do indeed have a tricky
problem when it comes to separating your classes into different
modules,. But it's not insoluble!
I thought about getting the module from self...
>>>class base(object):
def factory(self, type):
module = sys.modules[self.__class__. __module__]
return getattr(module, type)

...but my baseclass is used from several modules so this would be
inaccurate for me (the factory method only uses my 'types' module, so
a hard import works)
I am not sure yet if I understand the requirement properly. It seems
from the above code that you always want to create instances of a named
type defined in the same module as the subclass?
I'm still wondering what Arnaud meant by "make types register
themselves with the factory function"
You would have to ask him, but I suspect he means having each subtype
call a method of the base type to add itself to some dictionary so you
can call the factory method with a key that will tell it which subtype
to produce. This is a fairly sensible way to separate definitions form
references.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Nov 21 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
7273
by: Peter L. Buschman | last post by:
I have a package tree laid out like this foo/bar/module/submodule foo/bar/module/submodule/test foo/bar/module/submodule/test/test1.py foo/bar/module/submodule/test/test2.py .... What I want is for test1.py and test2.py to be able to import their parent "submodule" as part of a regression
5
2012
by: dody suria wijaya | last post by:
I found this problem when trying to split a module into two. Here's an example: ============== #Module a (a.py): from b import * class Main: pass ============== ==============
4
2744
by: Edvard Majakari | last post by:
Greetings, fellow Pythonistas! I'm about to create three modules. As an avid TDD fan I'd like to create typical 'use-cases' for each of these modules. One of them is rather large, and I wondered if it would be easy enough to create a code skeleton out of unit test module. Consider the following, though contrived, unit test code snippet: ==========================================================================
9
5009
by: ajikoe | last post by:
Hello, I have two modules (file1.py and file2.py) Is that ok in python (without any weird implication) if my module import each other. I mean in module file1.py there exist command import file2 and in module file2.py there exist command import file1? This is not working in C#. pujo
17
2030
by: Jacob Page | last post by:
I have created what I think may be a useful Python module, but I'd like to share it with the Python community to get feedback, i.e. if it's Pythonic. If it's considered useful by Pythonistas, I'll see about hosting it on Sourceforge or something like that. Is this a good forum for exposing modules to the public, or is there somewhere more-acceptable? Does this newsgroup find attachments acceptable? -- Jacob
0
1021
by: Steven Bethard | last post by:
Ok, so I have a module that is basically a Python wrapper around a big lookup table stored in a text file. The module needs to provide a few functions:: get_stem(word, pos, default=None) stem_exists(word, pos) ... Because there should only ever be one lookup table, I feel like these functions ought to be module globals. That way, you could just do
3
5413
by: Mudcat | last post by:
I have a directory structure that contains different modules that run depending on what the user selects. They are identical in name and structure, but what varies is the content of the functions. They will only need to be run once per execution. Example (directory level): Sys1: A B C
9
1224
by: noamsml | last post by:
Hi, I am new to python and am currently writing my first application. One of the problems I quickly ran into, however, is that python's imports are very different from php/C++ includes in the sense that they completely wrap the imported script in a module object. One of the problems with this was that a plugin system that I am making requires use of objects, classes and the such from the original script. Thus, on one hand, I am hesitant...
6
1782
by: Samuel | last post by:
Hi, Given the following directory structure: --------- |-- Obj.py |-- __init__.py |-- foo | |-- FooTest.py | `-- __init__.py
0
7955
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
7883
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8261
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8379
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
5839
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5418
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3911
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2391
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
1223
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.