473,246 Members | 1,375 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,246 software developers and data experts.

What is a good way of having several versions of a python moduleinstalled in parallell?

Hi!

I write, use and reuse a lot of small python programs for variuos purposes in my work. These use a growing number of utility modules that I'm continuously developing and adding to as new functionality is needed. Sometimes I discover earlier design mistakes in these modules, and rather than keeping old garbage I often rewrite the parts that are unsatisfactory. This often breaks backwards compatibility, and since I don't feel like updating all the code that relies on the old (functional but flawed) modules, I'm left with a hack library that depends on halting versions of my utility modules. The way I do it now is that I update the programs as needed when I need them, but this approach makes me feel a bit queasy. It seems to me like I'm thinking about this in the wrong way.

Does anyone else recognize this situation in general? How do you handle it?

I have a feeling it should be possible to have multiple versions of the modules installed simultaneously, and maybe do something like this:

mymodule/
+ mymodule-1.1.3/
+ mymodule-1.1.0/
+ mymodule-0.9.5/
- __init__.py

and having some kind of magic in __init__.py that let's the programmer choose version after import:

import mymodule
mymodule.require_version("1.1.3")

Is this a good way of thinking about it? What would be an efficient way of implementing it?

Cheers!
/Joel Hedlund
Sep 25 '07 #1
6 1263
Joel Hedlund wrote:
Hi!

I write, use and reuse a lot of small python programs for variuos purposes
in my work. These use a growing number of utility modules that I'm
continuously developing and adding to as new functionality is needed.
Sometimes I discover earlier design mistakes in these modules, and rather
than keeping old garbage I often rewrite the parts that are
unsatisfactory. This often breaks backwards compatibility, and since I
don't feel like updating all the code that relies on the old (functional
but flawed) modules, I'm left with a hack library that depends on halting
versions of my utility modules. The way I do it now is that I update the
programs as needed when I need them, but this approach makes me feel a bit
queasy. It seems to me like I'm thinking about this in the wrong way.

Does anyone else recognize this situation in general? How do you handle
it?

I have a feeling it should be possible to have multiple versions of the
modules installed simultaneously, and maybe do something like this:

mymodule/
+ mymodule-1.1.3/
+ mymodule-1.1.0/
+ mymodule-0.9.5/
- __init__.py

and having some kind of magic in __init__.py that let's the programmer
choose version after import:

import mymodule
mymodule.require_version("1.1.3")

Is this a good way of thinking about it? What would be an efficient way of
implementing it?
Use setuptools. It can exactly do that - install different versions parallel
as eggs, and with a pre-import require-statment you require the desired
one.

Diez
Sep 25 '07 #2
Diez B. Roggisch wrote:
Joel Hedlund wrote:
>Hi!

I write, use and reuse a lot of small python programs for variuos purposes
in my work. These use a growing number of utility modules that I'm
continuously developing and adding to as new functionality is needed.
Sometimes I discover earlier design mistakes in these modules, and rather
than keeping old garbage I often rewrite the parts that are
unsatisfactory. This often breaks backwards compatibility, and since I
don't feel like updating all the code that relies on the old (functional
but flawed) modules, I'm left with a hack library that depends on halting
versions of my utility modules. The way I do it now is that I update the
programs as needed when I need them, but this approach makes me feel a bit
queasy. It seems to me like I'm thinking about this in the wrong way.

Does anyone else recognize this situation in general? How do you handle
it?

I have a feeling it should be possible to have multiple versions of the
modules installed simultaneously, and maybe do something like this:

mymodule/
+ mymodule-1.1.3/
+ mymodule-1.1.0/
+ mymodule-0.9.5/
- __init__.py

and having some kind of magic in __init__.py that let's the programmer
choose version after import:

import mymodule
mymodule.require_version("1.1.3")

Is this a good way of thinking about it? What would be an efficient way of
implementing it?

Use setuptools. It can exactly do that - install different versions parallel
as eggs, and with a pre-import require-statment you require the desired
one.

Diez
Of course a much simpler, less formal solution, is to install the
libraries required by a program along with that program in its own
directory. This more or less guarantees you will get out of sync.

Otherwise, three words:

test driven development

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline

Sep 25 '07 #3
First of all, thanks for all the input - it's appreciated.
Otherwise, three words:

test driven development
Do you also do this for all the little stuff, the small hacks you just
whip together to get a particular task done? My impression is that doing
proper unittests adds a lot of time to development, and I'm thinking
that this may be a low return investment for the small programs.

I try to aim for reusability and generalizability also in my smaller
hacks mainly as a safeguard. My reasoning here is that if I mess up
somehow, sooner or later I'll notice, and then I have a chance of making
realistic damage assessments. But even so, I must admit that I tend to
do quite little testing for these small projects... Maybe I should be
rethinking this?

Cheers!
/Joel
Sep 25 '07 #4
First of all, thanks for all the input - it's appreciated.
Otherwise, three words:

test driven development
Do you also do this for all the little stuff, the small hacks you just
whip together to get a particular task done? My impression is that doing
proper unittests adds a lot of time to development, and I'm thinking
that this may be a low return investment for the small programs.

I try to aim for reusability and generalizability also in my smaller
hacks mainly as a safeguard. My reasoning here is that if I mess up
somehow, sooner or later I'll notice, and then I have a chance of making
realistic damage assessments. But even so, I must admit that I tend to
do quite little testing for these small projects... Maybe I should be
rethinking this?

Cheers!
/Joel
Sep 25 '07 #5
Joel Hedlund <jo**********@gmail.comwrites:
Do you also do [test-driven development] for all the little stuff,
the small hacks you just whip together to get a particular task
done? My impression is that doing proper unittests adds a lot of
time to development, and I'm thinking that this may be a low return
investment for the small programs.
My impression is that failing to have reliable code adds a lot of time
to debugging and maintenance, and it is far cheaper to have tested and
flushed out the obvious bugs early in the development process.
I try to aim for reusability and generalizability also in my smaller
hacks mainly as a safeguard.
In which case, you will be maintaining that code beyond the initial
use for which you thought it up. Maintaining code without unit tests
is far more costly than maintaining code with tests, because you have
no regression test suite: you are prone to chasing down bugs that you
though you'd fixed in an earlier version of the code, but can't figure
out when you broke it again. This time is entirely wasted.

Instead, in code that has unit tests, a bug found means another unit
test to be added (to find and demonstrate that bug). This is work you
must do anyway, to be sure that you can actually reproduce the bug;
test-driven development merely means that you take that test case and
*keep it* in your unit test. Then, once you're assured that you will
find the bug again any time it reappears, go ahead and fix it.
My reasoning here is that if I mess up somehow, sooner or later I'll
notice
With test-driven development I get much closer to the "sooner" end of
that, and much more reliably.
Maybe I should be rethinking this?
That's my opinion, yes.

--
\ "My mother was like a sister to me, only we didn't have sex |
`\ quite so often." -- Emo Philips |
_o__) |
Ben Finney
Sep 26 '07 #6
Diez B. Roggisch wrote:
Sounds good to me. IMHO there are two ways one gathers tests:

- concrete bugs appear, and one writes a test that reproduces the bug &
eventually after the fix runs smoothly

- new features are planned/implemented, and the tests accompany them right
from the start, to allow .. .well, to test them :)

I always found it difficult to "just think" of new tests. Of course if you
_start_ with TDD, point two is applied right from the start and should
apply.
an approach that works for me is to start by adding "sanity checks";
that is, straightforward test code that simply imports all modules,
instantiates objects, calls the important methods with common argument
types/counts, etc, to make sure that the library isn't entirely braindead.

this has a bunch of advantages:

- it gets you started with very little effort (especially if you use
doctest; just tinker a little at the interactive prompt, and you
have a first version)
- it gives you a basic structure which makes it easier to add more
detailed tests
- it gives you immediate design feedback -- if it's difficult to
think of a even a simple test, is the design really optimal?
- it quickly catches build and installation issues during future
development (including refactoring).

and probably a few more things that I cannot think of right now.

</F>

Sep 26 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

220
by: Brandon J. Van Every | last post by:
What's better about Ruby than Python? I'm sure there's something. What is it? This is not a troll. I'm language shopping and I want people's answers. I don't know beans about Ruby or have...
226
by: Stephen C. Waterbury | last post by:
This seems like it ought to work, according to the description of reduce(), but it doesn't. Is this a bug, or am I missing something? Python 2.3.2 (#1, Oct 20 2003, 01:04:35) on linux2 Type...
28
by: David MacQuigg | last post by:
I'm concerned that with all the focus on obj$func binding, &closures, and other not-so-pretty details of Prothon, that we are missing what is really good - the simplification of classes. There are...
92
by: Reed L. O'Brien | last post by:
I see rotor was removed for 2.4 and the docs say use an AES module provided separately... Is there a standard module that works alike or an AES module that works alike but with better encryption?...
137
by: Philippe C. Martin | last post by:
I apologize in advance for launching this post but I might get enlightment somehow (PS: I am _very_ agnostic ;-). - 1) I do not consider my intelligence/education above average - 2) I am very...
125
by: Sarah Tanembaum | last post by:
Beside its an opensource and supported by community, what's the fundamental differences between PostgreSQL and those high-price commercial database (and some are bloated such as Oracle) from...
121
by: typingcat | last post by:
First of all, I'm an Asian and I need to input Japanese, Korean and so on. I've tried many PHP IDEs today, but almost non of them supported Unicode (UTF-8) file. I've found that the only Unicode...
3
by: Johan Johansson (Sweden) | last post by:
I wonder why it is that no more than 10 clients can access a webservice at a time, and how to increase that number. The eleventh client get a Http 403.9 (Access forbidden : To many users are...
66
by: John | last post by:
Hi What are the advantages actually achieved of managed code? I am not talking of theory but in reality. Thanks Regards
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.