Good practice when writing modules...

[r0g]

I'm collecting together a bunch of fairly random useful functions I have
written over the years into a module. Generally speaking is it best to

a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...

b) Include them in each function individually.

Stylistically I like the idea of b) better as it makes for easy
refactoring in the future but is there any drawback to doing it this
way? i.e. does it get repeatedly parsed, use more memory, become
inefficient when several functions that use the same includes are
invoked etc Or does it really not matter?

Performance is probably not important for the majority of these
functions but there's the odd one that may end up in an inner loop some day.

Cheers,

Roger.

[Arnaud Delobelle]

r0g <ai******@technicalbloke.comwrites:

I'm collecting together a bunch of fairly random useful functions I have
written over the years into a module. Generally speaking is it best to

a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...

b) Include them in each function individually.

Stylistically I like the idea of b) better as it makes for easy
refactoring in the future but is there any drawback to doing it this
way? i.e. does it get repeatedly parsed, use more memory, become
inefficient when several functions that use the same includes are
invoked etc Or does it really not matter?

I guess it costs you a dictionary lookup (in sys.modules) and an
assignment every time the import statement is executed. But the module
is not reimported every time.

Performance is probably not important for the majority of these
functions but there's the odd one that may end up in an inner loop some day.

It depends on the function but unless it is a function with special
status (such as testing, or a main() function) I would import stuff at
the top of the module. I makes it clear what the module depends on.

--
Arnaud

[bearophileHUGS]

r0g:

a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...
b) Include them in each function individually.

This is a interesting topic, that requires some care.
Generally I suggest you put them at the top, so they can be found and
seen with less problems (*), it's the standard.
If a function is called only once in a while (like a plotting
function), and to import its module(s) it needs lot of time and memory
(like matplotlib), then you may move the import inside the function
itself, especially if such function isn't speed-critical.
(*)
I generally agree with the PEP8 but regarding this I don't follow it:
I feel free to put more than one of them on the same line:
import os, sys, ...
I generally list the built-in modules first, then the well known one
(pyparsing, numpy, etc), and then my ones or less known ones.

I also put a comment after each import, with the name of the function/
class it is used into:

import foo # used by baz()
import bar # used by spam()

....
def baz(n):
return foo(n) * 10
....
def spam(k):
return bar(k) - 20

Time ago I have read an interesting article that says that comments of
today will often become statements and declaration and tags of
tomorrow, that is stuff that the interpreter/compiler/editor
understands. This means that such annotations of mine may be something
fit to become a syntax of the language. I don't have ideas on how such
syntax can be, if you have suggestions I'm listening.

Finally in modules that contain many different things, and where each
of them uses generally distinct modules, as a compromise I sometimes
write code like this:

import foo # used by baz()

def baz(n):
return foo(n) * 10
import bar # used by spam()

def spam(k):
return bar(k) - 20

My editor has a command that finds and lists me all the global imports
(not indented) of the module.

Bye,
bearophile

[r0g]

be************@lycos.com wrote:

r0g:

>a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...
b) Include them in each function individually.

This is a interesting topic, that requires some care.
Generally I suggest you put them at the top, so they can be found and
seen with less problems (*), it's the standard.
If a function is called only once in a while (like a plotting
function), and to import its module(s) it needs lot of time and memory
(like matplotlib), then you may move the import inside the function
itself, especially if such function isn't speed-critical.
(*)
I generally agree with the PEP8 but regarding this I don't follow it:
I feel free to put more than one of them on the same line:
import os, sys, ...
I generally list the built-in modules first, then the well known one
(pyparsing, numpy, etc), and then my ones or less known ones.

I also put a comment after each import, with the name of the function/
class it is used into:

import foo # used by baz()
import bar # used by spam()

...
def baz(n):
return foo(n) * 10
...
def spam(k):
return bar(k) - 20

Time ago I have read an interesting article that says that comments of
today will often become statements and declaration and tags of
tomorrow, that is stuff that the interpreter/compiler/editor
understands. This means that such annotations of mine may be something
fit to become a syntax of the language. I don't have ideas on how such
syntax can be, if you have suggestions I'm listening.

Finally in modules that contain many different things, and where each
of them uses generally distinct modules, as a compromise I sometimes
write code like this:

import foo # used by baz()

def baz(n):
return foo(n) * 10
import bar # used by spam()

def spam(k):
return bar(k) - 20

My editor has a command that finds and lists me all the global imports
(not indented) of the module.

Bye,
bearophile

Thanks for the suggestions guys. I hadn't thought about putting comments
after the imports, that's a good idea, although I guess you need to be
disciplined in keeping them up to date. All the same, given the
seemingly negligible performance hit and the nature of this particular
module I think I will probably go with includes within the functions
themselves anyway...

The module I am compiling is kind of a scrapbook of snippets for my own
development use, it has no coherent theme and I wouldn't be distributing
it or using the whole thing in a production environment anyway, just
copying the relevant functions into a new module when needed. I'm
thinking having the imports inline might make that process easier when I
do need to do it and once copied I can always move them out of the
functions declarations.

Having said that, having your editor trace these dependencies for you is
an interesting idea too. Which editor are you using? (not trying to
start an editor flame war anyone, shh!). Can this be done from idle or
are you aware of a python script that can do this maybe?

Thanks again,

Roger.

[Scott David Daniels]

be************@lycos.com wrote:

r0g:

>a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...
b) Include them in each function individually.

This is a interesting topic, that requires some care.
Generally I suggest you put them at the top, so they can be found and
seen with less problems (*), it's the standard.
If a function is called ...[rarely] ... you may move the import inside
the function itself, especially if such function isn't speed-critical.

I would amend this advice by adding that you should (in such cases) put
a commented-out import at the top level (so you can look at the top of a
module's source and see what other modules it relies upon.

--Scott David Daniels
Sc***********@Acm.Org

[John Machin]

On Nov 15, 9:08*pm, [e-mail] wrote:

I also put a comment after each import, with the name of the function/
class it is used into:

import foo # used by baz()
import bar # used by spam()

Why bother with the ()? Why bother at all? Surely this rapidly becomes
tedious clutter:
import xyz # used by abc(), def(), ghi(), ...
Do you remember to adjust the comment when a function/class is changed
so that it doesn't use the module any more?

...
def baz(n):
* * return foo(n) * 10
...
def spam(k):
* * return bar(k) - 20

Your modules are callable???

Time ago I have read an interesting article that says that comments of
today will often become statements and declaration and tags of
tomorrow, that is stuff that the interpreter/compiler/editor
understands. This means that such annotations of mine may be something
fit to become a syntax of the language.

I hope not.

I don't have ideas on how such
syntax can be, if you have suggestions I'm listening.

Suggestion: no such syntax!!

[Robert Kern]

r0g wrote:

The module I am compiling is kind of a scrapbook of snippets for my own
development use, it has no coherent theme and I wouldn't be distributing
it or using the whole thing in a production environment anyway, just
copying the relevant functions into a new module when needed. I'm
thinking having the imports inline might make that process easier when I
do need to do it and once copied I can always move them out of the
functions declarations.

This is something of a special case, so I don't think the usual style guides
entirely apply. What I would do is keep the imports outside of the functions,
but put them before the functions that use them. Repeat the imports as
necessary. For example, if f() uses modules foo and bar, while g() uses only foo:

############################

import foo
import bar

def f():
...

############################

import foo

def g():
...

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Robert Kern]

Robert Kern wrote:

r0g wrote:

>The module I am compiling is kind of a scrapbook of snippets for my own
development use, it has no coherent theme and I wouldn't be distributing
it or using the whole thing in a production environment anyway, just
copying the relevant functions into a new module when needed. I'm
thinking having the imports inline might make that process easier when I
do need to do it and once copied I can always move them out of the
functions declarations.

This is something of a special case, so I don't think the usual style
guides entirely apply. What I would do is keep the imports outside of
the functions, but put them before the functions that use them.

Actually, I lie. What I really do is keep a Mercurial repository of all of the
snippets I use in separate files.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[bearophileHUGS]

John Machin:

import foo # used by baz()
import bar # used by spam()

Why bother with the ()?

I code in other language too beside Python, in those languages there
are other things (like templates in D language) beside functions, so
my comment helps me remember that baz() is a function instead of a
template written in lowercase: baz!().

Why bother at all?

If I remove a function/class that uses baz from the module, I can go
at its top and find in a short time what imports to remove. Probably
there are other purposes too.

Do you remember to adjust the comment when a function/class is changed
so that it doesn't use the module any more?

Most of the times yes. Even if I forget doing it 2%-5% of the times,
it's worth it, for me. Notice that I have just shown what coding
practices seem currently good for me, for certain kinds of programs,
etc. Feel free to use your own.

Your modules are callable???

No, they aren't, it's a mistake of mine, sorry.

Time ago I have read an interesting article that says that comments of
today will often become statements and declaration and tags of
tomorrow, that is stuff that the interpreter/compiler/editor
understands. This means that such annotations of mine may be something
fit to become a syntax of the language.

I hope not.

Can you explain why you don't like my idea?

Bye,
bearophile

[John Machin]

On Nov 16, 6:58*pm, [e-mail] wrote:

John Machin:

import foo # used by baz()
import bar # used by spam()

Why bother with the ()?

I code in other language too beside Python, in those languages there
are other things (like templates in D language) beside functions, so
my comment helps me remember that baz() is a function instead of a
template written in lowercase: baz!().

Why bother at all?

If I remove a function/class that uses baz from the module, I can go
at its top and find in a short time what imports to remove.

pychecker tells you about modules that are imported but not used.

Probably
there are other purposes too.

Do you remember to adjust the comment when a function/class is changed
so that it doesn't use the module any more?

Most of the times yes. Even if I forget doing it 2%-5% of the times,
it's worth it, for me. Notice that I have just shown what coding
practices seem currently good for me, for certain kinds of programs,
etc. Feel free to use your own.

Your modules are callable???

No, they aren't, it's a mistake of mine, sorry.

Time ago I have read an interesting article that says that comments of
today will often become statements and declaration and tags of
tomorrow, that is stuff that the interpreter/compiler/editor
understands. This means that such annotations of mine may be something
fit to become a syntax of the language.

I hope not.

Can you explain why you don't like my idea?

Yes. The vast majority of imports are unconditional and not done
dynamically via the __import__ built-in function and all the
information required about what function/class/method needs what
module to be imported is ALREADY in your source file. If you want to
faff about duplicating that in comments, feel free. However suggesting
that this be enshrined in language syntax is mind -boggling.

[Nick Craig-Wood]

r0g <ai******@technicalbloke.comwrote:

I'm collecting together a bunch of fairly random useful functions I have
written over the years into a module. Generally speaking is it best to

a) Import all the other modules these functions depend on into the
modules global namespace by putting them at the top of the module or
should I...

b) Include them in each function individually.

Stylistically I like the idea of b) better as it makes for easy
refactoring in the future but is there any drawback to doing it this
way? i.e. does it get repeatedly parsed, use more memory, become
inefficient when several functions that use the same includes are
invoked etc Or does it really not matter?

b) has the disadvantage / possible advantage that if the module it
depends on isn't present then it doesn't complain until run time. a)
will always complain at compile time (ie when the script is loaded).

I tend to put all my imports on the top, and I tend to write them like
this

from a_module import fn1, fn2, fn3

Which fails definitively at compile time rather than waiting for the
fn1() to be called.

Importing things into your local namespace is also the quickest, but
that isn't why I do it!

There are arguments against doing this, which I'm sure you'll hear
shortly ;-)

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

[Lawrence D'Oliveiro]

Scott David Daniels wrote:

I would amend this advice by adding that you should (in such cases) put
a commented-out import at the top level (so you can look at the top of a
module's source and see what other modules it relies upon.

That's what your editor's search function is for.