redundant imports

[max(01)*]

hi everybody.

suppose that code-1.py imports code-2.py and code-3.py (because it uses
names from both), and that code-2.py imports code-3.py.

if python were c, code-1.c should only *include* code-2.c, because the
latter in turns includes code-3.c.

inclusion of modules in c is a purely preprocessing textual matter
(compilation is deferred to after the fact), i guess, so that such
things are possible. import of modules in python is a different beast,
so the "redundancy " is (i think) necessary.

any comment/suggestion/idea?

bye

macs

[Tim Jarman]

max(01)* wrote:

hi everybody.

suppose that code-1.py imports code-2.py and code-3.py (because it uses
names from both), and that code-2.py imports code-3.py.

if python were c, code-1.c should only *include* code-2.c, because the
latter in turns includes code-3.c.

inclusion of modules in c is a purely preprocessing textual matter
(compilation is deferred to after the fact), i guess, so that such
things are possible. import of modules in python is a different beast,
so the "redundancy " is (i think) necessary.

any comment/suggestion/idea?

bye

macs

It's not as redundant as it looks. Once a module has been imported it goes
into sys.modules and any subsequent imports refer to that original import,
so the overhead of reading and parsing the file is only incurred once. As
you're probably aware, Python also caches compilation results in *.pyc
files; it will only compile the imported module if it changed since the
last compilation.

Check out the docs for the full skinny, in particular
http://www.python.org/doc/2.4/ref/import.html

HTH,
Tim J

--
Website: www DOT jarmania FULLSTOP com

[Peter Hansen]

max(01)* wrote:

hi everybody.

suppose that code-1.py imports code-2.py and code-3.py (because it uses
names from both), and that code-2.py imports code-3.py.

if python were c, code-1.c should only *include* code-2.c, because the
latter in turns includes code-3.c.

inclusion of modules in c is a purely preprocessing textual matter
(compilation is deferred to after the fact), i guess, so that such
things are possible. import of modules in python is a different beast,
so the "redundancy " is (i think) necessary.

any comment/suggestion/idea?

You're mixed up about this whole idea.

You say first that "[code-1] uses names from both",
which by definition means that it needs to import
both. (I say by definition because import is *how*
a module gets names from another module.)

Then you say that code-1 can choose not to include
code-3 because some other module is including it...
but so what? That doesn't change the fact that
code-1 needs names from code-3, and just because
code-3 is imported elsewhere is no reason to think
that code-1 magically gets its names too. Should
all modules magically see all names in all modules
which are imported, even by other modules? That
would pretty much defeat most of the value of
namespaces.

Anyway, why this concern over so-called redundant
imports? The source code itself is not parsed
and compiled all over again, and even the .pyc file
is not re-read... once any module has imported a
module any other import just retrieves a reference
to that module from the sys.modules dictionary,
which is practically a free operation.

-Peter

[max(01)*]

Tim Jarman wrote:

max(01)* wrote:

hi everybody.

suppose that code-1.py imports code-2.py and code-3.py (because it uses
names from both), and that code-2.py imports code-3.py.

if python were c, code-1.c should only *include* code-2.c, because the
latter in turns includes code-3.c.

inclusion of modules in c is a purely preprocessing textual matter
(compilatio n is deferred to after the fact), i guess, so that such
things are possible. import of modules in python is a different beast,
so the "redundancy " is (i think) necessary.

any comment/suggestion/idea?

bye

macs

It's not as redundant as it looks.

that's why i used quotes ;-)
Once a module has been imported it goes
into sys.modules and any subsequent imports refer to that original import,
so the overhead of reading and parsing the file is only incurred once. As
you're probably aware, Python also caches compilation results in *.pyc
files; it will only compile the imported module if it changed since the
last compilation.

this leads me to another question. since *.pyc files are automatically
created the first time an import statement in executed on a given
module, i guess that if i ship a program with modules for use in a
directory where the user has no write privileges then i must ship the
*.pyc files along too. right?
Check out the docs for the full skinny, in particular
http://www.python.org/doc/2.4/ref/import.html

HTH,
Tim J

thanks a lot

bye

macs

[max(01)*]

Peter Hansen wrote:

max(01)* wrote:

hi everybody.

suppose that code-1.py imports code-2.py and code-3.py (because it
uses names from both), and that code-2.py imports code-3.py.

if python were c, code-1.c should only *include* code-2.c, because the
latter in turns includes code-3.c.

inclusion of modules in c is a purely preprocessing textual matter
(compilation is deferred to after the fact), i guess, so that such
things are possible. import of modules in python is a different beast,
so the "redundancy " is (i think) necessary.

any comment/suggestion/idea?

You're mixed up about this whole idea.

that's why i am asking for advice here.
You say first that "[code-1] uses names from both",
which by definition means that it needs to import
both. (I say by definition because import is *how*
a module gets names from another module.)

Then you say that code-1 can choose not to include
code-3 because some other module is including it...
but so what? That doesn't change the fact that
code-1 needs names from code-3, and just because
code-3 is imported elsewhere is no reason to think
that code-1 magically gets its names too. Should
all modules magically see all names in all modules
which are imported, even by other modules?
that's how inclusion works in c, which is why i asked for clarification
about the differences between c-style includes and python-style imports.

now i can see that python-style import is more like the inclusion of
declarations in c (the "names"), usually shipped as *.h header files.

the real *definition* of names is python (contents of variables,
definition of functions, and so on) i guess is inserted later in the
process of execution.
That
would pretty much defeat most of the value of
namespaces.

Anyway, why this concern over so-called redundant
imports? The source code itself is not parsed
and compiled all over again, and even the .pyc file
is not re-read... once any module has imported a
module any other import just retrieves a reference
to that module from the sys.modules dictionary,
which is practically a free operation.

-Peter

my concern was motivated by a (clumsy) attempt to understand the
difference of mechanism between the approach to modular programming in a
more "traditiona l" language (c) and python.

thanks again for your couseling.

bye

macs

[Peter Hansen]

max(01)* wrote:

this leads me to another question. since *.pyc files are automatically
created the first time an import statement in executed on a given
module, i guess that if i ship a program with modules for use in a
directory where the user has no write privileges then i must ship the
*.pyc files along too. right?

Not required except for performance reasons. If the .pyc
files don't exist, the .py files are recompiled and the
resulting bytecode is simply held in memory and not cached
and the next startup will recompile all over again.

Note also that the main file (the one invoke from the
command line) is never cached in a .pyc...

-Peter

[Steve Holden]

Peter Hansen wrote:

max(01)* wrote:

this leads me to another question. since *.pyc files are automatically
created the first time an import statement in executed on a given
module, i guess that if i ship a program with modules for use in a
directory where the user has no write privileges then i must ship the
*.pyc files along too. right?

Not required except for performance reasons. If the .pyc
files don't exist, the .py files are recompiled and the
resulting bytecode is simply held in memory and not cached
and the next startup will recompile all over again.

Note also that the main file (the one invoke from the
command line) is never cached in a .pyc...

-Peter

Also, beware that you ship the right .pyc files - they are
version-dependent, so shipping 2.3 binaries to a 2.4 user will actually
cause a small slow-down, since the interpreter will have to check the
..pyc's for the correct magic number before ignoring them.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/

[max(01)*]

Peter Hansen wrote:

max(01)* wrote:

this leads me to another question. since *.pyc files are automatically
created the first time an import statement in executed on a given
module, i guess that if i ship a program with modules for use in a
directory where the user has no write privileges then i must ship the
*.pyc files along too. right?

Not required except for performance reasons. If the .pyc
files don't exist, the .py files are recompiled and the
resulting bytecode is simply held in memory and not cached
and the next startup will recompile all over again.

Note also that the main file (the one invoke from the
command line) is never cached in a .pyc...

but the other files *are* compiled, right? so the initial question
remains unanswered: *if* they are compiled, where are they put, if the
corresponding *.py files are on a non-writeable directory?

[Peter Hansen]

max(01)* wrote:

Peter Hansen wrote:

Not required except for performance reasons. If the .pyc
files don't exist, the .py files are recompiled and the
resulting bytecode is simply held in memory and not cached
and the next startup will recompile all over again.
but the other files *are* compiled, right?

Yes, definitely. I did say that.
so the initial question remains unanswered:
No it doesn't. I thought I was clear, but I can reword
it for you: the files are compiled *in-memory* and the
results are never written to disk.
*if* they are compiled, where are they put, if the
corresponding *.py files are on a non-writeable directory?

They are not put anywhere. Compilation is a process
by which the source is converted to executable bytecodes.
This says nothing about where either the source or the
compiled result resides. The actual compilation works
on a long series of bytes in memory, and the result is
just another long series of bytes. Nothing requires that
either of these things are even stored in a file.

Not sure what else I could say to make it clearer...

(I'm not misunderstandin g you am I? I get the feeling
we're not quite on the same page here.)
-Peter

[max(01)*]

Peter Hansen wrote:

max(01)* wrote:

Peter Hansen wrote:

Not required except for performance reasons. If the .pyc
files don't exist, the .py files are recompiled and the
resulting bytecode is simply held in memory and not cached
and the next startup will recompile all over again.

but the other files *are* compiled, right?

Yes, definitely. I did say that.

so the initial question remains unanswered:

No it doesn't. I thought I was clear, but I can reword
it for you: the files are compiled *in-memory* and the
results are never written to disk.

> *if* they are compiled, where are they put, if the

corresponding *.py files are on a non-writeable directory?

They are not put anywhere. Compilation is a process
by which the source is converted to executable bytecodes.
This says nothing about where either the source or the
compiled result resides. The actual compilation works
on a long series of bytes in memory, and the result is
just another long series of bytes. Nothing requires that
either of these things are even stored in a file.

ok, maybe it is an implementation-dependent issue after all.

i am working on a debian woody platform with the standard python2.3 package.

consider this:

ma**@172.17.1.2 01:~/tmp/import-enigma$ ll
total 8
-rw-r--r-- 1 max2 max2 36 2005-04-02 17:44 imported.py
-rw-r--r-- 1 max2 max2 33 2005-04-02 17:44 importer.py
ma**@172.17.1.2 01:~/tmp/import-enigma$ cat importer.py
import imported
imported.fun_1( )
ma**@172.17.1.2 01:~/tmp/import-enigma$ cat imported.py
def fun_1():
print "I am fun_1()"
ma**@172.17.1.2 01:~/tmp/import-enigma$ python importer.py
I am fun_1()
ma**@172.17.1.2 01:~/tmp/import-enigma$ ll
total 12
-rw-r--r-- 1 max2 max2 36 2005-04-02 17:44 imported.py
-rw-r--r-- 1 max2 max2 307 2005-04-02 17:45 imported.pyc
-rw-r--r-- 1 max2 max2 33 2005-04-02 17:44 importer.py
ma**@172.17.1.2 01:~/tmp/import-enigma$

see?

bye

macs