By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,732 Members | 1,435 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,732 IT Pros & Developers. It's quick & easy.

Import Problem

P: n/a
I've found a case where it seems that Python is importing two copies of a
module without any reason or indication. It took me a while to verify that
this is what is occuring: I had to write a __import__ hook to trace the
actual activity. The source code for the hook is below.

There are several examples of the problem in the trace; this is one
particularly juicy one.

-----------Part 1 ----------------
'0059' Import 'TypeAdapter' global: 'fit.Fixture' local: '9547968' 'None'

'0060' Import 're' global: 'fit.TypeAdapter' local: '9965904' 'None'
'0060' path: 're' module: '9387920' dict: '9415248'
'0060' result 're' is at '9387920'
'0060' path: 're' module: '9387920' dict: '9415248'
----------- end of part 1 ------------------------

What this shows is import # 59 importing 'TypeAdapter' from
'fit.Fixture.'

TypeAdapter is then importing 're'. Before the import, re is
already in the sys.modules; the import returns the expected
address, and the sys.modules entry remains the same. This
isn't a problem, it's simply an example of trace output for
a correct action.

-------------- Part 2 --------------------------
'0113' Import 'compiler.pyassem' global: 'compiler.pycodegen' local:
'10439248' '('TupleArg',)'
'0113' path: 'compiler.pyassem' module: '10508624' dict: '10515056'
'0113' result 'compiler.pyassem' is at '10508624'
'0113' path: 'compiler.pyassem' module: '10508624' dict: '10515056'
'0081' result 'pycodegen' is at '10396368'
'0081' path: 'compiler.pycodegen' module: '10396368' dict: '10439248'
'0066' result 'compiler' is at '10006320'
'0066' path: 'compiler' module: '10006320' dict: '9968496'
'0059' result 'TypeAdapter' is at '9973392'
'0059' path: 'fit.TypeAdapter' module: '9973392' dict: '9965904'
--------------- End of Part 2 ------------------------------

This shows the final unwinding of the TypeAdapter import that occurs
about 54 imports later! Note the address of the TypeAdapter module.
The fact that there's no path entry for 59 before the result of the
import shows that it was not in the sys.modules dictionary at that
point.

--------------- Part 3 ----------------------------------------
'0118' Import 'fit.eg.Division' global: 'None' local: 'None' 'None'

'0119' Import 'fit.ColumnFixture' global: 'fit.eg.Division' local:
'10636576' '('ColumnFixture',)'
------------------- End of Part 3 ------------------------------

This simply shows the start of the traces for imports 118 and 119,
which are in the next entry.

--------------- Part 4 ----------------------------------------
'0125' Import 'fit.TypeAdapter' global: 'fit.ColumnFixture' local:
'10637008' 'None'
'0125' path: 'fit.TypeAdapter' module: '9973392' dict: '9965904'
'0125' result 'fit.TypeAdapter' is at '8976560'
'0125' path: 'fit.TypeAdapter' module: '9973392' dict: '9965904'
'0119' result 'fit.ColumnFixture' is at '10650256'
'0119' path: 'fit.ColumnFixture' module: '10650256' dict: '10637008'
'0118' result 'fit.eg.Division' is at '8976560'
'0118' path: 'fit.eg.Division' module: '10603024' dict: '10636576'
--------------- End of Part 4 ------------------------------------

Entry 125 is the next import of TypeAdapter, in module ColumnFixture
which is in module Division. Note that sys.modules (the two lines that
start with 'path') still shows the same address for TypeAdapter as the
previous trace entries: that hasn't changed. However, the third line
shows that the result of the import is at a different address!

I'm baffled about why it's occurring. System is ActiveState Python
2.3.3. on Windows XP.

The import hook is:

---------------------- Beginning of module ImportSpike ------------
# Override for Import statement

print "in ImportSpike"
import __builtin__
import sys
from types import ModuleType

seqNum = [0]

def identifyGlobalDict(aDict, seqNum):
if aDict is None:
return "None"
for key, value in sys.modules.items():
if value is None: continue
try:
if value.__dict__ == aDict:
return key
except:
pass
return id(aDict)

def findModule(name, seqNum):
for key, value in sys.modules.items():
if value is None: continue
if key.endswith(name):
if len(key) == len(name) or key[-(len(name)+1)] == '.':
print "'%04i' path: '%s' module: '%s' dict: '%s'" % (
seqNum, key, id(value), id(value.__dict__))
if type(value) != ModuleType:
print "'%04i' ***** '%s' is '%s'" % (
seqNum, key, type(value))

def newImport(path, globals = None, locals = None, nameList = None):
seqNum[0] += 1
localSeqNum = seqNum[0]
localObj = locals and id(locals)
globalObj = globals and id(globals)
print " "
print "'%04i' Import '%s' global: '%s' local: '%s' '%s'" % (
localSeqNum, path, identifyGlobalDict(globals, localSeqNum),
localObj, nameList)
findModule(path, localSeqNum)
result = oldImport(path, globals, locals, nameList)
print "'%04i' result '%s' is at '%s'" % (localSeqNum, path, id(result))
findModule(path, localSeqNum)
return result

oldImport = __import__

__builtin__.__import__ = newImport
------------------------- End of module ImportSpike -------------------

Any idea of what's going on?

John Roth


Jul 18 '05 #1
Share this question for a faster answer!
Share on Google+

This discussion thread is closed

Replies have been disabled for this discussion.