Using marshal to manually "import" a python module

[David Hirschfield]

I had a situation recently that required I manually load python bytecode
from a .pyc file on disk.

So, for the most part, I took code from imputil.py which loads the .pyc
data via the marshal module and then exec's it into a newly created
module object (created by imp.new_module( )). The relevant pieces of code
are in imputil.py_suff ix_importer and imputil.Importe r._process_resu lt
(where it exec's the code into the module's __dict__)

The strange thing is, it worked fine locally on my two machines (32bit
running python 2.3.5 and 64bit running python 2.4.1), but when run by a
64bit machine on the network, it would fail every time in the following
manner:

My marshal/exec-based loader would load the module from the pyc
apparently without problems, but if I then pulled a specific function
attr out of the resulting module object, and that function called
another function defined within the same module, it would raise a
"TypeError: 'NoneType' object is not callable" exception when attempting
to call that second function. So the setup is:

module blah:
def A():
...

def B():
x = A()

compiled to bytecode: blah.pyc
then, in my program:

m = my_marshal_load er("blah.pyc")
f = getattr(m,"B")

x = f()

raises "TypeError: 'NoneType' object is not callable" inside that call
to f(), on the line x = A(), as though A is a None and not the function
object defined in the module.

I can't figure out why this would work locally, but not when the module
is loaded across a network. In fact, I have no idea what would ever
cause it not to see A as a function. I'm stumped, and this is over my
head as far as intimate knowledge of the direct loading of python
bytecode via marshal is concerned...so I'm not clear on the best way to
debug it. If anyone has an inkling of what might be going on, I'd love
to hear it.

Thanks in advance,
-Dave

--
Presenting:
mediocre nebula.

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

The strange thing is, it worked fine locally on my two machines (32bit

running python 2.3.5 and 64bit running python 2.4.1), but when run by a
64bit machine on the network, it would fail every time in the following
manner:

It may not that much be an issue of 32-bit vs. 64-bit, but of Python 2.3
vs. Python 2.4. The marshal format changes across versions, and files
created in one version might not be understood correctly by another version.

Now, you don't give enough details on resolving this completely: what
specific microprocessors , and what specific operating systems are you
using? What is the specific code of the module that fails to load (e.g.
minimal example), and what is the resulting byte code?

Off-hand, I can't see why a 2.3-generated file shouldn't be loadable
by a 2.4 interpreter, but you say "it" fails without saying what "it"
is (i.e. running the 64-bit machine to generate a marshal file over
the net that is then read by the 32-bit machine, or vice versa).

I can't figure out why this would work locally, but not when the module
is loaded across a network. In fact, I have no idea what would ever
cause it not to see A as a function. I'm stumped, and this is over my
head as far as intimate knowledge of the direct loading of python
bytecode via marshal is concerned...so I'm not clear on the best way to
debug it.

I would trace through the loading, either in a debugger, or by adding
printf statements into marshal.

Regards,
Martin