469,609 Members | 1,228 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,609 developers. It's quick & easy.

Import/Create module from buffer

Hi people,

I'm currently working on python embedding with C++. My goal is that
the C++ part handle files writing/reading so that the Python part only
works with buffers.

I succeeded in buffer exchanges. The problem is that some of the files
I read/write are python files so that, before embedding, I imported
them as modules when I needed them.

But now that the Python part only receive buffers, I can't do it
anymore. So I wonder :
- is it possible to import module from a buffer instead of files?
- or is it possible to create a module object from my buffer?

I looked on the Internet but didn't find anything about that. Does
anyone have an idea?

Thanks
Benjamin
Jun 27 '08 #1
5 2507
En Mon, 12 May 2008 05:49:22 -0300, Gruik <be**************@gmail.comescribió:
I'm currently working on python embedding with C++. My goal is that
the C++ part handle files writing/reading so that the Python part only
works with buffers.

I succeeded in buffer exchanges. The problem is that some of the files
I read/write are python files so that, before embedding, I imported
them as modules when I needed them.

But now that the Python part only receive buffers, I can't do it
anymore. So I wonder :
- is it possible to import module from a buffer instead of files?
- or is it possible to create a module object from my buffer?
Yes, first compile the buffer to get a code object, then use PyImport_ExecCodeModule. See how this function is used in import.c

--
Gabriel Genellina

Jun 27 '08 #2
On May 12, 12:31*pm, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
En Mon, 12 May 2008 05:49:22 -0300, Gruik <benjamin.frem...@gmail.comescribió:
I'm currently working on python embedding with C++. My goal is that
the C++ part handle files writing/reading so that the Python part only
works with buffers.
I succeeded in buffer exchanges. The problem is that some of the files
I read/write are python files so that, before embedding, I imported
them as modules when I needed them.
But now that the Python part only receive buffers, I can't do it
anymore. So I wonder :
- is it possible to import module from a buffer instead of files?
- or is it possible to create a module object from my buffer?

Yes, first compile the buffer to get a code object, then use PyImport_ExecCodeModule. See how this function is used in import.c

--
Gabriel Genellina
Thanks for your quick answer !
I think I'll have no problem with that in C++ and I'm going to try it
right after this message.

But before that 1 question: what if I'm in Python ?
Following your solution, I did that in Python :

def load_buffer(buffer) :
compiled_buffer = compile(buffer, "module_name", "exec")
exec(compiled_buffer)

It works great except that I can't have a module object and that it is
as if I did "from module import *"
But I need the module object and not an "import *" behavior.
Any idea about the way to do that?

Benjamin
Jun 27 '08 #3
Gruik wrote:
But before that 1 question: what if I'm in Python ?
Following your solution, I did that in Python :

def load_buffer(buffer) :
compiled_buffer = compile(buffer, "module_name", "exec")
exec(compiled_buffer)

It works great except that I can't have a module object and that it is
as if I did "from module import *"
But I need the module object and not an "import *" behavior.
Any idea about the way to do that?
Something along the lines of:

import new
mymodule = new.module("mymodule")
exec <<<code>>in mymodule.__dict__

--irmen
Jun 27 '08 #4
On May 12, 1:48*pm, Irmen de Jong <irmen.NOS...@xs4all.nlwrote:
Gruik wrote:
But before that 1 question: what if I'm in Python ?
Following your solution, I did that in Python :
* * def load_buffer(buffer) :
* * * * compiled_buffer = compile(buffer, "module_name", "exec")
* * * * exec(compiled_buffer)
It works great except that I can't have a module object and that it is
as if I did "from module import *"
But I need the module object and not an "import *" behavior.
Any idea about the way to do that?

Something along the lines of:

import new
mymodule = new.module("mymodule")
exec <<<code>>in mymodule.__dict__

--irmen
Yeah it works !

exec(compiled_module, globals(), mymodule.__dict__)

Just to add mymodule to sys.modules and it's good!

Thanks again
Benjamin
Jun 27 '08 #5
En Mon, 12 May 2008 08:09:45 -0300, Gruik <be**************@gmail.comescribió:
On May 12, 12:31*pm, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
>En Mon, 12 May 2008 05:49:22 -0300, Gruik <benjamin.frem...@gmail.comescribió:
I'm currently working on python embedding with C++. My goal is that
the C++ part handle files writing/reading so that the Python part only
works with buffers.
I succeeded in buffer exchanges. The problem is that some of the files
I read/write are python files so that, before embedding, I imported
them as modules when I needed them.
But now that the Python part only receive buffers, I can't do it
anymore. So I wonder :
- is it possible to import module from a buffer instead of files?
- or is it possible to create a module object from my buffer?

Yes, first compile the buffer to get a code object, then use PyImport_ExecCodeModule. See how this function is used in import.c

Thanks for your quick answer !
I think I'll have no problem with that in C++ and I'm going to try it
right after this message.

But before that 1 question: what if I'm in Python ?
Following your solution, I did that in Python :

def load_buffer(buffer) :
compiled_buffer = compile(buffer, "module_name", "exec")
exec(compiled_buffer)

It works great except that I can't have a module object and that it is
as if I did "from module import *"
But I need the module object and not an "import *" behavior.
Any idea about the way to do that?
Almost - you have to create a new module and exec the code into its namespace:

pyimport new
pyfoo = new.module("foo", "This is the foo module")
pyexec "def f(): pass" in foo.__dict__
pyfoo
<module 'foo' (built-in)>
pyfoo.f
<function f at 0x00A3E230>
pydir(foo)
['__builtins__', '__doc__', '__name__', 'f']

(replace "def f(): pass" with the compiled buffer)

Two differences with a "normal" module:

- it has no __file__ attribute - you may want to add it, with the original filename, or leave it off to show that it's not loaded from a file.
- it does not exist in sys.modules, so other modules cannot import it. If you want to allow that: sys.modules['foo'] = foo (perhaps one should check previously that there is no module named "foo" before replacing it...)

--
Gabriel Genellina

Jun 27 '08 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by simo | last post: by
2 posts views Thread by Fritz Bosch | last post: by
4 posts views Thread by Steve Holden | last post: by
2 posts views Thread by Torsten Mohr | last post: by
16 posts views Thread by didier.doussaud | last post: by
49 posts views Thread by Martin Unsal | last post: by
10 posts views Thread by nisp | last post: by
reply views Thread by devrayhaan | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.