473,403 Members | 2,270 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,403 software developers and data experts.

Using a dictionary to pass data to/from embedded python functions

I have created a Dict object in a C++ App that calls (embedded) Python
functions.
The Dict is to be used to pass variable data between the C++ App and the
python functions.

However I cannot get the Python functions to 'see' the Dict created in the
C++ App.

The C++ app creates the Dict using the following functions:

// Create the dict for the viable data
m_pVarDictionary = PyDict_New();

// Create interface module
m_pInterfaceModule = PyImport_AddModule("InterfaceModule");
// Add the Dictionary to the interface Module
int status = PyModule_AddObject(m_pInterfaceModule, "VarDictionary" ,
m_pVarDictionary);
// Load the script module
m_pScriptModule = PyImport_ImportModule("MyScriptModule");

The C++ app calls the functions using the following functions:

PyObject* func = PyObject_GetAttrString(m_pScriptModule, "functionName");

if (func && PyCallable_Check(func))
{

PyObject* ret = PyObject_CallObject(func, NULL);

}
My test python function looks like this:
"""----------------------------------------------------------------------
Simple scripts for testing Dict Access
-------------------------------------------------------------------------"""
from InterfaceModule import VarDictionary

def hello():
# dumps the contents of VarDictionary
d2 = VarDictionary
f1.write(len(d2))
f1.write('\n')

for k, v in d2.iteritems():
f1.write(k)
f1.write (':')
f1.write(str(v))
f1.write('\n')

f1.close()
This python code throws an exception when it attempts to access the
"VarDictionary".
Does anyone know why this fails ?

I have tried adding code that dumps the Dict returned from vars(),
and could not see "VarDictionary" as an entry in the Vars() dict.
Aug 11 '06 #1
7 2378
wardm <wa*****@gmail.comwrote:
I have created a Dict object in a C++ App that calls (embedded) Python
functions.
[[snip snip]]
This python code throws an exception when it attempts to access the
"VarDictionary".
Does anyone know why this fails ?
It fails due to some code of yours that you have not included: in other
words, the minimal application which embeds Python using _only_ the code
you show us does not exhibit the failure.

I wrote the following file za.c:

#include "Python.h"
#include "stdio.h"
#include "stdlib.h"

int main()
{
printf("start\n");
putenv("PYTHONPATH=.");
Py_Initialize();
printf("inited\n");
PyObject* m_pVarDictionary = PyDict_New();
printf("dict is %p\n", m_pVarDictionary);
PyObject* m_pInterfaceModule =
PyImport_AddModule("InterfaceModule");
printf("modu is %p\n", m_pInterfaceModule);
int status = PyModule_AddObject(m_pInterfaceModule, "VarDictionary"
,
m_pVarDictionary);
printf("stat is %d\n", status);
PyObject* m_pScriptModule = PyImport_ImportModule("MyScriptModule");
printf("impo is %p\n", m_pScriptModule);

PyObject* func = PyObject_GetAttrString(m_pScriptModule,
"functionName");
printf("func is %p\n", func);
if (func && PyCallable_Check(func)) {
PyObject* ret = PyObject_CallObject(func, NULL);
printf("retu is %p\n", ret);
}
printf("done\n");
return 0;
}

and the following file MyScriptModule.py:

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceModule)
print "that's all"
return

and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

and observed exactly the kind of output I predicted [[Note: the exact
addresses printed of course do not matter]]:

start
inited
dict is 0x51c780
modu is 0x520230
stat is 0
impo is 0x5202d0
func is 0x513770
hello
['VarDictionary', '__doc__', '__name__']
that's all
retu is 0xe57c0
done
brain:~/pyex alex$

As you see, in particular, VarDictionary is right up there in
InterfaceModule's dir.
There's a well-known essay by Eric Raymond, "How to ask questions the
smart way", at <http://catb.org/~esr/faqs/smart-questions.html-- it
seems to me that you, quite commendably, follow most of Eric's advice,
but it's still worth reading -- the key point by which you could help us
to help you is what Eric mentions as "If you have a large, complicated
test case that is breaking a program, try to trim it and make it as
small as possible".

In this case, you should try to trim your code down to the smallest
program using this approach, which you believe should work in a certain
way but actually doesn't. The exact code you posted plus the minimal
additions to make it a compilable program does work the way you appear
to desire, as I show above; therefore, there must be something else in
your code that is breaking things. Do tiny steps of addition and
restructuring to move this minimal skeleton towards the direction of
your bigger program (that does not behave this way) until you've found
exactly the "largest" version that still succeeds, and the minisculely
larger "smallest" version that fails -- the tiny difference between the
two must then be the root of the problem, and if the reason is not clear
at that point then posting here again is exactly the right thing to do.

Perhaps you're falling afoul of the fact that (as documented e.g. at
<http://docs.python.org/api/moduleObjects.html>) PyModule_AddObject is a
"convenience function" that steals a reference to the value -- so you
end up with just one reference to the dictionary object, and if for some
reason you decref it, the object gets garbage collected. But, that's
just a wild guess on my part, since you have not shown us any code
performing (for example) any decrefs.
Alex
Aug 12 '06 #2
Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDictionary" in the
MyScriptModule.py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemString( m_pVarDictionary, "tk1",
Py_BuildValue("s","test1Val"));
PyDict_SetItemString( m_pVarDictionary, "tk2",
Py_BuildValue("s","test2Val"));

Then tried various things in the Python code to display the contents of the
"VarDictionary",
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceModule)
print "that's all"
print VarDictionary
return

Even though "VarDictionary " is in the Dir, every time I try to use the
"VarDictionary" the program fails.
Am I doing something wrong when I try and reference "VarDictionary" in
Python ?
I need to be able to get/set entries in VarDictionary from the Python
function.
"Alex Martelli" <al***@mac.comwrote in message
news:1h***************************@mac.com...
wardm <wa*****@gmail.comwrote:
>I have created a Dict object in a C++ App that calls (embedded) Python
functions.
[[snip snip]]
>This python code throws an exception when it attempts to access the
"VarDictionary".
Does anyone know why this fails ?

It fails due to some code of yours that you have not included: in other
words, the minimal application which embeds Python using _only_ the code
you show us does not exhibit the failure.

I wrote the following file za.c:

#include "Python.h"
#include "stdio.h"
#include "stdlib.h"

int main()
{
printf("start\n");
putenv("PYTHONPATH=.");
Py_Initialize();
printf("inited\n");
PyObject* m_pVarDictionary = PyDict_New();
printf("dict is %p\n", m_pVarDictionary);
PyObject* m_pInterfaceModule =
PyImport_AddModule("InterfaceModule");
printf("modu is %p\n", m_pInterfaceModule);
int status = PyModule_AddObject(m_pInterfaceModule, "VarDictionary"
,
m_pVarDictionary);
printf("stat is %d\n", status);
PyObject* m_pScriptModule = PyImport_ImportModule("MyScriptModule");
printf("impo is %p\n", m_pScriptModule);

PyObject* func = PyObject_GetAttrString(m_pScriptModule,
"functionName");
printf("func is %p\n", func);
if (func && PyCallable_Check(func)) {
PyObject* ret = PyObject_CallObject(func, NULL);
printf("retu is %p\n", ret);
}
printf("done\n");
return 0;
}

and the following file MyScriptModule.py:

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceModule)
print "that's all"
return

and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

and observed exactly the kind of output I predicted [[Note: the exact
addresses printed of course do not matter]]:

start
inited
dict is 0x51c780
modu is 0x520230
stat is 0
impo is 0x5202d0
func is 0x513770
hello
['VarDictionary', '__doc__', '__name__']
that's all
retu is 0xe57c0
done
brain:~/pyex alex$

As you see, in particular, VarDictionary is right up there in
InterfaceModule's dir.
There's a well-known essay by Eric Raymond, "How to ask questions the
smart way", at <http://catb.org/~esr/faqs/smart-questions.html-- it
seems to me that you, quite commendably, follow most of Eric's advice,
but it's still worth reading -- the key point by which you could help us
to help you is what Eric mentions as "If you have a large, complicated
test case that is breaking a program, try to trim it and make it as
small as possible".

In this case, you should try to trim your code down to the smallest
program using this approach, which you believe should work in a certain
way but actually doesn't. The exact code you posted plus the minimal
additions to make it a compilable program does work the way you appear
to desire, as I show above; therefore, there must be something else in
your code that is breaking things. Do tiny steps of addition and
restructuring to move this minimal skeleton towards the direction of
your bigger program (that does not behave this way) until you've found
exactly the "largest" version that still succeeds, and the minisculely
larger "smallest" version that fails -- the tiny difference between the
two must then be the root of the problem, and if the reason is not clear
at that point then posting here again is exactly the right thing to do.

Perhaps you're falling afoul of the fact that (as documented e.g. at
<http://docs.python.org/api/moduleObjects.html>) PyModule_AddObject is a
"convenience function" that steals a reference to the value -- so you
end up with just one reference to the dictionary object, and if for some
reason you decref it, the object gets garbage collected. But, that's
just a wild guess on my part, since you have not shown us any code
performing (for example) any decrefs.
Alex

Aug 12 '06 #3
wardm <wa*****@gmail.comwrote:
Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDictionary" in the
MyScriptModule.py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemString( m_pVarDictionary, "tk1",
Py_BuildValue("s","test1Val"));
PyDict_SetItemString( m_pVarDictionary, "tk2",
Py_BuildValue("s","test2Val"));

Then tried various things in the Python code to display the contents of the
"VarDictionary",
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceModule)
print "that's all"
print VarDictionary
Note the wrong indentation in this latter print statement: this would
already cause a syntax error (unless the leading 'p' happened to be
aligned with the leading 'd' of 'def', in which case the function would
be terminated, the latest print would happen at import-time, and the
FOLLOWING statement:
return
....would then be a syntax error (return outside of function). But,
there's more:

Even though "VarDictionary " is in the Dir, every time I try to use the
"VarDictionary" the program fails.
"VarDictionary" is in the dir(...) *** of InterfaceModule ***, of
course, so you need to refer to it as InterfaceModule.VarDictionary in
your Python code -- the barename, nor qualified by modulename, just will
not work, of course!!!

Adding the two C code lines you quote, and changing the Python example
code to:

def functionName():
print "hello"
print dir(InterfaceModule)
print "VarDictionary is:", InterfaceModule.VarDictionary
print "that's all"

changes that part of the output to:

hello
['VarDictionary', '__doc__', '__name__']
VarDictionary is: {'tk2': 'test2Val', 'tk1': 'test1Val'}
that's all
With all due respect, it looks like you're trying to run before you can
walk -- or more specifically, to embed Python in C++ before you become
familiar with the most elementary and fundamental aspects of Python,
such as indentation and the need to qualify compound names. You might
want to consider getting a good Python book -- such as, my own Python in
A Nutshell (2nd ed), Aahz and Stef Maruch's Python For Dummies, Wesley
Chun's Core Python Programming (2nd ed) -- they're all very recent (mine
is just out, Aahz's and Stef's I believe is due to hit bookstores in
September or October), and any of them might serve you well (if you're
OK with books not necessarily covering the very latest release of Python
[and the issues you're having suggest that this is not really the
problem!], there are many other good books, such as Magnus Lie Hetland's
"Beginning Python", Beazley's "Python Essential Reference", Lutz and
Ascher's "Learning Python",
Aug 12 '06 #4
Thanks again for your help, I agree, it seems I need to read a good book on
Python.

One last question, will Python allow me to add new items to
InterfaceModule.VarDictionary
from the Python functions I call ?
"Alex Martelli" <al***@mac.comwrote in message
news:1h***************************@mac.com...
wardm <wa*****@gmail.comwrote:
>Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDictionary" in the
MyScriptModule.py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemString( m_pVarDictionary, "tk1",
Py_BuildValue("s","test1Val"));
PyDict_SetItemString( m_pVarDictionary, "tk2",
Py_BuildValue("s","test2Val"));

Then tried various things in the Python code to display the contents of
the
"VarDictionary",
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceModule)
print "that's all"
print VarDictionary

Note the wrong indentation in this latter print statement: this would
already cause a syntax error (unless the leading 'p' happened to be
aligned with the leading 'd' of 'def', in which case the function would
be terminated, the latest print would happen at import-time, and the
FOLLOWING statement:
> return

...would then be a syntax error (return outside of function). But,
there's more:

>Even though "VarDictionary " is in the Dir, every time I try to use the
"VarDictionary" the program fails.

"VarDictionary" is in the dir(...) *** of InterfaceModule ***, of
course, so you need to refer to it as InterfaceModule.VarDictionary in
your Python code -- the barename, nor qualified by modulename, just will
not work, of course!!!

Adding the two C code lines you quote, and changing the Python example
code to:

def functionName():
print "hello"
print dir(InterfaceModule)
print "VarDictionary is:", InterfaceModule.VarDictionary
print "that's all"

changes that part of the output to:

hello
['VarDictionary', '__doc__', '__name__']
VarDictionary is: {'tk2': 'test2Val', 'tk1': 'test1Val'}
that's all
With all due respect, it looks like you're trying to run before you can
walk -- or more specifically, to embed Python in C++ before you become
familiar with the most elementary and fundamental aspects of Python,
such as indentation and the need to qualify compound names. You might
want to consider getting a good Python book -- such as, my own Python in
A Nutshell (2nd ed), Aahz and Stef Maruch's Python For Dummies, Wesley
Chun's Core Python Programming (2nd ed) -- they're all very recent (mine
is just out, Aahz's and Stef's I believe is due to hit bookstores in
September or October), and any of them might serve you well (if you're
OK with books not necessarily covering the very latest release of Python
[and the issues you're having suggest that this is not really the
problem!], there are many other good books, such as Magnus Lie Hetland's
"Beginning Python", Beazley's "Python Essential Reference", Lutz and
Ascher's "Learning Python",

Aug 13 '06 #5
wardm <wa*****@gmail.comwrote:
Thanks again for your help, I agree, it seems I need to read a good book on
Python.

One last question, will Python allow me to add new items to
InterfaceModule.VarDictionary
from the Python functions I call ?
Yes, no problem with that.
Alex
Aug 13 '06 #6

Alex Martelli wrote:
>
I wrote the following file za.c:
<sample code snipped>
and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

I needed to include a lot more flags to make the example compile.
In particular:
-L /usr/local/lib/python2.4/config/ -lpython2.4 -lrt -lm -ldl -lutil

Why did you not need them? Is this a misconfiguration on my
box, or something entirely different? I've seen this type of thing
come up a lot, and I can't tell if it is simply the author omitting
flags for the sake of brevity, or if I'm actually missing something.

--
Bill Pursell

Aug 13 '06 #7
Bill Pursell <bi**********@gmail.comwrote:
Alex Martelli wrote:

I wrote the following file za.c:

<sample code snipped>
and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za


I needed to include a lot more flags to make the example compile.
In particular:
-L /usr/local/lib/python2.4/config/ -lpython2.4 -lrt -lm -ldl -lutil
You no doubt needed that for linking, not for compiling (I needed the -L
and -l too for linking, see above, just not the "other" libraries).

Why did you not need them? Is this a misconfiguration on my
box, or something entirely different? I've seen this type of thing
come up a lot, and I can't tell if it is simply the author omitting
flags for the sake of brevity, or if I'm actually missing something.
Neither: we're just talking about different platforms -- probably both
Unix variants, but with different linking-loader behavior and/or
dependencies among libraries. Considering that my platform's ld is
doubtlessly the single most prominent difference between it and other
Unix variants from a developer's viewpoint (see
<http://developer.apple.com/documenta.../ManPages/man1
/ld.1.htmlfor more details) I was remiss in not mentioning my
platform; I apologize, I was concentrating on the C++ and Python sources
rather than on the build process (but I should have mentioned it in the
Note, just as I mentioned that I was using 2.5 but it didn't matter!).
Alex
Aug 13 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Robert | last post by:
Python doesn't know the class of a method when container not direct class attribute: >>> class X: .... def f():pass .... g=f .... l= .... >>> X.g <unbound method X.f>
5
by: Lukas Holcik | last post by:
Hi everyone! How can I simply search text for regexps (lets say <a href="(.*?)">(.*?)</a>) and save all URLs(1) and link contents(2) in a dictionary { name : URL}? In a single pass if it could....
8
by: Joakim Persson | last post by:
Hello all. I am involved in a project where we have a desire to improve our software testing tools, and I'm in charge of looking for solutions regarding the logging of our software (originating...
4
by: Noah | last post by:
I have a dictionary that I would like to expand to satisfy a function's agument list. I can used the ** syntax to pass a dictionary, but this only works if each key in the dictionary matches an...
7
by: David Bear | last post by:
I have a dictionary that contains a row of data intended for a data base. The dictionary keys are the field names. The values are the values to be inserted. I am looking for a good pythonic...
4
by: 63q2o4i02 | last post by:
Hi, I'm writing a hand-written recursive decent parser for SPICE syntax parsing. In one case I have one function that handles a bunch of similar cases (you pass the name and the number of...
70
by: jojoba | last post by:
Hello! Does anyone know how to find the name of a python data type. Conside a dictionary: Banana = {} Then, how do i ask python for a string representing the name of the above dictionary...
5
by: Bill Jackson | last post by:
What is the benefit of clearing a dictionary, when you can just reassign it as empty? Similarly, suppose I generate a new dictionary b, and need to have it accessible from a. What is the best...
0
by: napolpie | last post by:
DISCUSSION IN USER nappie writes: Hello, I'm Peter and I'm new in python codying and I'm using parsying to extract data from one meteo Arpege file. This file is long file and it's composed by...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.