473,563 Members | 2,724 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Using a dictionary to pass data to/from embedded python functions

I have created a Dict object in a C++ App that calls (embedded) Python
functions.
The Dict is to be used to pass variable data between the C++ App and the
python functions.

However I cannot get the Python functions to 'see' the Dict created in the
C++ App.

The C++ app creates the Dict using the following functions:

// Create the dict for the viable data
m_pVarDictionar y = PyDict_New();

// Create interface module
m_pInterfaceMod ule = PyImport_AddMod ule("InterfaceM odule");
// Add the Dictionary to the interface Module
int status = PyModule_AddObj ect(m_pInterfac eModule, "VarDiction ary" ,
m_pVarDictionar y);
// Load the script module
m_pScriptModule = PyImport_Import Module("MyScrip tModule");

The C++ app calls the functions using the following functions:

PyObject* func = PyObject_GetAtt rString(m_pScri ptModule, "functionName") ;

if (func && PyCallable_Chec k(func))
{

PyObject* ret = PyObject_CallOb ject(func, NULL);

}
My test python function looks like this:
"""----------------------------------------------------------------------
Simple scripts for testing Dict Access
-------------------------------------------------------------------------"""
from InterfaceModule import VarDictionary

def hello():
# dumps the contents of VarDictionary
d2 = VarDictionary
f1.write(len(d2 ))
f1.write('\n')

for k, v in d2.iteritems():
f1.write(k)
f1.write (':')
f1.write(str(v) )
f1.write('\n')

f1.close()
This python code throws an exception when it attempts to access the
"VarDictionary" .
Does anyone know why this fails ?

I have tried adding code that dumps the Dict returned from vars(),
and could not see "VarDiction ary" as an entry in the Vars() dict.
Aug 11 '06 #1
7 2392
wardm <wa*****@gmail. comwrote:
I have created a Dict object in a C++ App that calls (embedded) Python
functions.
[[snip snip]]
This python code throws an exception when it attempts to access the
"VarDictionary" .
Does anyone know why this fails ?
It fails due to some code of yours that you have not included: in other
words, the minimal application which embeds Python using _only_ the code
you show us does not exhibit the failure.

I wrote the following file za.c:

#include "Python.h"
#include "stdio.h"
#include "stdlib.h"

int main()
{
printf("start\n ");
putenv("PYTHONP ATH=.");
Py_Initialize() ;
printf("inited\ n");
PyObject* m_pVarDictionar y = PyDict_New();
printf("dict is %p\n", m_pVarDictionar y);
PyObject* m_pInterfaceMod ule =
PyImport_AddMod ule("InterfaceM odule");
printf("modu is %p\n", m_pInterfaceMod ule);
int status = PyModule_AddObj ect(m_pInterfac eModule, "VarDiction ary"
,
m_pVarDictionar y);
printf("stat is %d\n", status);
PyObject* m_pScriptModule = PyImport_Import Module("MyScrip tModule");
printf("impo is %p\n", m_pScriptModule );

PyObject* func = PyObject_GetAtt rString(m_pScri ptModule,
"functionName") ;
printf("func is %p\n", func);
if (func && PyCallable_Chec k(func)) {
PyObject* ret = PyObject_CallOb ject(func, NULL);
printf("retu is %p\n", ret);
}
printf("done\n" );
return 0;
}

and the following file MyScriptModule. py:

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "that's all"
return

and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

and observed exactly the kind of output I predicted [[Note: the exact
addresses printed of course do not matter]]:

start
inited
dict is 0x51c780
modu is 0x520230
stat is 0
impo is 0x5202d0
func is 0x513770
hello
['VarDictionary' , '__doc__', '__name__']
that's all
retu is 0xe57c0
done
brain:~/pyex alex$

As you see, in particular, VarDictionary is right up there in
InterfaceModule 's dir.
There's a well-known essay by Eric Raymond, "How to ask questions the
smart way", at <http://catb.org/~esr/faqs/smart-questions.html-- it
seems to me that you, quite commendably, follow most of Eric's advice,
but it's still worth reading -- the key point by which you could help us
to help you is what Eric mentions as "If you have a large, complicated
test case that is breaking a program, try to trim it and make it as
small as possible".

In this case, you should try to trim your code down to the smallest
program using this approach, which you believe should work in a certain
way but actually doesn't. The exact code you posted plus the minimal
additions to make it a compilable program does work the way you appear
to desire, as I show above; therefore, there must be something else in
your code that is breaking things. Do tiny steps of addition and
restructuring to move this minimal skeleton towards the direction of
your bigger program (that does not behave this way) until you've found
exactly the "largest" version that still succeeds, and the minisculely
larger "smallest" version that fails -- the tiny difference between the
two must then be the root of the problem, and if the reason is not clear
at that point then posting here again is exactly the right thing to do.

Perhaps you're falling afoul of the fact that (as documented e.g. at
<http://docs.python.org/api/moduleObjects.h tml>) PyModule_AddObj ect is a
"convenienc e function" that steals a reference to the value -- so you
end up with just one reference to the dictionary object, and if for some
reason you decref it, the object gets garbage collected. But, that's
just a wild guess on my part, since you have not shown us any code
performing (for example) any decrefs.
Alex
Aug 12 '06 #2
Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDiction ary" in the
MyScriptModule. py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemS tring( m_pVarDictionar y, "tk1",
Py_BuildValue(" s","test1Val")) ;
PyDict_SetItemS tring( m_pVarDictionar y, "tk2",
Py_BuildValue(" s","test2Val")) ;

Then tried various things in the Python code to display the contents of the
"VarDictionary" ,
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "that's all"
print VarDictionary
return

Even though "VarDiction ary " is in the Dir, every time I try to use the
"VarDiction ary" the program fails.
Am I doing something wrong when I try and reference "VarDiction ary" in
Python ?
I need to be able to get/set entries in VarDictionary from the Python
function.
"Alex Martelli" <al***@mac.comw rote in message
news:1h******** *************** ****@mac.com...
wardm <wa*****@gmail. comwrote:
>I have created a Dict object in a C++ App that calls (embedded) Python
functions.
[[snip snip]]
>This python code throws an exception when it attempts to access the
"VarDictionary ".
Does anyone know why this fails ?

It fails due to some code of yours that you have not included: in other
words, the minimal application which embeds Python using _only_ the code
you show us does not exhibit the failure.

I wrote the following file za.c:

#include "Python.h"
#include "stdio.h"
#include "stdlib.h"

int main()
{
printf("start\n ");
putenv("PYTHONP ATH=.");
Py_Initialize() ;
printf("inited\ n");
PyObject* m_pVarDictionar y = PyDict_New();
printf("dict is %p\n", m_pVarDictionar y);
PyObject* m_pInterfaceMod ule =
PyImport_AddMod ule("InterfaceM odule");
printf("modu is %p\n", m_pInterfaceMod ule);
int status = PyModule_AddObj ect(m_pInterfac eModule, "VarDiction ary"
,
m_pVarDictionar y);
printf("stat is %d\n", status);
PyObject* m_pScriptModule = PyImport_Import Module("MyScrip tModule");
printf("impo is %p\n", m_pScriptModule );

PyObject* func = PyObject_GetAtt rString(m_pScri ptModule,
"functionName") ;
printf("func is %p\n", func);
if (func && PyCallable_Chec k(func)) {
PyObject* ret = PyObject_CallOb ject(func, NULL);
printf("retu is %p\n", ret);
}
printf("done\n" );
return 0;
}

and the following file MyScriptModule. py:

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "that's all"
return

and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

and observed exactly the kind of output I predicted [[Note: the exact
addresses printed of course do not matter]]:

start
inited
dict is 0x51c780
modu is 0x520230
stat is 0
impo is 0x5202d0
func is 0x513770
hello
['VarDictionary' , '__doc__', '__name__']
that's all
retu is 0xe57c0
done
brain:~/pyex alex$

As you see, in particular, VarDictionary is right up there in
InterfaceModule 's dir.
There's a well-known essay by Eric Raymond, "How to ask questions the
smart way", at <http://catb.org/~esr/faqs/smart-questions.html-- it
seems to me that you, quite commendably, follow most of Eric's advice,
but it's still worth reading -- the key point by which you could help us
to help you is what Eric mentions as "If you have a large, complicated
test case that is breaking a program, try to trim it and make it as
small as possible".

In this case, you should try to trim your code down to the smallest
program using this approach, which you believe should work in a certain
way but actually doesn't. The exact code you posted plus the minimal
additions to make it a compilable program does work the way you appear
to desire, as I show above; therefore, there must be something else in
your code that is breaking things. Do tiny steps of addition and
restructuring to move this minimal skeleton towards the direction of
your bigger program (that does not behave this way) until you've found
exactly the "largest" version that still succeeds, and the minisculely
larger "smallest" version that fails -- the tiny difference between the
two must then be the root of the problem, and if the reason is not clear
at that point then posting here again is exactly the right thing to do.

Perhaps you're falling afoul of the fact that (as documented e.g. at
<http://docs.python.org/api/moduleObjects.h tml>) PyModule_AddObj ect is a
"convenienc e function" that steals a reference to the value -- so you
end up with just one reference to the dictionary object, and if for some
reason you decref it, the object gets garbage collected. But, that's
just a wild guess on my part, since you have not shown us any code
performing (for example) any decrefs.
Alex

Aug 12 '06 #3
wardm <wa*****@gmail. comwrote:
Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDiction ary" in the
MyScriptModule. py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemS tring( m_pVarDictionar y, "tk1",
Py_BuildValue(" s","test1Val")) ;
PyDict_SetItemS tring( m_pVarDictionar y, "tk2",
Py_BuildValue(" s","test2Val")) ;

Then tried various things in the Python code to display the contents of the
"VarDictionary" ,
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "that's all"
print VarDictionary
Note the wrong indentation in this latter print statement: this would
already cause a syntax error (unless the leading 'p' happened to be
aligned with the leading 'd' of 'def', in which case the function would
be terminated, the latest print would happen at import-time, and the
FOLLOWING statement:
return
....would then be a syntax error (return outside of function). But,
there's more:

Even though "VarDiction ary " is in the Dir, every time I try to use the
"VarDiction ary" the program fails.
"VarDiction ary" is in the dir(...) *** of InterfaceModule ***, of
course, so you need to refer to it as InterfaceModule .VarDictionary in
your Python code -- the barename, nor qualified by modulename, just will
not work, of course!!!

Adding the two C code lines you quote, and changing the Python example
code to:

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "VarDiction ary is:", InterfaceModule .VarDictionary
print "that's all"

changes that part of the output to:

hello
['VarDictionary' , '__doc__', '__name__']
VarDictionary is: {'tk2': 'test2Val', 'tk1': 'test1Val'}
that's all
With all due respect, it looks like you're trying to run before you can
walk -- or more specifically, to embed Python in C++ before you become
familiar with the most elementary and fundamental aspects of Python,
such as indentation and the need to qualify compound names. You might
want to consider getting a good Python book -- such as, my own Python in
A Nutshell (2nd ed), Aahz and Stef Maruch's Python For Dummies, Wesley
Chun's Core Python Programming (2nd ed) -- they're all very recent (mine
is just out, Aahz's and Stef's I believe is due to hit bookstores in
September or October), and any of them might serve you well (if you're
OK with books not necessarily covering the very latest release of Python
[and the issues you're having suggest that this is not really the
problem!], there are many other good books, such as Magnus Lie Hetland's
"Beginning Python", Beazley's "Python Essential Reference", Lutz and
Ascher's "Learning Python",
Aug 12 '06 #4
Thanks again for your help, I agree, it seems I need to read a good book on
Python.

One last question, will Python allow me to add new items to
InterfaceModule .VarDictionary
from the Python functions I call ?
"Alex Martelli" <al***@mac.comw rote in message
news:1h******** *************** ****@mac.com...
wardm <wa*****@gmail. comwrote:
>Thanks Alex for your help, (and advice on focusing the point of my
question).

I was able to compile and run your example OK, but when I try to use the
"VarDictionary " in the
MyScriptModule .py code, I get an exception.

I added the following code to the C app just to add two entries to the
Dictionary

PyDict_SetItemS tring( m_pVarDictionar y, "tk1",
Py_BuildValue( "s","test1Val") );
PyDict_SetItemS tring( m_pVarDictionar y, "tk2",
Py_BuildValue( "s","test2Val") );

Then tried various things in the Python code to display the contents of
the
"VarDictionary ",
such as adding the "print VarDictionary" below.

import InterfaceModule

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "that's all"
print VarDictionary

Note the wrong indentation in this latter print statement: this would
already cause a syntax error (unless the leading 'p' happened to be
aligned with the leading 'd' of 'def', in which case the function would
be terminated, the latest print would happen at import-time, and the
FOLLOWING statement:
> return

...would then be a syntax error (return outside of function). But,
there's more:

>Even though "VarDiction ary " is in the Dir, every time I try to use the
"VarDictionary " the program fails.

"VarDiction ary" is in the dir(...) *** of InterfaceModule ***, of
course, so you need to refer to it as InterfaceModule .VarDictionary in
your Python code -- the barename, nor qualified by modulename, just will
not work, of course!!!

Adding the two C code lines you quote, and changing the Python example
code to:

def functionName():
print "hello"
print dir(InterfaceMo dule)
print "VarDiction ary is:", InterfaceModule .VarDictionary
print "that's all"

changes that part of the output to:

hello
['VarDictionary' , '__doc__', '__name__']
VarDictionary is: {'tk2': 'test2Val', 'tk1': 'test1Val'}
that's all
With all due respect, it looks like you're trying to run before you can
walk -- or more specifically, to embed Python in C++ before you become
familiar with the most elementary and fundamental aspects of Python,
such as indentation and the need to qualify compound names. You might
want to consider getting a good Python book -- such as, my own Python in
A Nutshell (2nd ed), Aahz and Stef Maruch's Python For Dummies, Wesley
Chun's Core Python Programming (2nd ed) -- they're all very recent (mine
is just out, Aahz's and Stef's I believe is due to hit bookstores in
September or October), and any of them might serve you well (if you're
OK with books not necessarily covering the very latest release of Python
[and the issues you're having suggest that this is not really the
problem!], there are many other good books, such as Magnus Lie Hetland's
"Beginning Python", Beazley's "Python Essential Reference", Lutz and
Ascher's "Learning Python",

Aug 13 '06 #5
wardm <wa*****@gmail. comwrote:
Thanks again for your help, I agree, it seems I need to read a good book on
Python.

One last question, will Python allow me to add new items to
InterfaceModule .VarDictionary
from the Python functions I call ?
Yes, no problem with that.
Alex
Aug 13 '06 #6

Alex Martelli wrote:
>
I wrote the following file za.c:
<sample code snipped>
and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za

I needed to include a lot more flags to make the example compile.
In particular:
-L /usr/local/lib/python2.4/config/ -lpython2.4 -lrt -lm -ldl -lutil

Why did you not need them? Is this a misconfiguratio n on my
box, or something entirely different? I've seen this type of thing
come up a lot, and I can't tell if it is simply the author omitting
flags for the sake of brevity, or if I'm actually missing something.

--
Bill Pursell

Aug 13 '06 #7
Bill Pursell <bi**********@g mail.comwrote:
Alex Martelli wrote:

I wrote the following file za.c:

<sample code snipped>
and proceeded to compile and execute as follows: [[Note: it does not
matter that I'm using 2.5, the code is just as fine with previous
versions -- it just happens that 2.5 is what I'm using right now in
order to help out with 2.5's beta testing]]:

brain:~/pyex alex$ gcc -c za.c -I/usr/local/include/python2.5
brain:~/pyex alex$ gcc -o za za.o -L/usr/local/lib/python2.5/config/
-lpython2.5
brain:~/pyex alex$ ./za


I needed to include a lot more flags to make the example compile.
In particular:
-L /usr/local/lib/python2.4/config/ -lpython2.4 -lrt -lm -ldl -lutil
You no doubt needed that for linking, not for compiling (I needed the -L
and -l too for linking, see above, just not the "other" libraries).

Why did you not need them? Is this a misconfiguratio n on my
box, or something entirely different? I've seen this type of thing
come up a lot, and I can't tell if it is simply the author omitting
flags for the sake of brevity, or if I'm actually missing something.
Neither: we're just talking about different platforms -- probably both
Unix variants, but with different linking-loader behavior and/or
dependencies among libraries. Considering that my platform's ld is
doubtlessly the single most prominent difference between it and other
Unix variants from a developer's viewpoint (see
<http://developer.apple.com/documenta.../ManPages/man1
/ld.1.htmlfor more details) I was remiss in not mentioning my
platform; I apologize, I was concentrating on the C++ and Python sources
rather than on the build process (but I should have mentioned it in the
Note, just as I mentioned that I was using 2.5 but it didn't matter!).
Alex
Aug 13 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
3039
by: Robert | last post by:
Python doesn't know the class of a method when container not direct class attribute: >>> class X: .... def f():pass .... g=f .... l= .... >>> X.g <unbound method X.f>
5
2339
by: Lukas Holcik | last post by:
Hi everyone! How can I simply search text for regexps (lets say <a href="(.*?)">(.*?)</a>) and save all URLs(1) and link contents(2) in a dictionary { name : URL}? In a single pass if it could. Or how can I replace the html &entities; in a string "blablabla&amp;blablabal&amp;balbalbal" with the chars they mean using re.sub? I found out they are...
8
4572
by: Joakim Persson | last post by:
Hello all. I am involved in a project where we have a desire to improve our software testing tools, and I'm in charge of looking for solutions regarding the logging of our software (originating from embedded devices). Currently, we are using a heavyweight, proprietary log tool developed by another part of the company. This tool contains all...
4
3344
by: Noah | last post by:
I have a dictionary that I would like to expand to satisfy a function's agument list. I can used the ** syntax to pass a dictionary, but this only works if each key in the dictionary matches an argument. I cannot pass a dictionary that has more keys than the function has arguments. # Example 1 - This works: # Prints "hello world!" def...
7
20958
by: David Bear | last post by:
I have a dictionary that contains a row of data intended for a data base. The dictionary keys are the field names. The values are the values to be inserted. I am looking for a good pythonic way of expressing this, but I have a problem with the way lists are represented when converted to strings. Lets say my dictionary is
4
1505
by: 63q2o4i02 | last post by:
Hi, I'm writing a hand-written recursive decent parser for SPICE syntax parsing. In one case I have one function that handles a bunch of similar cases (you pass the name and the number of tokens you're looking for). In another case I have a function that handles a different set of tokens and so it can't use the same arguments as the...
70
27343
by: jojoba | last post by:
Hello! Does anyone know how to find the name of a python data type. Conside a dictionary: Banana = {} Then, how do i ask python for a string representing the name of the above dictionary (i.e. 'Banana')?
5
1942
by: Bill Jackson | last post by:
What is the benefit of clearing a dictionary, when you can just reassign it as empty? Similarly, suppose I generate a new dictionary b, and need to have it accessible from a. What is the best method, under which circumstances? .... a = b
0
2036
by: napolpie | last post by:
DISCUSSION IN USER nappie writes: Hello, I'm Peter and I'm new in python codying and I'm using parsying to extract data from one meteo Arpege file. This file is long file and it's composed by word and number arguments like this: GRILLE EURAT5 Coin Nord-Ouest : 46.50/ 0.50 Coin Sud-E Hello, I'm Peter and I'm new in python codying and I'm...
0
7580
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
1
7634
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7945
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
5208
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3634
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3618
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2079
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1194
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
916
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.