473,835 Members | 2,287 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Embedded Python interpreter and sockets

Hi,

Here is a problem I came across and need some help with. I developed a
little Python script with some classes which runs standalone and
communicates with a database via sockets. So far everything works fine.
I also successfully embedded the Python interpreter into a plugin for a
commercial 3D package (in this case Houdini - http://www.sidefx.com/ -
using the HDK). I can use the interpreter to run Python commands and
get values back and forth between Python and C++. Anyway, I can also
successfully import SOME modules, but unfortunately not ALL. One
example would be the socket module which works fine in the standalone
version of my Python script but NOT from the embedded interpreter:

Traceback (most recent call last):
File "<string>", line 40, in ?
File "/swdevl/jwalter/Houdini/ROPS/HoudiniToMayaBa ke/damn.py", line
10, in ?
import socket
File
"/job/SYSTEMS/3PS/python/2.4.1/linux_intel//lib/python2.4/socket.py",
line 45, in ?
import _socket
ImportError:
/job/SYSTEMS/3PS/python/2.4.1/linux_intel/lib/python2.4/lib-dynload/_socket.so:
undefined symbol: PyObject_Generi cGetAttr

Other modules which get imported before just work fine (e.g. re and
string) ...

The Python library gets linked into the plugin DSO as a static library
.... Any ideas what's going wrong and why?

Cheers,

Jan

Nov 22 '05 #1
5 3236
Hi Jan,

I believe the problem lies with how Houdini uses dlopen() to open your
plugin. It uses RTLD_LOCAL to load your plugin, which means that all
your plugin's symbols (including the python symbols) are private to
that library. Subsequent dlopen() calls, including those made by the
python library, won't be able to see any of those symbols. This
problem isn't unique to Python, or Houdini, it's just a matter of how
dlopen() is being used.

One thing that may work (I just tried here, and it looks like it
does...but it's late on a Friday night, so I may have overlooked
something) is creating a wrapper plugin. Say you have your plugin
called myplugin.so which links aginst libpython. Create a new plugin
called myplugin_wrappe r.so which exports the same required symbols.
myplugin_wrappe r will need to dlopen("myplugi n.so", RTLD_GLOBAL), and
then call the real methods inside myplugin.so. Since myplugin.so is
the one linked against libpython, and it is being loaded with
RTLD_GLOBAL, then the python symbols should be available to other
shared libraries loaded at a future time.

Let me know if this works out! If you need any help, head on over to
http://www.sidefx.com/forum.

Cheers,
Chris

Nov 22 '05 #2
Hi Jan,

I believe the problem lies with how Houdini uses dlopen() to open your
plugin. It uses RTLD_LOCAL to load your plugin, which means that all
your plugin's symbols (including the python symbols) are private to
that library. Subsequent dlopen() calls, including those made by the
python library, won't be able to see any of those symbols. This
problem isn't unique to Python, or Houdini, it's just a matter of how
dlopen() is being used.

One thing that may work (I just tried here, and it looks like it
does...but it's late on a Friday night, so I may have overlooked
something) is creating a wrapper plugin. Say you have your plugin
called myplugin.so which links aginst libpython. Create a new plugin
called myplugin_wrappe r.so which exports the same required symbols.
myplugin_wrappe r will need to dlopen("myplugi n.so", RTLD_GLOBAL), and
then call the real methods inside myplugin.so. Since myplugin.so is
the one linked against libpython, and it is being loaded with
RTLD_GLOBAL, then the python symbols should be available to other
shared libraries loaded at a future time.

Let me know if this works out! If you need any help, head on over to
http://www.sidefx.com/forum.

Cheers,
Chris

Nov 22 '05 #3
Hi Chris,

Thanks for your help. I'll try that ...

Cheers,

Jan

Nov 22 '05 #4
Hey Chris,

I fixed the problem in another way (don't ask me why that works). One
detail I didn't talk about is that I use the Boost.Python library. So I
just made sure that I load the socket module before I import my own
Python script (using that socket module):

....
object
main_module((ha ndle<>(borrowed (PyImport_AddMo dule("__main__" )))));
object
socket_module(( handle<>(borrow ed(PyImport_Add Module("socket" )))));
object main_namespace = main_module.att r("__dict__") ;
handle<>
result((allow_n ull(PyRun_Strin g("...",
Py_file_input,
main_namespace. ptr(),
main_namespace. ptr()))));
....

Maybe someone can shed some light on what's going on here but for the
moment I'm happy that it works at all !!!

Cheers,

Jan

Nov 22 '05 #5
Hi,

actually that didn't solve the problem. As soon as you do something
with the socket module it fails. Well, the solution I came up with is
simply link the ../_socket.so into my Houdini plugin DSO which is ugly
but solves the problem for the moment ...

Happy hacking,

Jan

Nov 23 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2156
by: quadric | last post by:
Hi, I have an application that has an embedded/extended Python interpreter. I need to add database capabilities and for reasons to lengthy to explain in this email, also require an embedded database server. I have tentatively chosen mySQL 4.0 due to the apparent ease of embedding, along with it having all the necessary database functionality I need.
7
2472
by: Chris | last post by:
Hi I am posting this on both the perl and python groups My intention is not to start a war or anything else, I would just like some pragmatic advice. My apologies to the python group I am not very conversant with Python at this stage. that could change soon though.
4
1832
by: Paul Miller | last post by:
Some background first - we have some software that embeds a Python interpreter into a host application. Scripts are loaded dynamically and used. But we want to support the ability to edit scripts while the app is running. This means we have to "unload" the script and cause a reload. Normal module reloading tricks don't work because if you try to reimport a script that imports another script, and if the nested script is changed, it might...
0
303
by: wahn | last post by:
Hi, Here is a problem I came across and need some help with. I developed a little Python script with some classes which runs standalone and communicates with a database via sockets. So far everything works fine. I also successfully embedded the Python interpreter into a plugin for a commercial 3D package (in this case Houdini - http://www.sidefx.com/ - using the HDK). I can use the interpreter to run Python commands and get values back...
20
22554
by: Jack | last post by:
Is there a Python packaging that is specifically for embedded systems? ie, very small and configurable so the user gets to select what modules to install? For Linux-based embedded systems in particular? I'm thinking of running it on the Linksys's Linux-based open source router WRT54G. It has 4MB flash and 16MB RAM. I think another model has 16MB flash. Any possibilities of running Python on these systems?
0
287
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 406 open (-10) / 3479 closed (+16) / 3885 total ( +6) Bugs : 931 open ( +1) / 6349 closed (+16) / 7280 total (+17) RFE : 245 open ( +1) / 244 closed ( +0) / 489 total ( +1) New / Reopened Patches ______________________
3
2021
by: ycollet | last post by:
Hello, I've written a C embedded application. I want to open a python gui application in my C program but when I do : PyRun_String( "import gui.py", file_input, pDictionary, pDictionary ); the interpreter emits an error: tkinter module not defined What script must I load to initialize the embedded python interpreter
0
1013
by: Brett C. | last post by:
I have been working on making Python a secure interpreter to run when embedded in terms of resources with an object representation (e.g., files but not memory or CPU). To save myself from repeating myself and missing some details of exactly what I have done I am going to point to a post to my Python blog (and I have replied to people in the comments so if you have a question it may have been answered there):...
4
2840
by: Chris8Boyd | last post by:
I am embedding Python in a MSVC++ (2005) application. The application creates some environment and then launches a Python script that will call some functions exported from the MSVC++ application. I want to be able to debug the Python script by using a debug server, like Winpdb (winpdb.org). I use ActivePython 2.5.2.2, Microsoft Visual Studio 2005, and Winpdb 1.3.8.
0
9803
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9652
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10811
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10523
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
7766
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6966
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5636
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
3993
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3088
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.