473,739 Members | 2,602 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Python/C and PYTHONPATH

Hi,

I've been experimenting on embedding Python to a C software, and ran into
a little problem with PYTHONPATH (I'm running on linux). Here's the deal:

When trying to call PyImport_Import ("xyz"), Python returns an error "No
module named xyz". The problem _seems_ to be that I had no PYTHONPATH
variable defined (though python-shell works ok regardless), since the
following in bash helps:

<try, no go>
PYTHONPATH=
export PYTHONPATH
<try, works ok>

I'm not defining anything as the PATH, and now it works. Why doesn't it
look from "." without a dummy PYTHONPATH? Or is this actually a bug fixed
in a newer release (running 2.3.?)?

What is the "official" way to solve this? PySetProgramNam e()? Shell
script?

Thanks,
- Tero

--

Jul 18 '05 #1
8 3045
You'd need to tell us how you're invoking Python. When the Python
executable is invoked without arguments, the current directory is on the
sys.path:

$ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import sys
sys.path[0] ''

When you run a script, the directory of that script is in sys.path[0]:
$ echo "import sys; print sys.path[0]" > /tmp/script.py
$ python /tmp/script.py
/tmp

Some Python editors may put the directory where they reside on sys.path,
but not the current directory. They might do other startup that is
different from normal. The newest Python I have on this machine
is 2.3b1, and its idle puts these elements at the start of Pythonpath:

Python 2.3b1 (#16, May 19 2003, 10:22:28)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "copyright" , "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help import sys
sys.path[:2]

['/home/jepler', '/usr/local/bin']

.... so instead of "the current directory" ('' or '.'), sys.path[0]
is "the directory idle was invoked from", and that's followed by "the
directory where the idle script lives".

Jeff

Jul 18 '05 #2
Jeff Epler <je****@unpytho nic.net> wrote:
You'd need to tell us how you're invoking Python. When the Python
Take a look at:
http://www.python.org/doc/current/ex...embedding.html. Basically,
it's about same (and I tested with it too, same results: failing)
executable is invoked without arguments, the current directory is on the
sys.path: $ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import sys
sys.path[0]
''

When you run a script, the directory of that script is in sys.path[0]:
$ echo "import sys; print sys.path[0]" > /tmp/script.py
$ python /tmp/script.py
/tmp


If I run "python" as a shell (python <scriptname>) , everything works ok.
Says path is ok: [ '', '<thelib>', ... ]. However, from C, something works
differently: PyRun_SimpleStr ing("import sys\nprint sys.path"); Shows no ''
(the rest is ok). The question is: Why is there no '', and what is the
"official/safe" way to set it? Just insert it to the list? (Setting
"PYTHONPATH = " helps, but I'd like it to run "off the box")

Tero

Jul 18 '05 #3

Fair warning: At this point, I couldn't even get the example you quoted
to link properly. Of course, I didn't dig too deep, as I'm in a hurry.

throw this c code into a .h file (my name for it is envdump.h)

---code---
#include <stdio.h>

extern char **environ;

void fdump_env(FILE *fd)
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
fprintf(fd, "%i : %s\n", ptr[i]);
i++;
}
}
void dump_env()
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
printf("%i : %s\n", i, ptr[i]);
i++;
}
}
---code---

Now, just inside the main function (or, perhaps, wherever you please, but
I'd try the line right before the first if statement) call dump_env() and
it will spit out all the environment variables that your C program
received. You might find something there that's of interest.

Or, call setenv ("man 3 setenv") to manually set a python-path.
you can use that in combo with getenv ("man 3 getenv") to append to an
existing PYTHONPATH, if you so choose.

Give it a go, please let me know how it turns out. If this doesn't shed
some light on the matter I might have another idea once I'm not rushing
around.

HTH

Sam Walters

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #4
Samuel Walters <sw************ *@yahoo.com> wrote:
Fair warning: At this point, I couldn't even get the example you quoted
to link properly. Of course, I didn't dig too deep, as I'm in a hurry. ---code---
.... snip ...
Or, call setenv ("man 3 setenv") to manually set a python-path.
you can use that in combo with getenv ("man 3 getenv") to append to an
existing PYTHONPATH, if you so choose. Give it a go, please let me know how it turns out. If this doesn't shed
some light on the matter I might have another idea once I'm not rushing
around.


(Sorry for the delay, I was offline)

I'll see if it's actually the C-part that's causing problems, but I
worked it around by adding:

PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");

right after the Py_Initialize() .

Works ok. Guess I'm "allowed" to do that(?)

Thanks,
- Tero

--

Jul 18 '05 #5
|Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
2004 22:39:19 +0000|
I'll see if it's actually the C-part that's causing problems, but I worked
it around by adding:

PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");

right after the Py_Initialize() .

Works ok. Guess I'm "allowed" to do that(?)

IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
ham-handed. try this snippet:

---untested code---
#include <string.h>
#include <stdlib.h>
/* prototyping may not be neccessary... dunno...*/
extern char *getenv(const char *name);
extern char *setenv(const char *name, const char *value, int overwrite);
/* comment them out if gcc complains about redeclarations. */

/* append something to pythonpath */
/* returns 0 on failure, 1 on creation of the env variable and 2 on an append
oh, and -1 in cases of catastrophic miscompilation */
int myPyPathAppend( char *extrapath)
{
char *buffer = NULL;
/* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
char eventual_path[1024] = { 0 };
/* take note: after getenv, buffer points to an external constant character buffer.
do NOT try to modify it directly. use strcpy(char *dest, char *src)
(he says knowingly... is she into photography mate?)
*/
if( (buffer = getenv("PYTHONP ATH")) == NULL )
{
/* we're here because PYTHONPATH is not already part of the environment. */

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 1;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual _path, buffer);
strcat(eventual _path, ";");
strcat(eventual _path, extrapath);

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual _path, buffer);
strcat(eventual _path, ";");
strcat(eventual _path, extrapath);

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
return -1; /* deader code... should *really* never reach here */
}
---untested code---
I haven't tested, compiled or even read through this code.
I'm late for a party and still added comments
That means you get punctuation patrol :-P
Check the semicolons, check the braces
Hey, I hear that in some companies they call this teamwork methodology
"extreme-programming" We're buzzword compliant!

HTH
(Danm... I'm such a code monkey)

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #6
Samuel Walters <sw************ *@yahoo.com> wrote:
|Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
2004 22:39:19 +0000|
I'll see if it's actually the C-part that's causing problems, but I worked
it around by adding:

PyRun_SimpleStr ing("import sys\nsys.path.i nsert(0,'')");

right after the Py_Initialize() .

Works ok. Guess I'm "allowed" to do that(?) IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
ham-handed. try this snippet:


Ok, there are comments here, somewhere:
---untested code---
#include <string.h>
#include <stdlib.h>
/* prototyping may not be neccessary... dunno...*/
extern char *getenv(const char *name);
extern char *setenv(const char *name, const char *value, int overwrite);
/* comment them out if gcc complains about redeclarations. */ /* append something to pythonpath */
/* returns 0 on failure, 1 on creation of the env variable and 2 on an append
oh, and -1 in cases of catastrophic miscompilation */
int myPyPathAppend( char *extrapath)
{
char *buffer = NULL;
/* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
char eventual_path[1024] = { 0 };
/* take note: after getenv, buffer points to an external constant character buffer.
do NOT try to modify it directly. use strcpy(char *dest, char *src)
(he says knowingly... is she into photography mate?)
*/
if( (buffer = getenv("PYTHONP ATH")) == NULL )
{
/* we're here because PYTHONPATH is not already part of the environment. */

setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 1;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */
Here. I might not want to add ';' or ':', depending on the OS (probably
not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
want to run it on a new system, I'd have to find out the delimiter... It
also needs a buffer underrun check on that [1024]. I'll stick with the
PyRun_... for now, but I'll definitely save this code, so thanks. Again.

Also, found this piece from sys.path docs (now that my net is up and
running):
... "A program is free to modify this (sys.path) list for its own
purposes." ...


/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual _path, buffer);
strcat(eventual _path, ";");
strcat(eventual _path, extrapath); setenv("PYTHONP ATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONP ATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
One if and two elses, have you started "getting ready" for the party
already? ;) Or maybe it's fuzzy logic ;)
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */
/* find the "=" in the buffer...
.... snip ...
}
return -1; /* deader code... should *really* never reach here */
}
---untested code---
I haven't tested, compiled or even read through this code.
I'm late for a party and still added comments
That means you get punctuation patrol :-P
Check the semicolons, check the braces
Hey, I hear that in some companies they call this teamwork methodology
"extreme-programming" We're buzzword compliant!


- Tero

--

Jul 18 '05 #7
On Mon, Jan 05, 2004 at 04:17:52PM +0000, Tero Pihlajakoski wrote:
Jeff Epler <je****@unpytho nic.net> wrote:
You'd need to tell us how you're invoking Python. When the Python


Take a look at:
http://www.python.org/doc/current/ex...embedding.html. Basically,
it's about same (and I tested with it too, same results: failing)


Here's the program I came up with:

#include <Python.h>

int
main(int argc, char *argv[])
{
Py_SetProgramNa me(argv[0]);
Py_Initialize() ;
PyRun_SimpleStr ing("import sys; print sys.path, sys.executable" );
PySys_SetArgv(a rgc-1, argv+1);
PyRun_SimpleStr ing("import sys; print sys.path, sys.executable" );
Py_Finalize();
return 0;
}

This program demonstrates that the initial sys.path element
corresponding to the location of the script is created when
PySys_SetArgv() is called:

$ ./embed
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

That first element depends on the location of the script, as shown here:

$ ./embed /tmp/x
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['/tmp', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

I don't know where this is documented---PySys_SetArgv is mentioned in
the "api" document, but this side-effect is not:
http://www.python.org/doc/current/ap...ng.html#l2h-35
http://www.python.org/doc/current/ap...n.html#l3h-713
You might want to explore just what PySys_SetArgv and submit a
documentation patch on sourceforge, both for the api document and for
the ext document. Interestingly, calling PySys_SetArgv multiple times
inserts multiple items in sys.path.

Jeff

Jul 18 '05 #8
Ack... definitely not pep-7 compliant code

And to think... A co-worker once called me "obsessive" about indention.
(They say "obsessive, " I say "consistent and legible.")

Well, It seems PAN likes to perform a stealth word-wrap when you edit from
the non-attached editor. (I used vim because I'm comfy hammering out code
there) Nope, on second look, I must have accidentally mashed some keys
while in vim. That extraneous "else" block is just a copy of the "else"
block.

If you run it through indent, and clean up a couple of oddball line-wraps
it will at least be legible. Remove the unneeded prototypes, embarrassing
extra else block, and it will compile. Remove the whole block about
searching for the = and fix the typo in the second setenv call (extrapath
should be eventual_path) and it will run without segfault. Then replace
the ";" with ":" and it will run properly on a linux system. For myself,
I made the snippet a little more general and added the option of choosing
which pathlist env variable and changed the function name to
envPathAppend. Then it's not a bad little snippet.
|Thus Spake Tero Pihlajakoski On the now historical date of Tue, 06 Jan
2004 01:04:31 +0000|
Here. I might not want to add ';' or ':', depending on the OS (probably
not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
want to run it on a new system, I'd have to find out the delimiter... It
also needs a buffer underrun check on that [1024]. I'll stick with the
PyRun_... for now, but I'll definitely save this code, so thanks. Again.
That buffer overrun is not the only bit of braindead logic in that
snippet. The "success/failure" condition testing leaves a couple of loose
ends.
Also, found this piece from sys.path docs (now that my net is up and
running):
... "A program is free to modify this (sys.path) list for its own
purposes." ...


Hmmm... I suppose it's a matter of aesthetics and scope of the project
you're working on. If you're writing a big program for which the python
interpretor is only one small or medium part of the functionality and the
majority is c/c++, then you're opening up a whole big realm of
cross-platform issues. However, if the bulk of your code is python, then
perhaps you should have the main program be in python with your extra c
code as a module. That would be cleaner and probably more portable.

I suspect that at this point you're just trying to tinker with the
interface, then use what's most comfortable. Now at least, you have an
option. Besides, it seems that someone already posted a cleaner fix than
I could have given you.

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

20
3853
by: BJ MacNevin | last post by:
Hi all, I teach middle school and am currently trying to bring some computer science to the students. Our district has a wonderfully linked network throughout all our schools... done via MS Windows Network. In order to protect the network, our district's IT department does not want things installed on the system (or at least makes it VERY difficult to get it done). SO, I am using MSW Logo installed onto a CD-ROM... we just stick in the...
3
9732
by: Stephen Ferg | last post by:
I need a little help here. I'm developing some introductory material on Python for non-programmers. The first draft includes this statement. Is this correct? ----------------------------------------------------------------- When loading modules, Python looks for modules in the following places in the following order:
1
1201
by: D.S. Hein | last post by:
I have installed python 2.3 on my Windows 2K laptop and I can run python from the command prompt. I have tried various ways to set PYTHONPATH with the various directories where I have python programs (i.e. xxxxxx.py) but no matter how I set PYTHONPATH I keep getting a message that python can't open the python file. If I try to specify the full pathname by typing: python c:\Documents and Settings\-rest of path- I get an error message that...
3
9988
by: jg.campbell.ng | last post by:
I'm beginning learning Python and OpenGL in Python. Python fine. But difficulties with OpenGL; presumably with the installation of OpenGL. OS = Linux FC5. Python program gl_test.py: from OpenGL.GLUT import *
18
6776
by: cjl | last post by:
Hey: I am trying to run python from a usb drive under windows xp. I installed python "for this user" on to a machine, then copied the entire Python24 directory to the usb drive. I have the following in a batch file at the root of the drive: @path=%PATH%;%CD%Python24;%CD%Python24\libs;%CD%Python24\Scripts;%CD%Python24\Lib\site-packages;%CD%Python24\DLLs @set pythonpath = %CD%Python24
1
5069
by: yinglcs | last post by:
Hi, i have a simple test which tries to load 'svn' moddule. # python -c "from svn import client" Traceback (most recent call last): File "<string>", line 1, in ? ImportError: No module named svn I have checked I have sub-directories 'libsvn', 'svn' under
12
1411
by: pistacchio | last post by:
hi! i'm a php user and a python programmer. i'd love to use python for my server side needs but i can't seem to find what i'm looking for. for most of my php work i use mysql and tinyButStrong (http://www.tinybutstrong.com) which is a very lightweight template engine that offers powerful functionalities. you insert TBS tags in web pages like: <div align="center" class="title-page" </div> and it replaces with the value of global...
0
1052
by: mathieu | last post by:
Hi there, I am currently involved in managing a large python/c++ project (using swig to automagically wrap c++ code to python). So far my current experience was that python was a second class citizen and extremely little python code was written and everything seemed to work. Now this is the contrary, large portion of code are written python with a few expection of c++ code for core algorithms. So as you might
7
1628
by: greg | last post by:
Thomas Philips wrote: Have a look in /Library/Frameworks/Python.framework/Versions/2.5 You can't -- this feature only exists in the Search window, which is a different kind of window from the normal Finder
0
8969
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8792
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9479
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9337
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9266
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8215
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6754
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6054
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
2
2748
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.