472,804 Members | 800 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,804 software developers and data experts.

Python/C and PYTHONPATH

Hi,

I've been experimenting on embedding Python to a C software, and ran into
a little problem with PYTHONPATH (I'm running on linux). Here's the deal:

When trying to call PyImport_Import("xyz"), Python returns an error "No
module named xyz". The problem _seems_ to be that I had no PYTHONPATH
variable defined (though python-shell works ok regardless), since the
following in bash helps:

<try, no go>
PYTHONPATH=
export PYTHONPATH
<try, works ok>

I'm not defining anything as the PATH, and now it works. Why doesn't it
look from "." without a dummy PYTHONPATH? Or is this actually a bug fixed
in a newer release (running 2.3.?)?

What is the "official" way to solve this? PySetProgramName()? Shell
script?

Thanks,
- Tero

--

Jul 18 '05 #1
8 2982
You'd need to tell us how you're invoking Python. When the Python
executable is invoked without arguments, the current directory is on the
sys.path:

$ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import sys
sys.path[0] ''

When you run a script, the directory of that script is in sys.path[0]:
$ echo "import sys; print sys.path[0]" > /tmp/script.py
$ python /tmp/script.py
/tmp

Some Python editors may put the directory where they reside on sys.path,
but not the current directory. They might do other startup that is
different from normal. The newest Python I have on this machine
is 2.3b1, and its idle puts these elements at the start of Pythonpath:

Python 2.3b1 (#16, May 19 2003, 10:22:28)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help import sys
sys.path[:2]

['/home/jepler', '/usr/local/bin']

.... so instead of "the current directory" ('' or '.'), sys.path[0]
is "the directory idle was invoked from", and that's followed by "the
directory where the idle script lives".

Jeff

Jul 18 '05 #2
Jeff Epler <je****@unpythonic.net> wrote:
You'd need to tell us how you're invoking Python. When the Python
Take a look at:
http://www.python.org/doc/current/ex...embedding.html. Basically,
it's about same (and I tested with it too, same results: failing)
executable is invoked without arguments, the current directory is on the
sys.path: $ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import sys
sys.path[0]
''

When you run a script, the directory of that script is in sys.path[0]:
$ echo "import sys; print sys.path[0]" > /tmp/script.py
$ python /tmp/script.py
/tmp


If I run "python" as a shell (python <scriptname>), everything works ok.
Says path is ok: [ '', '<thelib>', ... ]. However, from C, something works
differently: PyRun_SimpleString("import sys\nprint sys.path"); Shows no ''
(the rest is ok). The question is: Why is there no '', and what is the
"official/safe" way to set it? Just insert it to the list? (Setting
"PYTHONPATH= " helps, but I'd like it to run "off the box")

Tero

Jul 18 '05 #3

Fair warning: At this point, I couldn't even get the example you quoted
to link properly. Of course, I didn't dig too deep, as I'm in a hurry.

throw this c code into a .h file (my name for it is envdump.h)

---code---
#include <stdio.h>

extern char **environ;

void fdump_env(FILE *fd)
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
fprintf(fd, "%i : %s\n", ptr[i]);
i++;
}
}
void dump_env()
{
char **ptr;
int i = 0;
ptr = environ;
while(ptr[i] != NULL)
{
printf("%i : %s\n", i, ptr[i]);
i++;
}
}
---code---

Now, just inside the main function (or, perhaps, wherever you please, but
I'd try the line right before the first if statement) call dump_env() and
it will spit out all the environment variables that your C program
received. You might find something there that's of interest.

Or, call setenv ("man 3 setenv") to manually set a python-path.
you can use that in combo with getenv ("man 3 getenv") to append to an
existing PYTHONPATH, if you so choose.

Give it a go, please let me know how it turns out. If this doesn't shed
some light on the matter I might have another idea once I'm not rushing
around.

HTH

Sam Walters

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #4
Samuel Walters <sw*************@yahoo.com> wrote:
Fair warning: At this point, I couldn't even get the example you quoted
to link properly. Of course, I didn't dig too deep, as I'm in a hurry. ---code---
.... snip ...
Or, call setenv ("man 3 setenv") to manually set a python-path.
you can use that in combo with getenv ("man 3 getenv") to append to an
existing PYTHONPATH, if you so choose. Give it a go, please let me know how it turns out. If this doesn't shed
some light on the matter I might have another idea once I'm not rushing
around.


(Sorry for the delay, I was offline)

I'll see if it's actually the C-part that's causing problems, but I
worked it around by adding:

PyRun_SimpleString("import sys\nsys.path.insert(0,'')");

right after the Py_Initialize().

Works ok. Guess I'm "allowed" to do that(?)

Thanks,
- Tero

--

Jul 18 '05 #5
|Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
2004 22:39:19 +0000|
I'll see if it's actually the C-part that's causing problems, but I worked
it around by adding:

PyRun_SimpleString("import sys\nsys.path.insert(0,'')");

right after the Py_Initialize().

Works ok. Guess I'm "allowed" to do that(?)

IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
ham-handed. try this snippet:

---untested code---
#include <string.h>
#include <stdlib.h>
/* prototyping may not be neccessary... dunno...*/
extern char *getenv(const char *name);
extern char *setenv(const char *name, const char *value, int overwrite);
/* comment them out if gcc complains about redeclarations. */

/* append something to pythonpath */
/* returns 0 on failure, 1 on creation of the env variable and 2 on an append
oh, and -1 in cases of catastrophic miscompilation */
int myPyPathAppend(char *extrapath)
{
char *buffer = NULL;
/* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
char eventual_path[1024] = { 0 };
/* take note: after getenv, buffer points to an external constant character buffer.
do NOT try to modify it directly. use strcpy(char *dest, char *src)
(he says knowingly... is she into photography mate?)
*/
if( (buffer = getenv("PYTHONPATH")) == NULL )
{
/* we're here because PYTHONPATH is not already part of the environment. */

setenv("PYTHONPATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONPATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 1;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual_path, buffer);
strcat(eventual_path, ";");
strcat(eventual_path, extrapath);

setenv("PYTHONPATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONPATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */

/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual_path, buffer);
strcat(eventual_path, ";");
strcat(eventual_path, extrapath);

setenv("PYTHONPATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONPATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
return -1; /* deader code... should *really* never reach here */
}
---untested code---
I haven't tested, compiled or even read through this code.
I'm late for a party and still added comments
That means you get punctuation patrol :-P
Check the semicolons, check the braces
Hey, I hear that in some companies they call this teamwork methodology
"extreme-programming" We're buzzword compliant!

HTH
(Danm... I'm such a code monkey)

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #6
Samuel Walters <sw*************@yahoo.com> wrote:
|Thus Spake Tero Pihlajakoski On the now historical date of Mon, 05 Jan
2004 22:39:19 +0000|
I'll see if it's actually the C-part that's causing problems, but I worked
it around by adding:

PyRun_SimpleString("import sys\nsys.path.insert(0,'')");

right after the Py_Initialize().

Works ok. Guess I'm "allowed" to do that(?) IMLK (In My Limited Knowledge) that seems okay, but it also feels a bit
ham-handed. try this snippet:


Ok, there are comments here, somewhere:
---untested code---
#include <string.h>
#include <stdlib.h>
/* prototyping may not be neccessary... dunno...*/
extern char *getenv(const char *name);
extern char *setenv(const char *name, const char *value, int overwrite);
/* comment them out if gcc complains about redeclarations. */ /* append something to pythonpath */
/* returns 0 on failure, 1 on creation of the env variable and 2 on an append
oh, and -1 in cases of catastrophic miscompilation */
int myPyPathAppend(char *extrapath)
{
char *buffer = NULL;
/* size to taste... You should do this dynamically to avoid future buffer overrun attacks*/
char eventual_path[1024] = { 0 };
/* take note: after getenv, buffer points to an external constant character buffer.
do NOT try to modify it directly. use strcpy(char *dest, char *src)
(he says knowingly... is she into photography mate?)
*/
if( (buffer = getenv("PYTHONPATH")) == NULL )
{
/* we're here because PYTHONPATH is not already part of the environment. */

setenv("PYTHONPATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONPATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 1;
}
return -1; /* dead code... should never reach here */
}
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */
Here. I might not want to add ';' or ':', depending on the OS (probably
not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
want to run it on a new system, I'd have to find out the delimiter... It
also needs a buffer underrun check on that [1024]. I'll stick with the
PyRun_... for now, but I'll definitely save this code, so thanks. Again.

Also, found this piece from sys.path docs (now that my net is up and
running):
... "A program is free to modify this (sys.path) list for its own
purposes." ...


/* find the "=" in the buffer...
from string.h
extern char *strstr (__const char *__haystack, __const char *__needle)
there's a better way to do this, but I can't recall the function off the top of my head
*/
buffer = strstr(buffer, "=") + 1; /* +1 because buffer points to the equals. we want the string starting after it. */

/* copy the old PYTHONPATH string */
strcpy(eventual_path, buffer);
strcat(eventual_path, ";");
strcat(eventual_path, extrapath); setenv("PYTHONPATH", extrapath, 1); /* the last argument makes sure that we create the env var*/
/* did it go happen .. you should check this more rigorously*/
if( (buffer = getenv("PYTHONPATH")) == NULL)
{
/* we failed... abend. */
return 0;
}
else
{
/* success! cheers! */
return 2;
}
return -1; /* dead code... should never reach here */
}
One if and two elses, have you started "getting ready" for the party
already? ;) Or maybe it's fuzzy logic ;)
else
{
/* PYTHONPATH already exists. append ';', then our new path and update it. */
/* find the "=" in the buffer...
.... snip ...
}
return -1; /* deader code... should *really* never reach here */
}
---untested code---
I haven't tested, compiled or even read through this code.
I'm late for a party and still added comments
That means you get punctuation patrol :-P
Check the semicolons, check the braces
Hey, I hear that in some companies they call this teamwork methodology
"extreme-programming" We're buzzword compliant!


- Tero

--

Jul 18 '05 #7
On Mon, Jan 05, 2004 at 04:17:52PM +0000, Tero Pihlajakoski wrote:
Jeff Epler <je****@unpythonic.net> wrote:
You'd need to tell us how you're invoking Python. When the Python


Take a look at:
http://www.python.org/doc/current/ex...embedding.html. Basically,
it's about same (and I tested with it too, same results: failing)


Here's the program I came up with:

#include <Python.h>

int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]);
Py_Initialize();
PyRun_SimpleString("import sys; print sys.path, sys.executable");
PySys_SetArgv(argc-1, argv+1);
PyRun_SimpleString("import sys; print sys.path, sys.executable");
Py_Finalize();
return 0;
}

This program demonstrates that the initial sys.path element
corresponding to the location of the script is created when
PySys_SetArgv() is called:

$ ./embed
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

That first element depends on the location of the script, as shown here:

$ ./embed /tmp/x
['/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]
['/tmp', '/usr/lib/python2.2', '/usr/lib/python2.2/plat-linux2', ...]

I don't know where this is documented---PySys_SetArgv is mentioned in
the "api" document, but this side-effect is not:
http://www.python.org/doc/current/ap...ng.html#l2h-35
http://www.python.org/doc/current/ap...n.html#l3h-713
You might want to explore just what PySys_SetArgv and submit a
documentation patch on sourceforge, both for the api document and for
the ext document. Interestingly, calling PySys_SetArgv multiple times
inserts multiple items in sys.path.

Jeff

Jul 18 '05 #8
Ack... definitely not pep-7 compliant code

And to think... A co-worker once called me "obsessive" about indention.
(They say "obsessive," I say "consistent and legible.")

Well, It seems PAN likes to perform a stealth word-wrap when you edit from
the non-attached editor. (I used vim because I'm comfy hammering out code
there) Nope, on second look, I must have accidentally mashed some keys
while in vim. That extraneous "else" block is just a copy of the "else"
block.

If you run it through indent, and clean up a couple of oddball line-wraps
it will at least be legible. Remove the unneeded prototypes, embarrassing
extra else block, and it will compile. Remove the whole block about
searching for the = and fix the typo in the second setenv call (extrapath
should be eventual_path) and it will run without segfault. Then replace
the ";" with ":" and it will run properly on a linux system. For myself,
I made the snippet a little more general and added the option of choosing
which pathlist env variable and changed the function name to
envPathAppend. Then it's not a bad little snippet.
|Thus Spake Tero Pihlajakoski On the now historical date of Tue, 06 Jan
2004 01:04:31 +0000|
Here. I might not want to add ';' or ':', depending on the OS (probably
not)? I can solve this with #ifdefs for WIN32 and Linux, but everytime I
want to run it on a new system, I'd have to find out the delimiter... It
also needs a buffer underrun check on that [1024]. I'll stick with the
PyRun_... for now, but I'll definitely save this code, so thanks. Again.
That buffer overrun is not the only bit of braindead logic in that
snippet. The "success/failure" condition testing leaves a couple of loose
ends.
Also, found this piece from sys.path docs (now that my net is up and
running):
... "A program is free to modify this (sys.path) list for its own
purposes." ...


Hmmm... I suppose it's a matter of aesthetics and scope of the project
you're working on. If you're writing a big program for which the python
interpretor is only one small or medium part of the functionality and the
majority is c/c++, then you're opening up a whole big realm of
cross-platform issues. However, if the bulk of your code is python, then
perhaps you should have the main program be in python with your extra c
code as a module. That would be cleaner and probably more portable.

I suspect that at this point you're just trying to tinker with the
interface, then use what's most comfortable. Now at least, you have an
option. Besides, it seems that someone already posted a cleaner fix than
I could have given you.

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

20
by: BJ MacNevin | last post by:
Hi all, I teach middle school and am currently trying to bring some computer science to the students. Our district has a wonderfully linked network throughout all our schools... done via MS...
3
by: Stephen Ferg | last post by:
I need a little help here. I'm developing some introductory material on Python for non-programmers. The first draft includes this statement. Is this correct? ...
1
by: D.S. Hein | last post by:
I have installed python 2.3 on my Windows 2K laptop and I can run python from the command prompt. I have tried various ways to set PYTHONPATH with the various directories where I have python...
3
by: jg.campbell.ng | last post by:
I'm beginning learning Python and OpenGL in Python. Python fine. But difficulties with OpenGL; presumably with the installation of OpenGL. OS = Linux FC5. Python program gl_test.py: from...
18
by: cjl | last post by:
Hey: I am trying to run python from a usb drive under windows xp. I installed python "for this user" on to a machine, then copied the entire Python24 directory to the usb drive. I have the...
1
by: yinglcs | last post by:
Hi, i have a simple test which tries to load 'svn' moddule. # python -c "from svn import client" Traceback (most recent call last): File "<string>", line 1, in ? ImportError: No module named...
12
by: pistacchio | last post by:
hi! i'm a php user and a python programmer. i'd love to use python for my server side needs but i can't seem to find what i'm looking for. for most of my php work i use mysql and tinyButStrong...
0
by: mathieu | last post by:
Hi there, I am currently involved in managing a large python/c++ project (using swig to automagically wrap c++ code to python). So far my current experience was that python was a second class...
7
by: greg | last post by:
Thomas Philips wrote: Have a look in /Library/Frameworks/Python.framework/Versions/2.5 You can't -- this feature only exists in the Search window, which is a different kind of window...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 2 August 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
by: erikbower65 | last post by:
Using CodiumAI's pr-agent is simple and powerful. Follow these steps: 1. Install CodiumAI CLI: Ensure Node.js is installed, then run 'npm install -g codiumai' in the terminal. 2. Connect to...
0
linyimin
by: linyimin | last post by:
Spring Startup Analyzer generates an interactive Spring application startup report that lets you understand what contributes to the application startup time and helps to optimize it. Support for...
0
by: erikbower65 | last post by:
Here's a concise step-by-step guide for manually installing IntelliJ IDEA: 1. Download: Visit the official JetBrains website and download the IntelliJ IDEA Community or Ultimate edition based on...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Sept 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
by: Rina0 | last post by:
I am looking for a Python code to find the longest common subsequence of two strings. I found this blog post that describes the length of longest common subsequence problem and provides a solution in...
5
by: DJRhino | last post by:
Private Sub CboDrawingID_BeforeUpdate(Cancel As Integer) If = 310029923 Or 310030138 Or 310030152 Or 310030346 Or 310030348 Or _ 310030356 Or 310030359 Or 310030362 Or...
0
by: lllomh | last post by:
How does React native implement an English player?
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.