473,385 Members | 2,029 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Reading variables from a forked child (Python C/API)

Greetings,
I'm having a little trouble getting an idea running. I am writing a C
program which is really a frontend to a Python program. Now, my C
program starts up, does some initialisation like initialisation of it's
variables and Py_Initialize() and then it fork()s. After forking the
child launches the python program with Py_Main() and with the parent I
want to be able to read the variables of the Python program. I have
tried many ways but cannot find a solution to do this. The ONLY testcase
in fact that I could get working is this:

#include <Python.h>
#include <stdio.h>

int main(){
char* buffer;
PyObject* mod;
PyObject* dict;
PyObject* obj;

Py_Initialize();

PyRun_SimpleString("foo = 'bar'");
mod = PyImport_AddModule("__main__");
dict = PyModule_GetDict(mod);
obj = PyMapping_GetItemString(dict, "foo");
buffer = PyString_AsString(obj);
printf(buffer);
Py_Finalize();
return 0;
}

And that only proves to me that I can get the variables out of python,
nothing to do with forking. When i try doing very much the same thing to
a forked child I get sedfaults. In other words the child gets
Py_Main()'d and executes the child, while in the parent i do much the
same as the above with the exception of PyImport_AddModule() being
changed to PyImport_ImportModule() because main is already preset i gather.

So my questions are these:

Does the fact that I initialise before the fork have anything to do with
my failure? And if so, is it because some pointers are invalidated?

Is the problem to do with the inability to perform operations on a
python environment initialised through C while it's executing? Because
i'm technically executing operations on it from the parent while the
child in the same environment is running the program.

In the program that the child is running, i need to access some
variables it has in __main__. main also has an instantiated object 'd'
which i also need some variables from, i.e. __main__.d.foo. How can I
access these?

Overall, I would REALLY appreciate an example of how to do what I want
to achieve, examples help me a lot so if you wouldn't mind typing one up
or pointing me to some relevant documentation i'd appreciate it.

Thankyou.
Jul 21 '05 #1
2 2324
Quoth MrEntropy <fa**@doesntexist.dud>:

| I'm having a little trouble getting an idea running. I am writing a C
| program which is really a frontend to a Python program. Now, my C
| program starts up, does some initialisation like initialisation of it's
| variables and Py_Initialize() and then it fork()s. After forking the
| child launches the python program with Py_Main() and with the parent I
| want to be able to read the variables of the Python program. I have
| tried many ways but cannot find a solution to do this.

That's because there is no solution. After a fork, the only effect
observable in the parent process is the return value of fork, which
will have a non-zero value. Subsequent execution in the child process
occurs completely independently from the parent, and leaves no trace
whatever. So you can't read variables from memory, that were set by
the child.

Past that, I deleted the rest of your post because it sort of avoids
the question of what you're really trying to accomplish and what errors
you actually got, but note that just for the sake of avoiding segmentation
faults etc., it's a good idea when writing in C to check return values:

obj = PyMapping_GetItemString(dict, "foo");
if (obj) {
...
} else {
...
}

Anyway, if you really need a subprocess, you're going to have to
communicate with it via some sort of I/O, like UNIX pipes or temporary
files or something. You probably don't need to call Python from C,
may as well just invoke python (cf. os.spawnv)

Donn Cave, do**@drizzle.com
Jul 21 '05 #2
#ifdef def
Donn Cave wrote:
Quoth MrEntropy <fa**@doesntexist.dud>:

| I'm having a little trouble getting an idea running. I am writing a C
| program which is really a frontend to a Python program. Now, my C
| program starts up, does some initialisation like initialisation of it's
| variables and Py_Initialize() and then it fork()s. After forking the
| child launches the python program with Py_Main() and with the parent I
| want to be able to read the variables of the Python program. I have
| tried many ways but cannot find a solution to do this.

That's because there is no solution. After a fork, the only effect
observable in the parent process is the return value of fork, which
will have a non-zero value. Subsequent execution in the child process
occurs completely independently from the parent, and leaves no trace
whatever. So you can't read variables from memory, that were set by
the child.


Though there is probably some solution involving a third party
such as POSIX threads, Linux has a clone(2) procedure which
fork(2) is a macro of. It is tricky to use, but allows for
having the child process have the same pid, memory, memory
mappings, file tables, etc. BSD mmap(2) is more portable and can
be used for interprocess sharing. The solution which Don Cave
provided is probably the one that Mr. Entropy needs to start
with. More specifically, ...

#else

int main (int argc, char **argv)
{
int fd0, fd1, fd2;
{
int fd_in [2], fd_out [2], fd_err [2];
pipe (fd_in); fd0 = fd_in [1];
pipe (fd_out); fd1 = fd_out [0];
pipe (fd_err); fd2 = fd_err [0];
if (!fork ()) {
dup2 (fd_in [0], 0); close (fd0);
dup2 (fd_out [1], 1); close (fd1);
dup2 (fd_err [1], 2); close (fd2);
execl ("python", "python", 0);
perror ("execution failed");
exit (1);
}
}

write (fd0, "print None\n", 11);
write (1, b, read (fd1, b, sizeof (b) /* 5 or more */));

#define write0(q...) write (fd0, "print " #q "\n", \
sizeof (#q) + 5 + 1 - 1)
write0 (True); write (1, b, read (fd1, b, sizeof (b)));
write0 (5, str (5.0), ...);
write (1, b, read (fd1, b, sizeof (b)));
return write0 (True, False, None,
True and False,
True and None,
False and None,
True and False and None) && 0;
}

#endif
Jul 21 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

21
by: John Lin | last post by:
Howdy, I want to know how to tell if a forked process is done. Actually, my real question is that I want to run a shell script inside of a python script, and after the shell script has...
3
by: Greg Lindstrom | last post by:
Hello- I am running python 2.3. on an HP-9000 box running Unix and have a POSIX script that sets up my production environment. I would like to run the script from inside a python routine and...
2
by: RL | last post by:
Hello Perl gurus, 1. I have a web page where I can push a button (dospawn.html). 2. This button calls a CGI script (spawnboss.cgi) 3. spawnboss.cgi calls a forking perl script (forkme.pl) 4....
1
by: Alexander N. Spitzer | last post by:
I am trying to write a program that will fork a process, and execute the given task... the catch is that if it runs too long, I want to clean it up. this seemed pretty straight forward with a...
2
by: James Colannino | last post by:
Hey everyone. I'm writing a small application in Python that uses os.fork() to create a separate process in which another application is run in the background. The problem is that I need to know...
2
by: gregarican | last post by:
I have an application I'm writing using PyQt. I'm trying to create the various windows by subclassing Qt objects. I have a subclassed QMainWindow as the parent, and then a series of subclassed...
0
by: dan.jakubiec | last post by:
I'm trying to write a Python app which accepts a socket connection and then spawns another Python process to handle it. I need it to run under both Linux and Windows. I have it working under...
7
by: MD | last post by:
Hi, I would like to access "variables" defined in my Python program in a C module extension for Python. Is this possible? I looked at the Python C API reference but didn't find anything there...
1
by: scsoce | last post by:
A child thread has a long-time executions, how to suspend it and resume back the orignial place ? I know it' nature to use singal, but child thread cannot get signal as Python Manual say. And i...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.