Hi,
I need a little help here managing child processes in Python. I'm not so
skilled in system programming so I hope you can give me some good
suggestions.
I have a very CPU and memory intensive task that has to be repeated many
times (with different input parameters). So I've thought to write a
Python script that create a child process, wait for it to finish and
then starts again another child with different parameters.
The children processes will save all the results to the disk (using
cPickle module), so the parent process has to wait each child only and
then starts the next one. The problem is that I can't work out a
solution. Sorry, but system programming is not my job.
Any suggestions about how to realize that? A skeleton of the script will
just suffice.
Thanks in advance,
Andrea.
2  1582 
Unless you are going to start more than one child
process in parallel and there is enough I/O to
make it worthwhile, there's no reason for using
child processes at all. Just program the application
as a single loop. You can't speed up CPU bound
applications with child processes.
HTH,
Larry Bates
Syscon, Inc.
"Rech" <re**@MUORISPAM fastwebnet.it> wrote in message
news:re******** *************** *@individual.ne t... Hi,
I need a little help here managing child processes in Python. I'm not so
skilled in system programming so I hope you can give me some good
suggestions.
I have a very CPU and memory intensive task that has to be repeated many
times (with different input parameters). So I've thought to write a
Python script that create a child process, wait for it to finish and
then starts again another child with different parameters.
The children processes will save all the results to the disk (using
cPickle module), so the parent process has to wait each child only and
then starts the next one. The problem is that I can't work out a
solution. Sorry, but system programming is not my job.
Any suggestions about how to realize that? A skeleton of the script will
just suffice.
Thanks in advance,
Andrea.
Yes I know, but the CPU-intensive tasks eat a lot of memory and for
weird reasons they don't free it (I know bad programming, but re-writing
all the code is impratical at this point). Running them in child
processes solves the problem because when a child dies it frees all the
allocated memory.
Andrea.
In article <vo************ ********@comcas t.com>,
"Larry Bates" <lb****@swamiso ft.com> wrote: Unless you are going to start more than one child
process in parallel and there is enough I/O to
make it worthwhile, there's no reason for using
child processes at all. Just program the application
as a single loop. You can't speed up CPU bound
applications with child processes.
HTH,
Larry Bates
Syscon, Inc. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: mikegw |
last post by:
I would appreciate some help to get started so I can go off and be able to
return with more intelligent questions.
My child processes will not need to speak to each other, they will only need
to return the calculation results and exit. I understand that all the child
processes will have to complete before the code can move to the next time
step. So with that in mind I expect that I will have to ensure that all the
children finish their...
|
by: Wizou |
last post by:
1. Create a TcpListener
2. Start a child Process
3. Kill the parent process
=> You can't bind to the port until you close the child process
Variant :
3. Terminate normally the parent process
=> The parent process stays as "Running" until the child process is
closed
|
by: Wizou |
last post by:
i'm using .NET 2.0, and i've made a lot of tests
i've come to the conclusion that TCP servers (tcplistener), started by
a father process, are somewhat inherited by child processes if using
UseShellExecute = false
why is that ?
try it ! even if the TcpListener is started *after* the
childProcess.Start, the TCP server is still registered in netstat -ao
and connectable with telnet, even if the father process stopped (or
|
by: preethamkumark |
last post by:
- The program first creates a shared memory buffer containing an array
of 20 integers.
- Each slot of the buffer can have either 0 or 1, where 0 represents an
empty slot, and 1
represents an occupied one.
- Initially, the buffer is empty. Thus, all the slots are initialized
with 0.
|
by: John Nagle |
last post by:
The major complaint I have about Python is that the packages
which connect it to other software components all seem to have
serious problems. As long as you don't need to talk to anything
outside the Python world, you're fine. But once you do, things
go downhill. MySQLdb has version and platform compatibility
problems. So does M2Crypto. The built-in SSL support is weak.
Even basic sockets don't quite work right; the socket module...
| | |
by: revenant81 |
last post by:
I'm writing a program which has to execute a command, get its output
and show it on a treeview.
This command runs for a very long time.
I want to end the execution of the command when the user closes my
application.
Right now I'm using an object my_child of type subprocess.Popen to
execute the command, inside a thread with an infinite loop where we
constantly ask for its output.
|
by: jewel87 |
last post by:
Hi everyone!
I'm writing some code in C under UNIX, which should give some output like this:
PARENT: pid = 10063
CHILD: my pid = 10064
CHILD: my parent's pid = 10063
CHILD: Sleeping...
PARENT: my child's pid = 10064
PARENT: Waiting for the child to exit...
CHILD: Done sleeping...
PARENT: child is dead.
|
by: manontheedge |
last post by:
I'm trying to write a program in C on Linux where the child processes get their process images replaced. I have in my code, a few child processes that I forked from a single parent process. I'm then attempting to replace their ( the child ) process images with separate executables. I have these executables written and placed in the directory.
My problem is when I'm using execlp. When it reaches execlp in my code, it does replace the...
|
by: rocco.rossi |
last post by:
I'm using the Python processing module. I've just run into a problem
though. Actually, it's a more general problem that isn't specific to
this module, but to the handling of Unix (Linux processes) in general.
Suppose for instance that for some reason or another, after forking
several child processes, the main process terminates or gets killed
(or segfaults or whatever) and the child processes are orphaned. Is
there any way to automatically...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
