Using Multi-core Machine

[stuartornum]

Hi,

I have a large amount of processing of files to do. About 20,000+ files need processed (Filepath in one big list).

Ideally what I would like to do is find out how many cores are available, split the list up in the number of cores, send each newly created list as new processes, and hopefully each core will pick up one process.

So far I have split the list up....

I am stuck on execute a function and leaving it running, I do not care about the return values, I just want to initiate say 4 processes all at once.

I have looked at os.fork() but there isn't much documentation about.

Thanks

[Laharl]

The reason there isn't much documentation about os.fork() is that it's just a Python wrapper to the C function of the same name (on *nix, at least), which is exhaustively documented.

You also may find the os.popen() line of functions useful, which will likely have similar docs.

[Subsciber123]

You might want to use threads. They work as follows:

thread.start_new_thread(function, argument_tuple [,keyword_dictionary])

This will either return None (successful) or raise an exception if it is unable to start a new thread.