Help | Site Map
Connecting Tech Pros Worldwide
Reply
 
LinkBack Thread Tools
  #1  
Old September 3rd, 2008, 12:30 PM
Newbie
 
Join Date: Sep 2008
Posts: 1
Default Using Multi-core Machine

Hi,

I have a large amount of processing of files to do. About 20,000+ files need processed (Filepath in one big list).

Ideally what I would like to do is find out how many cores are available, split the list up in the number of cores, send each newly created list as new processes, and hopefully each core will pick up one process.

So far I have split the list up....

I am stuck on execute a function and leaving it running, I do not care about the return values, I just want to initiate say 4 processes all at once.

I have looked at os.fork() but there isn't much documentation about.

Thanks
Reply
  #2  
Old September 3rd, 2008, 04:07 PM
Expert
 
Join Date: Sep 2007
Posts: 844
Default

The reason there isn't much documentation about os.fork() is that it's just a Python wrapper to the C function of the same name (on *nix, at least), which is exhaustively documented.

You also may find the os.popen() line of functions useful, which will likely have similar docs.
Reply
  #3  
Old September 7th, 2008, 04:18 PM
Subsciber123's Avatar
Member
 
Join Date: Nov 2006
Location: Eastern Time Zone
Posts: 83
Default

You might want to use threads. They work as follows:

thread.start_new_thread(function, argument_tuple [,keyword_dictionary])

This will either return None (successful) or raise an exception if it is unable to start a new thread.
Reply
Reply

Bookmarks


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

What is Bytes?

We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights. Get the best answers to your questions from over network members.
Post your question now . . .
It's fast and it's free

Popular Articles