using os.system(command) to send jobs to a Linux cluster

[irvg]

I have a "for" loop to do something over a number of files in a directory. In particular I need to send a command line job that uses each file in thar directory as an input to node in a linux cluster. Here is how the "for" loop looks like:

path = '/mypath/'
for file in os.listdir(path):
node_name = 'name' + ('%02d' % node)
comand = 'ssh ' + node_name + 'external program path and options'
node = node + 1
os.system(command)

My problem is that os.system(command) sends the ssh job to node number 1 in a Linux cluster, and waits until that is finished to send the second ssh job to node number 2 (the "for" loop does not progress until the job sent by os.system is done. I would like it to send the job to each node and not wait until each one has finished. Any ideas?

Thanks!

[jlm699]

You can look into the popen suite of commands for some non-blocking usage.
Link