need something like threads, or just multiple simulateous exec's

Serge A. Ribalchenko wrote:

Hi all,

I'm new in python coding...
Can someone point me how can I do such a usual thing? I need to ping
some hosts, and harvest results in one list. Just cause there is about
120 hosts in the LAN, and approx. 10% is down at working hours, pinging
each host at a time from cycle is VERY slow solution.
I did:
retv = os.popen('ping -c 1 ' + ip + ' |grep \"1 packet\" |awk \'{print
$4}\'')
res = retv.read()
In cycle. BUT. I need something FASTER. And I feel fear to scary word
'thread'. May someone give me some examle using threads for this
purpose? Thanks.
Best wishes,
~ Serge.

import threading
import Queue
import os

output = Queue.Queue()

def f(ip, output=output):
output.put(os.popen('ping -c 1 ' + ip + ' |grep \"1 packet\" \
|awk\'{print > $4}\'')
for ip in host:
threading.Thread(target=f, args=(ip,)).start()

for ip in host:
print output.get()
- Josiah

I'm not sure why you need to use threads here. popen()ed processes
execute asynchronously, blocking when they create enough unread output.
In this case, they'll generate very little output. Why not try this?

# Create all processes at once
pipes = [os.popen(...) for ip in hosts]

# Read their output
output = [pipe.read for pipe in pipes]

Suppose you decide you want to run only N os.popen()s at a time. Here
you'll want to use select() instead of threads. Something like this:

remaining_hosts = hosts[:]
pipes = []
output = []
for i in range(nthreads):
while remaining_hosts or pipes:
while remaining_hosts and len(pipes) < npipes:
host = remaining_hosts.pop()
pipes.append(os.popen(...))
ready = select.select([pipes], [], [])[0]
for pipe in ready:
# Assuming output arrives all at once
# (it should since it's one line)
output.append(pipe.read())
pipes.remove(pipe)
pipe.close()

Jeff