Parallel/distributed generator

[George Sakkis]

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
....
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

George

[Diez B. Roggisch]

George Sakkis wrote:

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

Have you tried using pyro to just return a generator? After all, it's just
an object with a next()-method, that raises a certain exception. I can't
see why pyro shouldn't be able to handle that.

diez

[Paul McGuire]

On May 23, 2:58 pm, [e-mail] wrote:

On May 23, 11:00 am, George Sakkis <[e-mail]:

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime

# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

George

DOLT!- Hide quoted text -

- Show quoted text -

I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul

[George Sakkis]

On May 23, 4:22 pm, Paul McGuire <[e-mail]:

On May 23, 2:58 pm, [e-mail] wrote:

On May 23, 11:00 am, George Sakkis <[e-mail]:

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime

# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

George

DOLT!- Hide quoted text -

- Show quoted text -

I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul

No, he probably sent it from Google groups as I did, which claimed
that "your message has not been sent, please try again later". Go
figure.

George

[half.italian]

On May 23, 1:22 pm, Paul McGuire <[e-mail]:

On May 23, 2:58 pm, [e-mail] wrote:

On May 23, 11:00 am, George Sakkis <[e-mail]:

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime

# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

George

DOLT!- Hide quoted text -

- Show quoted text -

I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul

Damn computers!

[Simon Brunning]

On 23 May 2007 11:00:56 -0700, George Sakkis <ge***********@gmail.comwrote:

I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way.

Kamaelia? http://tinyurl.com/35fjbr

--
Cheers,
Simon B.
si***@brunningonline.net
http://www.brunningonline.net/simon/blog/
GTalk: simon.brunning | MSN: small_values | Yahoo: smallvalues