By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,660 Members | 1,102 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,660 IT Pros & Developers. It's quick & easy.

Parallel/distributed generator

P: n/a
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
....
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?

George

May 23 '07 #1
Share this Question
Share on Google+
5 Replies


P: n/a
George Sakkis wrote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:

def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime

Is there any module out there that does anything close to this ?
Have you tried using pyro to just return a generator? After all, it's just
an object with a next()-method, that raises a certain exception. I can't
see why pyro shouldn't be able to handle that.

diez
May 23 '07 #2

P: n/a
On May 23, 2:58 pm, half.ital...@gmail.com wrote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:


I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
Is there any module out there that does anything close to this ?
George

DOLT!- Hide quoted text -

- Show quoted text -
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul

May 23 '07 #3

P: n/a
On May 23, 4:22 pm, Paul McGuire <p...@austin.rr.comwrote:
On May 23, 2:58 pm, half.ital...@gmail.com wrote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
Is there any module out there that does anything close to this ?
George
DOLT!- Hide quoted text -
- Show quoted text -

I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul
No, he probably sent it from Google groups as I did, which claimed
that "your message has not been sent, please try again later". Go
figure.

George

May 23 '07 #4

P: n/a
On May 23, 1:22 pm, Paul McGuire <p...@austin.rr.comwrote:
On May 23, 2:58 pm, half.ital...@gmail.com wrote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
Is there any module out there that does anything close to this ?
George
DOLT!- Hide quoted text -
- Show quoted text -

I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.

-- Paul
Damn computers!

May 24 '07 #5

P: n/a
On 23 May 2007 11:00:56 -0700, George Sakkis <ge***********@gmail.comwrote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e.
http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way.
Kamaelia? http://tinyurl.com/35fjbr

--
Cheers,
Simon B.
si***@brunningonline.net
http://www.brunningonline.net/simon/blog/
GTalk: simon.brunning | MSN: small_values | Yahoo: smallvalues
May 24 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.