Parallel/distributed generator | |
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e. http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
def iterprimes(start=1, end=None):
# ...
yield prime
# rpc-related initialization
....
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
Is there any module out there that does anything close to this ?
George | | | | re: Parallel/distributed generator
George Sakkis wrote: Quote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e. http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
>
def iterprimes(start=1, end=None):
# ...
yield prime
>
>
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
>
Is there any module out there that does anything close to this ?
Have you tried using pyro to just return a generator? After all, it's just
an object with a next()-method, that raises a certain exception. I can't
see why pyro shouldn't be able to handle that.
diez | | | | re: Parallel/distributed generator
On May 23, 2:58 pm, half.ital...@gmail.com wrote: Quote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:
>
>
>
>
> Quote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e. http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
> Quote:
def iterprimes(start=1, end=None):
# ...
yield prime
> Quote:
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
> Quote:
Is there any module out there that does anything close to this ?
> >
DOLT!- Hide quoted text -
>
- Show quoted text -
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
-- Paul | | | | re: Parallel/distributed generator
On May 23, 4:22 pm, Paul McGuire <p...@austin.rr.comwrote: Quote:
On May 23, 2:58 pm, half.ital...@gmail.com wrote:
>
>
> Quote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:
> Quote: Quote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e. http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
> Quote: Quote:
def iterprimes(start=1, end=None):
# ...
yield prime
> Quote: Quote:
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
> Quote: Quote:
Is there any module out there that does anything close to this ?
> > Quote:
DOLT!- Hide quoted text -
> Quote:
- Show quoted text -
>
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
>
-- Paul
No, he probably sent it from Google groups as I did, which claimed
that "your message has not been sent, please try again later". Go
figure.
George | | | | re: Parallel/distributed generator
On May 23, 1:22 pm, Paul McGuire <p...@austin.rr.comwrote: Quote:
On May 23, 2:58 pm, half.ital...@gmail.com wrote:
>
>
> Quote:
On May 23, 11:00 am, George Sakkis <george.sak...@gmail.comwrote:
> Quote: Quote:
I'm looking for any existing packages or ideas on how to implement the
equivalent of a generator (in the Python sense, i.e. http://www.python.org/dev/peps/pep-0255/) in a parallel/distributed
way. As a use case, imagine a function that generates a range of
primes. I'd like to be able to do something along the following lines:
> Quote: Quote:
def iterprimes(start=1, end=None):
# ...
yield prime
> Quote: Quote:
# rpc-related initialization
...
rpc_proxy = some_rpc_lib(iterprimes, start=1e6, end=1e12)
for prime in proxy:
print prime
> Quote: Quote:
Is there any module out there that does anything close to this ?
> > Quote:
DOLT!- Hide quoted text -
> Quote:
- Show quoted text -
>
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
I thought you were making a joke about parallel processing.
>
-- Paul
Damn computers! |  |
| | |  /bytes/about
We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights.
Get the best answers to your questions from over 226,510 network members.
|
