473,379 Members | 1,257 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,379 software developers and data experts.

python + cluster computing


Hi,

I found several Python bindings for the MPI protocol,
can anybody report some opinions and experiences ?

Greetings, Uwe.

--
Dr. rer. nat. Uwe Schmitt http://www.procoders.net
sc*****@procoders.net "A service to open source is a service to mankind."
Jul 18 '05 #1
1 4388
On Thu, 2003-07-10 at 00:27, Uwe Schmitt wrote:
Hi,

I found several Python bindings for the MPI protocol,
can anybody report some opinions and experiences ?


Peter Christen and I have used Ole Nielsen's PyPar package (see
http://datamining.anu.edu.au/~ole/pypar/) in our Febrl probabilistic
record linkage engine (see
http://datamining.anu.edu.au/projects/linkage.html) with excellent
results. I have also been experimenting with its use in a parallel data
summarisation engine. It is easy to install and is very easy to use,
mainly because it only wraps a small portion of MPI - but just those
parts you really need (at least we haven't missed anything yet). Its
ability to pass any picklable Python object around the cluster is a real
bonus, and the speed at which it passes Numeric Python arrays using
"raw" mode is very impressive. Ole is also very helpful, and is actively
involved in parallel computing work on a daily basis (using PyPar, no
doubt).

--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA/DHcneJFGqer5k9ARAlVOAKDzm4Xld02G+C6qRHh9ZaMX2lgTfQ Cgk1nw
lTw4djQgOQ4HsbzKXVRlV/0=
=hyxe
-----END PGP SIGNATURE-----

Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: Carl | last post by:
I have been using Python for quite some time now and I love it. I use it mainly for explorative computing and numerical prototyping, ie testing and trying out different kinds of algorithms and...
23
by: assaf__ | last post by:
Hello, I am beginning to work on a fairly large project and I'm considering to use python for most of the coding, but I need to make sure first that it is reliable enough. I need to make sure...
176
by: Thomas Reichelt | last post by:
Moin, short question: is there any language combining the syntax, flexibility and great programming experience of Python with static typing? Is there a project to add static typing to Python? ...
9
by: angel | last post by:
Hi Java and cpp support one function name multi function prototype. For example: A: int func1(String s){} int func1(int i){} B: int func1(String s){}
1
by: Jerald | last post by:
Running python 2.3.4 on valgrind (a tool like purify which checks the use of uninitialized memory, etc), gives a lot of errors. See below. jfj@cluster:~/> python -V Python 2.3.4...
0
by: Muaz Farooq | last post by:
Hi everybody, i am doing project on distributed computing, i was trying to configure my cluster(2 computers using cross cable), but was unable to run MPI programs on the cluster.But when i ran a...
43
by: parallelpython | last post by:
Has anybody tried to run parallel python applications? It appears that if your application is computation-bound using 'thread' or 'threading' modules will not get you any speedup. That is because...
7
by: Spectrum | last post by:
I am writing some Python code using the Message Passing Interface (MPI), an API used in parallel computing. There exist a number of Python implementations of MPI, but apparently they all rely on...
4
by: Asaf Hayman | last post by:
Is anyone familiar or aware of a successful enterprise class project in Python to control and monitor a cluster of computers? As a part of a bigger project my company needs to build a cluster...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.