python + cluster computing

Uwe Schmitt

Hi,

I found several Python bindings for the MPI protocol,
can anybody report some opinions and experiences ?

Greetings, Uwe.

--
Dr. rer. nat. Uwe Schmitt http://www.procoders.net
sc*****@procode rs.net "A service to open source is a service to mankind."

Jul 18 '05 #1

Subscribe Reply

4405

Tim Churches

On Thu, 2003-07-10 at 00:27, Uwe Schmitt wrote:

Hi,

I found several Python bindings for the MPI protocol,
can anybody report some opinions and experiences ?

Peter Christen and I have used Ole Nielsen's PyPar package (see
http://datamining.anu.edu.au/~ole/pypar/) in our Febrl probabilistic
record linkage engine (see
http://datamining.anu.edu.au/projects/linkage.html) with excellent
results. I have also been experimenting with its use in a parallel data
summarisation engine. It is easy to install and is very easy to use,
mainly because it only wraps a small portion of MPI - but just those
parts you really need (at least we haven't missed anything yet). Its
ability to pass any picklable Python object around the cluster is a real
bonus, and the speed at which it passes Numeric Python arrays using
"raw" mode is very impressive. Ole is also very helpful, and is actively
involved in parallel computing work on a daily basis (using PyPar, no
doubt).

--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA/DHcneJFGqer5k9A RAlVOAKDzm4Xld0 2G+C6qRHh9ZaMX2 lgTfQCgk1nw
lTw4djQgOQ4Hsbz KXVRlV/0=
=hyxe
-----END PGP SIGNATURE-----

Jul 18 '05 #2

Similar topics

2680

Python as a numerical prototyping language.

by: Carl | last post by:

I have been using Python for quite some time now and I love it. I use it mainly for explorative computing and numerical prototyping, ie testing and trying out different kinds of algorithms and computational schemes. The use of Python as my first-choice language has made me extremely productive. Now, I have always believed that Python is a poor performer in terms of numerical speed. My experience, however, is that the efficient use of...

Python

3420

Python for large projects

by: assaf__ | last post by:

Hello, I am beginning to work on a fairly large project and I'm considering to use python for most of the coding, but I need to make sure first that it is reliable enough. I need to make sure that I won't have surprises when my program runs on different real-world systems. So far I wrote a little script with python using urllib, and on one computer it failed completely because of a problem in getting the proxies (in my opinion this is a...

by: Thomas Reichelt | last post by:

Moin, short question: is there any language combining the syntax, flexibility and great programming experience of Python with static typing? Is there a project to add static typing to Python? Thank you, -- greetz tom

Python

1639

Does python support multi prototype.

by: angel | last post by:

Hi Java and cpp support one function name multi function prototype. For example: A: int func1(String s){} int func1(int i){} B: int func1(String s){}

Python

2952

valgrind python 2.3.4

by: Jerald | last post by:

Running python 2.3.4 on valgrind (a tool like purify which checks the use of uninitialized memory, etc), gives a lot of errors. See below. jfj@cluster:~/> python -V Python 2.3.4 jfj@cluster:~/> valgrind python ==10517== Memcheck, a.k.a. Valgrind, a memory error detector for x86-linux.

Python

1374

how to configure windows based cluster using MPI

by: Muaz Farooq | last post by:

Hi everybody, i am doing project on distributed computing, i was trying to configure my cluster(2 computers using cross cable), but was unable to run MPI programs on the cluster.But when i ran a program on a single computer it ran easily. Can anybody please tell me how to configure a windows based cluster on two computers

.NET Framework

4331

Parallel Python

by: parallelpython | last post by:

Has anybody tried to run parallel python applications? It appears that if your application is computation-bound using 'thread' or 'threading' modules will not get you any speedup. That is because python interpreter uses GIL(Global Interpreter Lock) for internal bookkeeping. The later allows only one python byte-code instruction to be executed at a time even if you have a multiprocessor computer. To overcome this limitation, I've created...

Python

4093

Extending Python with C: Cannot find MPI library

by: Spectrum | last post by:

I am writing some Python code using the Message Passing Interface (MPI), an API used in parallel computing. There exist a number of Python implementations of MPI, but apparently they all rely on the Numeric Python (numpy) package. I need to run my code on a particular machine made available by my university, which does not have numpy and will not be getting it. So I am trying to write my own mini-wrapper for MPI in C to extend my Python...

Python

2128

Does python is suitable for enterprise cluster management?

by: Asaf Hayman | last post by:

Is anyone familiar or aware of a successful enterprise class project in Python to control and monitor a cluster of computers? As a part of a bigger project my company needs to build a cluster management system. The aim of the system is to control and synchronize applications. Namely, a central management will get events from the remote applications and from a user and will react with commands to the applications. Existing tools don’t...

Python

9706

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...

General

9577

Changing the language in Windows 10

by: Hystou | last post by:

Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...

Windows Server

10325

Maximizing Business Potential: The Nexus of Website Design and Digital Marketing

by: jinu1996 | last post by:

In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...

Online Marketing

9140

AI Job Threat for Devs

by: agi2029 | last post by:

Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...

Career Advice

6847

Couldn’t get equations in html when convert word .docx file to html file in C#.

by: conductexam | last post by:

I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...

C# / C Sharp

5519

Trying to create a lan-to-lan vpn between two differents networks

by: TSSRALBI | last post by:

Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...

Networking - Hardware / Configuration

4295

transfer the data from one system to another through ip address

by: 6302768590 | last post by:

Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

C# / C Sharp

3815

How to add payments to a PHP MySQL app.

by: muto222 | last post by:

How can i add a mobile payment intergratation into php mysql website.

PHP

2990

Comprehensive Guide to Website Development in Toronto: Expert Insights from BSMN Consultancy

by: bsmnconsultancy | last post by:

In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

General