473,320 Members | 2,104 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

pyMPI and calling MPI functions using Boost.Python

Hello,

I'm trying to wrap a c++ library using MPI inside with boost.python
(or SWIG).
I managed to find that calling `MPI::Init()` embeded in a c++ funtion
would not work. So, I decided to use `pyMPI`.
To avoid overhead of pickling and unpickling while calling `mpi.send`
in Python, I'd rather call c++ functions `MPI::Isend()` etc. embeded
in a c++ function.

What I'd like to know is the following:
After starting `pyMPI` session, it seems to work call c++ function
`MPI::Isend()` or `MPI::Irecv()`, etc. from the `pyMPI`,
at least for my small test code attached below.

Is it guaranteed to work for larger scale code?
I mean for larger memory allocation, heavily repeated message passing,
, many paralell processes.
Or, the example worked only with some luck?
As for now, `mpiCC` is built without shared library.

Any suggestions?

Thank you.

--Natsu

----- A small test code mpitest.so works like this -----
$ mpirun -np 2 /usr/bin/pyMPI
Python 2.2.1 (#1, Jan 22 2003, 19:07:20)
[GCC 2.95.3 20010315 (release)] on linux2
Copyright (c) 2001, 2002 Python Software Foundation.
All Rights Reserved.

Copyright (c) 2000 BeOpen.com.
All Rights Reserved.

Copyright (c) 1995-2001 Corporation for National Research Initiatives.
All Rights Reserved.

Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam.
All Rights Reserved.
import mpi
import mpitest
print "hello ", mpi.rank, mpi.size hello 0 2
hello 1 2 bnd = mympi.bndry(6)
bnd.mpiTest(mpi.rank, mpi.size - 1 - mpi.rank)

100, 99, 98, 97, 96, 95, was a rtmp for node:0
100, 101, 102, 103, 104, 105, was a rtmp for node:1>>>
----- mpitest.so is made from the following using Boost.Python.
//ftest.cpp
#include "ftest.h"

//ftest.h
#include <iostream>
#include <mpi++.h>

class bndry{
MPI::Request sreq_, rreq_;
double* stmp_;
double* rtmp_;
public:
int n_;
bndry(int n) : n_(n) {
stmp_ = new double[n];
rtmp_ = new double[n];
}
~bndry(){
delete [] stmp_;
delete [] rtmp_;
};
void mpiTest(int me, int other) {
double c0 = 1.0;
if (me == 1) c0 = -1.0;
unsigned int ii = 0;
// Create diffrent arrays in diffrent nodes
for (int k = 0; k < n_; ++k) {
stmp_[ii++] = 100.0 + c0 * k;
}
sreq_ = MPI::COMM_WORLD.Isend(stmp_, ii, MPI::DOUBLE, other, me);
rreq_ = MPI::COMM_WORLD.Irecv(rtmp_, ii, MPI::DOUBLE,
other, other);
rreq_.Wait();
for (int k = 0; k < n_; ++k) {
cout << rtmp_[k] << ", ";
}
cout << " was a rtmp for node:" << me << endl;
sreq_.Wait();
}
};

//mpitest.cpp
#include <boost/python.hpp>
#include <boost/cstdint.hpp>
#include <fblock.h>
using namespace boost::python;
BOOST_PYTHON_MODULE(mpitest) {
class_< bndry >("bndry", init< const bndry& >())
.def(init< int >())
.def_readwrite("n_", &bndry::n_)
.def("mpiTest", &bndry::mpiTest)
;
}

Jul 18 '05 #1
0 2242

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Wen Jiang | last post by:
Hi, I have been using pyMPI to parallelize my code and found that the function mpi.send() leaks memory a lot and thus is not really working for large amount fo data communication. It actually...
10
by: Simon Elliott | last post by:
If I have two classes: class bar { public: void RunTheBar(void); }; class foo {
1
by: stringy | last post by:
I would like to be able to write a C++ function, to be wrapped into some python, to be able to communicate over pyMPI. As pyMPI is based on C++ I figure that this should be possible, although I'm...
7
by: Pankaj | last post by:
The module which i am creating is like Part A: 1. It does some processing by using python code. 2. The result of this python code execution is written to a text file. ] Part B: 1. I read a...
3
by: gabriel.becedillas | last post by:
Hi, I'm having problems wrapping a hierarchy of classes, actually having problems wrapping the base class. I don't need to use the WrapClass mechanism since I don't want to override classes in...
11
by: Osiris | last post by:
I have these pieces of C-code (NOT C++ !!) I want to call from Python. I found Boost. I have MS Visual Studio 2005 with C++. is this the idea: I write the following C source file:...
4
by: Shawn McGrath | last post by:
Hi, I'm trying to expose a C++ class' internals to python via boost::python. I can do integer/boolean functions fine, but as soon as I do a string get/set it craps out. ...
0
by: Stou Sandalski | last post by:
Hi, I have a python library created by wrapping the C++ library using Boost.Python, the problem is that the wrappers are not very pythonic.... so I want to add some methods that do not exist in...
1
by: smithken04 | last post by:
Hi! I am trying to find my way to a running parallel NumPy installation. Can you help me getting started wihtout losing my way several times? PyMPI and myMPI are apperently not too actively...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.