473,242 Members | 1,514 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,242 software developers and data experts.

pyMPI and calling MPI functions using Boost.Python

Hello,

I'm trying to wrap a c++ library using MPI inside with boost.python
(or SWIG).
I managed to find that calling `MPI::Init()` embeded in a c++ funtion
would not work. So, I decided to use `pyMPI`.
To avoid overhead of pickling and unpickling while calling `mpi.send`
in Python, I'd rather call c++ functions `MPI::Isend()` etc. embeded
in a c++ function.

What I'd like to know is the following:
After starting `pyMPI` session, it seems to work call c++ function
`MPI::Isend()` or `MPI::Irecv()`, etc. from the `pyMPI`,
at least for my small test code attached below.

Is it guaranteed to work for larger scale code?
I mean for larger memory allocation, heavily repeated message passing,
, many paralell processes.
Or, the example worked only with some luck?
As for now, `mpiCC` is built without shared library.

Any suggestions?

Thank you.

--Natsu

----- A small test code mpitest.so works like this -----
$ mpirun -np 2 /usr/bin/pyMPI
Python 2.2.1 (#1, Jan 22 2003, 19:07:20)
[GCC 2.95.3 20010315 (release)] on linux2
Copyright (c) 2001, 2002 Python Software Foundation.
All Rights Reserved.

Copyright (c) 2000 BeOpen.com.
All Rights Reserved.

Copyright (c) 1995-2001 Corporation for National Research Initiatives.
All Rights Reserved.

Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam.
All Rights Reserved.
import mpi
import mpitest
print "hello ", mpi.rank, mpi.size hello 0 2
hello 1 2 bnd = mympi.bndry(6)
bnd.mpiTest(mpi.rank, mpi.size - 1 - mpi.rank)

100, 99, 98, 97, 96, 95, was a rtmp for node:0
100, 101, 102, 103, 104, 105, was a rtmp for node:1>>>
----- mpitest.so is made from the following using Boost.Python.
//ftest.cpp
#include "ftest.h"

//ftest.h
#include <iostream>
#include <mpi++.h>

class bndry{
MPI::Request sreq_, rreq_;
double* stmp_;
double* rtmp_;
public:
int n_;
bndry(int n) : n_(n) {
stmp_ = new double[n];
rtmp_ = new double[n];
}
~bndry(){
delete [] stmp_;
delete [] rtmp_;
};
void mpiTest(int me, int other) {
double c0 = 1.0;
if (me == 1) c0 = -1.0;
unsigned int ii = 0;
// Create diffrent arrays in diffrent nodes
for (int k = 0; k < n_; ++k) {
stmp_[ii++] = 100.0 + c0 * k;
}
sreq_ = MPI::COMM_WORLD.Isend(stmp_, ii, MPI::DOUBLE, other, me);
rreq_ = MPI::COMM_WORLD.Irecv(rtmp_, ii, MPI::DOUBLE,
other, other);
rreq_.Wait();
for (int k = 0; k < n_; ++k) {
cout << rtmp_[k] << ", ";
}
cout << " was a rtmp for node:" << me << endl;
sreq_.Wait();
}
};

//mpitest.cpp
#include <boost/python.hpp>
#include <boost/cstdint.hpp>
#include <fblock.h>
using namespace boost::python;
BOOST_PYTHON_MODULE(mpitest) {
class_< bndry >("bndry", init< const bndry& >())
.def(init< int >())
.def_readwrite("n_", &bndry::n_)
.def("mpiTest", &bndry::mpiTest)
;
}

Jul 18 '05 #1
0 2234

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Wen Jiang | last post by:
Hi, I have been using pyMPI to parallelize my code and found that the function mpi.send() leaks memory a lot and thus is not really working for large amount fo data communication. It actually...
10
by: Simon Elliott | last post by:
If I have two classes: class bar { public: void RunTheBar(void); }; class foo {
1
by: stringy | last post by:
I would like to be able to write a C++ function, to be wrapped into some python, to be able to communicate over pyMPI. As pyMPI is based on C++ I figure that this should be possible, although I'm...
7
by: Pankaj | last post by:
The module which i am creating is like Part A: 1. It does some processing by using python code. 2. The result of this python code execution is written to a text file. ] Part B: 1. I read a...
3
by: gabriel.becedillas | last post by:
Hi, I'm having problems wrapping a hierarchy of classes, actually having problems wrapping the base class. I don't need to use the WrapClass mechanism since I don't want to override classes in...
11
by: Osiris | last post by:
I have these pieces of C-code (NOT C++ !!) I want to call from Python. I found Boost. I have MS Visual Studio 2005 with C++. is this the idea: I write the following C source file:...
4
by: Shawn McGrath | last post by:
Hi, I'm trying to expose a C++ class' internals to python via boost::python. I can do integer/boolean functions fine, but as soon as I do a string get/set it craps out. ...
0
by: Stou Sandalski | last post by:
Hi, I have a python library created by wrapping the C++ library using Boost.Python, the problem is that the wrappers are not very pythonic.... so I want to add some methods that do not exist in...
1
by: smithken04 | last post by:
Hi! I am trying to find my way to a running parallel NumPy installation. Can you help me getting started wihtout losing my way several times? PyMPI and myMPI are apperently not too actively...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.