473,513 Members | 2,430 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

pyMPI momory leak

Hi,
I have been using pyMPI to parallelize my code and found that the
function mpi.send() leaks memory a lot and thus is not really working
for large amount fo data communication. It actually fails after the
leak accumulates more than 2G. I wonder if others have the same
experience or I did something wrong. I compiled python 2.4, mpich
1.2.6, pyMPI 2.1b4 on Opteron cluster running Rocks 3.3.
Here is a small test script with 2 CPUs to demo the memory leak:

import mpi
n = 10000
i=0
data = [0]*40000
while i < n:
if mpi.rank==1:
mpi.send(data, 0)
elif mpi.rank==0:
msg, status = mpi.recv()
n+=1
if one watchs the memory usage using 'top', one can see one process use
little and constant amount of memory (recv for rank=0) and the other
process uses more and more memory (send for rank=1).

Jul 18 '05 #1
0 1054

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2264
by: Natsu Mizutani | last post by:
Hello, I'm trying to wrap a c++ library using MPI inside with boost.python (or SWIG). I managed to find that calling `MPI::Init()` embeded in a c++ funtion would not work. So, I decided to use...
1
1381
by: stringy | last post by:
I would like to be able to write a C++ function, to be wrapped into some python, to be able to communicate over pyMPI. As pyMPI is based on C++ I figure that this should be possible, although I'm...
0
2114
by: antsays | last post by:
I am trying to serialize a collection to disk using the code provided. This works just fine but when I try do copy and past the xml file to another location or sometimes even just click on the...
8
3394
by: ranjeet.gupta | last post by:
Dear All Is the Root Cause of the Memory corruption is the Memory leak, ?? suppose If in the code there is Memory leak, Do this may lead to the Memory Corruption while executing the program ? ...
17
4770
by: José Joye | last post by:
Hi, I have implemented a Service that is responsible for getting messages from a MS MQ located on a remote machine. I'm getting memory leak from time to time (???). In some situation, it is...
0
873
by: Carl J. Van Arsdall | last post by:
Hey, anyone use the PyMPI suite: http://sourceforge.net/projects/pympi/ ?? I am an engineer doing research with MPI and thought I'd do some digging to see if any scripting languages had adopted...
1
3382
by: smithken04 | last post by:
Hi! I am trying to find my way to a running parallel NumPy installation. Can you help me getting started wihtout losing my way several times? PyMPI and myMPI are apperently not too actively...
22
9303
by: Peter | last post by:
I am using VS2008. I have a Windows Service application which creates Crystal Reports. This is a multi theaded application which can run several reports at one time. My problem - there is a...
18
2123
by: Daniel Orner | last post by:
Hi all, I've been trying to pin down a memory leak in IE6 for several WEEKS now. I've done my share of googling etc., and know all about common leaks like circular references and closures, but...
0
7265
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
7171
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
1
7111
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
5692
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
5095
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
4751
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
3240
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
1605
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
0
461
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.