How to speed up this problem by MPI

[timlee]

Hi,

(1). I am wondering how I can speed up the time-consuming computation in the loop of my code below using MPI?

 int main(int argc, char ** argv)   
 {   
 // some operations           
 f(size);           
 // some operations         
 return 0;   
 }   
 
 void f(int size)   
 {   
 // some operations          
 int i;           
 double * array =  new double [size];           
 for (i = 0; i < size; i++) // how can I use MPI to speed up this loop to compute all elements in the array?   
 {   
 array[i] = complicated_computation(); // time comsuming computation   
 }           
 // some operations using all elements in array           
 delete [] array;  
 }
 

As shown in the code, I want to do some operations before and after the part to be paralleled with MPI, but I don't know how to specify where the parallel part begins and ends.

(2) My current code is using OpenMP to speed up the comutation.

 void f(int size)   
 {   
 // some operations           
 int i;           
 double * array =  new double [size];   
 omp_set_num_threads(_nb_threads);  
 #pragma omp parallel shared(array) private(i)  
 {
 #pragma omp for schedule(dynamic) nowait          
 for (i = 0; i < size; i++) // how can I use MPI to speed up this loop to compute all elements in the array?   
 {   
 array[i] = complicated_computation(); // time comsuming computation   
 }          
 } 
 // some operations using all elements in array           
 }
 

I wonder if I change to use MPI, is it possible to have the code written both for OpenMP and MPI? If it is possible, how to write the code and how to compile and run the code?

(3) Our cluster has three versions of MPI: mvapich-1.0.1, mvapich2-1.0.3, openmpi-1.2.6. Are their usage same? Especially in my case. Which one is best for me to use?

Thanks and regards!

[RedSon]

I don't know anything about MPI but if you were looking to speed up the run time of this program I would think about running array[i] = complicated_computation(); each in their own thread or something like that. If each element of the array is independent of the other elements, then you could run the complicated_computation() in parallel.

[weaknessforcats]

I recommend that you use a proxy for your array elements. Let the proxy worry about intializing.

You would have a second array of short with all elements initialized to 0. This is a map where short array[10] corresponds to double array[10].

Then instead of access the array of double directly, you call the proxy function with the desired element number, like proxy(10) for the 11th double element.

The proxy function looks at the short array[10] and if it's zero the proxy function calls the complicated calculation, sets short array[10] to 1 and sets double array[10] to the return of the complication calculation and then returns the double.

If the proxy finds short array[10] is not 0 it just returns double array[10].

This way only the double elements that are actually accessed ever get initialized.