[CPMD-list] OpenMP or OpenMP+MPI

[Kozin, I (Igor)]

Hi Axel and CPMD users/developers,
you might be interested to know that with Intel 8.1
compiler (Build 20040922) I was finally able to build 
CPMD 3.9.1 fully using OpenMP and OpenMP + MPI.
I made a few test runs and unfortunately do not see
any performance gains in using OpenMP. In fact the
opposite is true. (And yes, the program correctly reports
the number of OMP_NUM_THREADS) I am investigating the
problem and will update the list when I have more to say.

> IK> was anybody lucky enough to be able to build and run
> IK> CPMD using OpenMP or (better yet) OpenMP+MPI on a platform
> IK> other than IBM Power4?
> 
> IK> Not a problem to build the code but it always crashes.
> IK> I remember somebody was asking a similar question but
> IK> don't think it was answered. 
> 
> that must have been me. openmp is really tricky, since
> there are obviously differences in the openmp implementations
> between platforms. also, you should keep in mind, that for
> almost all platforms using openmp only makes sense, when 
> MPI has scaled out. especially on linux pcs, the penalty 

I observe pretty good scaling up to 32 CPUs.
Besides it seems very reasonable to try using OpenMP within
a node in large clusters of dual Xeons/Athlons connected by
GigE or slower.

Igor

> for thread creation kills most of the performance gain
> from the openmp (be sure to compare wall time and not
> TCPU). some level of threaded speedup can already be 
> achieved by just using a multi-threaded BLAS/LAPACK
> (e.g. intel MKL or a mt-enabled ATLAS).
> 
> the matter is current being investigated,
> but don't hold your breath. the best workaround, that i have
> found so far, is to compile only a small subset with openmp
> enabled. the respective objects are: 
> 
> util.o mltfft.o azzero.o fftutil.o rortv.o rortog.o vofrho.o 
> velupi.o velupa.o 
> these seem safe to be compiled with openmp on x86 machines.
> 
> 
> below is the top part of a modified makefile, that i
> have used to assemble the list of 'usable' objects and 
> test to the impact of using an mt-enabled atlas.
> 
> some additional info is at.
> http://www.theochem.ruhr-uni-bochum.de/~axel.kohlmeyer/cpmd-linux.html
> 
>