[CPMD-list] Intel compiler 10 seg fault resolution on quadcore Xeon.

[Axel Kohlmeyer]

On Sun, 14 Oct 2007, Joseph Hargitai wrote:

joseph,


JH> It appears that using version 10 of Intel, on quadcore 2.33Ghz 
JH> machines the -xT flag causes the seg-fault. The interconnect is 
JH> Infinband.

that is not overly surprising. the vectorization optimizations of
the intel compiler have been getting increasingly aggressive over
time and thus they have an increased risk of miscompiling the code.

it would be interesting to see where you get the segfault. e.g.
by enabling core dumps and then doing a stack trace on the coredump.
some of the output subroutines and particularly the memory allocation
and deallocation wrappers are frequently miscompiled by compilers
of different vendors. sometimes, those files have to be compiled
manually. i better not comment on vector machines, where significant
"manual postprocessing" of the makefile can be required thanks to
not overly cooperative compilers.

JH> Using Atlas from the CPMD site and the following conf file works. (it also works with 
JH>   CC='/usr/mpi/intel/mvapich-0.9.9/bin/mpicc'. 
JH> 
JH> Optimization level -O2 also works, but there is no increase at all in speed. This is stripped to minimal optimization:
JH> 
JH> 
JH> IRAT=2
JH>      CFLAGS=''
JH>      CPP='/lib/cpp -P -C -traditional'
JH>      CPPFLAGS='-D__Linux -D__PGI -DFFT_DEFAULT -DPOINTER8 -DLINUX_IFC \
JH>         -DPARALLEL'

you should also add the -DMYRINET define here. most infiniband 
implementations not support calling SYSTEM() except for very, 
very new kernel/driver combos. -DMYRINET will use alternate
methods to achieve what otherwise is done with composing a
small shell command in a string and then doing a CALL SYSTEM(string).

JH>      FFLAGS='-pc64 -unroll'
JH>      LFLAGS=' -L. -latlas_x86_64'

hmmmm... since you are on an intel quad-core, it might be worth
trying out the intel MKL instead. that could be a bit faster...

JH>      FFLAGS_GROMOS='-Dgood_luck $(FFLAGS)'
JH>        FC='/usr/mpi/intel/mvapich-0.9.9/bin/mpif77 -c'
JH>        CC='/usr/mpi/gcc/mvapich-0.9.9/bin/mpicc'
JH>        LD='/usr/mpi/intel/mvapich-0.9.9/bin/mpif77'
JH> 
JH> 

JH> as far as the results for various node/cpu combinations: (8 cores 
JH> per node)
JH> 
JH> 
JH> 5 nodes, 8 cpus, for 40 cpu total 
JH> 
JH>   ELAPSED TIME :    0 HOURS 20 MINUTES 51.31 SECONDS     
JH>  
JH> 
JH> reverse:
JH> 
JH> 8 nodes 5 cpus, for 40 cpu total
JH> 
JH>  CPU TIME :    0 HOURS 14 MINUTES 26.52 SECONDS     


not very surprising. it would actually be very interesting
to compare 8 nodes with 4PE vs 4 nodes with 8PE. since in 
the case of 4PE/node each of the MPI threads has twice the
cache and CPMD will benefit from it. with 5PE/node two of
them have to share the cache, and they should slow down all
of it. with 8PE/node you have a lot of memory contention
and collisions at the communication.


cheers,
    axel.

JH>    
JH> 
JH> 
JH> 8 nodes 6 cpus  for 48 cpu total
JH> 
JH>      CPU TIME :    0 HOURS 10 MINUTES  5.86 SECONDS     
JH>   
JH> 
JH> 
JH> 6 nodes 8 cpus - 48 cpu total
JH> 
JH>    CPU TIME :    0 HOURS 13 MINUTES 40.81 SECONDS     
JH>    ELAPSED TIME :    0 HOURS 14 MINUTES 18.46 SECONDS     
JH> 
JH> 
JH> 
JH> 6 nodes  7 cpus -  42 cpu total  
JH>  CPU TIME :    0 HOURS 14 MINUTES 31.68 SECONDS     
JH>  
JH> 
JH> 1 node 8 cpus - 8 cpu total -  
JH> 
JH>  CPU TIME :    1 HOURS 31 MINUTES 55.62 SECONDS     
JH>   
JH> 
JH> 2 nodes 8 cpus - 16 cpu total
JH> 
JH>        CPU TIME :    0 HOURS 44 MINUTES 42.87 SECONDS    
JH>   
JH> j
JH> _______________________________________________
JH> CPMD-list mailing list
JH> CPMD-list at cpmd.org
JH> http://cpmd.org/mailman/listinfo/cpmd-list
JH> 

-- 
=======================================================================
Axel Kohlmeyer   akohlmey at cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.