[CPMD-list] elapsed v. cpu time issue on Itanium

[Jeff Nucciarone]

I recently compiled CPMD 3.9.1 on an Itanium2 Linux system using the 
version 8 Intel compiler and mpi-gm (Myrinet mpigm-1.2.5..10).  I used 
Intel MKL version 7.

I noticed a large disparity between the elapsed running time and cpu 
time when running the femd test case (al001geo.inp in the CPMD-test femd 
subdirectory).

I ran the test case using 4 cpus. The following is a summation of the 
output:

  SUBROUTINE            CALLS         CPU TIME        ELAPSED TIME
      FFTCOM           661040            62.81               63.20
    S_INVFFT           347217            57.20               59.47
     S_FWFFT           311799            52.98               53.68
     FFT-G/S          1981096            40.16               42.60
       RGS_C             2858            31.78              116.29
     EHPSI_C            26642            23.07               24.24
       EVPSI           240599            11.26               11.19
  FRIESNER_C             2830             9.52               10.23
      GLOSUM           389289             9.33               10.03
       VBETA             2830             8.12                8.10
    OVLAP2_C            27451             6.77                6.72
    RHOOFR_C              284             6.13                6.03
      JACOBI           154455             3.94                4.14
   CALC_BILN            26642             1.31                1.29
     OVLAP_H             5860             1.02                1.03
     W_WFNIO               20             0.74                0.93
  ----------------------------------------------------------------
  TOTAL TIME                            326.13              419.17
  ****************************************************************

        CPU TIME :    0 HOURS  5 MINUTES 31.33 SECONDS
    ELAPSED TIME :    0 HOURS  7 MINUTES  4.64 SECONDS

  PROGRAM CPMD ENDED AT:   Wed Aug  4 16:29:33 2004


The big difference is in subroutine RGS_C:


       RGS_C             2858            31.78              116.29

The difference between elapsed and cpu time accounts for over 90% of the 
difference in run times overall.

I made several runs and this observation is consistent across all runs. 
Something in rgc_s seems to be triggering a lot of system activity.

I also built versions of CPMD of Xeon (also using Myrinet MPI) and 
Opteron (using Infiniband and MVICH). neither of these platforms has 
this behavious and elapsed and cpu time for those machines is consistent.

Has anyone else made this observation and if so is there any way to get 
around this performance issue?

Thanks,

   --Jeff
-- 
Jeff Nucciarone     nucci at psu.edu          http://www.personal.psu.edu/nucci
   Senior Research Programmer, High Performance Computing Group, ITS/ASET
   The Pennsylvania State University
                                "Don't just do it........  do it right."