test for 4096 cpus. was RE: [CPMD-list] Empty ENERGIES file on x86-64 systems

[Axel Kohlmeyer]

On Mon, 8 May 2006, Bernd Kallies wrote:

BK> Dear all,

dear bernd,

BK> I've set up a medium-to-large CPMD benchmark, which we want to use to
BK> rate machines regarding suitability for CPMD. The terms "medium" to
BK> "large" depend on what kind of machine people have at hand and what
BK> system sizes they are commonly using. For us, the benchmark is inbetween
BK> "medium" and "large", since we only have 512 IBM 1.3 GHz Power4 CPUs,
BK> which are 4 years old now.

indeed, the term medium or large depends on whether you
are running on a 'capability' machine or not. you have to
keep in mind that CPMD is run by more or less two groups
of people, those with access to 'capability' try hardware,
and they usually have projects that strain those machines
to the max and those with somewhat limited resources 
(frequently local cluster with 64 or less nodes).
so CPMD benchmarks should be available for both types of
setups and it is also interesting to see, how well a 'small'
problem will scale on a large machine and how large a problem
one can run on a small machine.

BK> The benchmark idea is to time an MD run, which is set up as a restart
BK> from a previous MD run regarding wavefunction and velocities. One either
BK> may use a provided restart file, or recalculate the wavefunction in a
BK> preceeding step from given ionic positions. Thus, different convergence
BK> behaviour on different platforms is avoided for benchmarking. I/O can be

well, if you - for example - are more interested in doing
geometry optimizations or BO-MD, you actually care about
how well and fast you can optimize a wavefunction. so again,
having both sets of data available would be helpful, one
just has to be aware, that some information is more useful
for a certain environment and some is less.

BK> switched off completely, except reading and writing the restart file.
BK> However, the number of timesteps one benchmarks has to ensure that
BK> reading/writing the restart file does not dominate the benchmark. It is
BK> not a good idea to mix calculation and I/O for benchmarking to my
BK> opinion, since both things have to be tuned in a completely different
BK> way.

hmmm, i do not agree entirely. if you run production you need to keep
certain data, and also the time spent on I/O has some effect. please
keep in mind, that on many large machines, one has a limit of only
a few hours per job and that this frequently has to include the time
of moving data to/from a scratch area to a more permanent location.
so the best way is to have both a 'regular' time and a 'best effort'
time that is tuned to yield the best performance. this way, one can
pick the numbers that one needs. 

BK> My benchmark system consists of an Fe3+ ion, which is complexed with an
BK> organic tetradentate chelate ligand named tris-carboxymethylamine (NTA).
BK> The whole thing is solvated in 229 waters, yielding a cubic box with
BK> about 36 a.u. box length. The system is a real-world problem and not
BK> artificial. Since Fe3+ is an open-shell system (sextett), one has to use
BK> LSD. The Fe-PP I use for the benchmark uses NLCC and was developed for
BK> Fe2+/Fe3+ with PBE. It performs good with 70 Ry cutoff and above when
BK> looking on the corresponding aqua ions. For 70 Ry the system needs
BK> nearly 2 Mio plane waves. The calculation needs about 47 GByte memory
BK> total, the restart file is about 7 GByte (!) on usual platforms (bigger
BK> on a NEC). Systems like this one are among the ones people want to look
BK> at at our site.

i would rate this system as more of a big one. on top of that, i would
think this could be a useful example to check out, how well 
norm-conserving pseudopotentials work compared with ultra-soft psps
(in terms of how much aggregate memory you need and how fast the
job will run), as usually the nc-psp do scale better but need much
more memory.

BK> All in all I believe that benchmarks like this one are suitable to
BK> measure CPMD-performance on a variety of "big" platforms. Smaller
BK> problems make no sense to my opinion.

true not for this kind of machine, but see above.
also you have to keep in mind, that it is not always
that easy to get access to this kind of resources
simply for benchmarking (or that you would want to
'waste' the valueable cpu time on that...).

BK> The benchmark run time can be tuned easily and does not suffer from
BK> different convergence behaviour, since wavefunction convergence is
BK> achieved outside of the benchmark. One also might play with plane wave
BK> cutoffs. To stay physical and increase system size, one might add
BK> addtional water. LSD seemed a good idea to me since it doubles the
BK> computational work to do. However, all this works only if one gets a
BK> converged wavefunction prior the benchmark, which is not easy with
BK> systems like this one.

again, just because of that, it is specifically interesting since
there are some machine/compiler/library combination that are notoriously
bad in that respect and documenting this, can be _very_ helpful.

this can make or break a project if you have to decide which 
machine you want to pick in case your HPC allocation went through.

as written before. i suggest that people interested in these topics
create a biocore account and send me their userid so i can add it
to the benchmark project i just created. discussing this in public
can be problematic, since occasionally some information has to be
cleared before making it public and this way we can contain and
track it more easily. results and summaries can then be easily
published on the CPMD.org server (and elsewhere :)).

sorry for the shamless plug for biocore, but IMNSHO this 
is the perfect tool for this kind of project.
here again the URL: http://www.ks.uiuc.edu/Research/biocore/

best,
    axel.

BK> 
BK> Any comments are welcome.
BK> 
BK> 

-- 
=======================================================================
Axel Kohlmeyer   akohlmey at cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.