Pre ccsm3_1_beta39 tags, Increasing throughput:

To maximize throughput of CCSM runs, utilizing SMT allows us to
effectively double the number of
MPI processes used and increase throughput by up to 30% at NO additional
cost. The following
modifications are required to files in your case directory:

  1. modify the file "env_pes.bluevista", doubling the number of MPI tasks (but not threads) assigned to each component;
  2. modify the file "env_mach.bluevista", changing the environment variable MAX_TASKS_PER_NODE from 8 to 16;
  3. after completion of the first two, reconfigure the case, using the commands:
    1. configure -cleanall
    2. configure -mach bluevista
  4. after reconfiguring, change the lines in the run and test scripts from #BSUB -R "span[ptile=8] to #BSUB -R "span[ptile=16]"
    At this point, the case is ready to be submitted.

Pre ccsm3_1_beta39 tags, Reducing model cost:

The SMT capabilities also allow us to run using effectively the same
number of MPI tasks processes but at about half the cost. There is some performance degradation, but the net cost savings should
be around 25%. For this option, the following modifications are
required to files in your case directory:

  1. modify the file "env_mach.bluevista", changing the environment variable: MAX_TASKS_PER_NODE from 8 to 16;
  2. after completion of the first two, reconfigure the case, using the commands
    1. configure -cleanall
    2. configure -mach bluevista
  3. after reconfiguring, change the lines in the run and test scripts from #BSUB -R "span[ptile=8]" to #BSUB -R "span[ptile=16]"
    At this point, the case is ready to be submitted.

For questions, please contact:
Jon Wolfe, Tony Craig, and Mariana Vertenstein

  • No labels