Versions Compared

Old Version 46

changes.mady.by.user Rebecca Buchholz

Saved on

compared with

New Version Current

changes.mady.by.user Rebecca Buchholz

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

*** PAGE actively in development ***

Section


Column
width40%

You will need to ssh into 

Code Block
languagetext
derecho.hpc.ucar.edu



Column
width60%


Tip
titleTips from CISL:

Tips for moving from Cheyenne to Derecho on https://ncar-hpc-docs.readthedocs.io/


Note
titleComprehensive slides of information from NCAR/CISL about the Derecho HPC:

https://docs.google.com/presentation/d/1BAYI9NdyAnZzU3KLat0CK2sQj34XE_K5Wan9zL8rsQ0/edit?usp=sharing

Old instructions for running on Cheyenne (the retired NCAR HPC) are here: Run CAM-Chem on Cheyenne




CESM2.2 on Derecho

Your $HOME and $WORK directories (on glade) are the same path on cheyenne and derecho. Therefore, it is recommended to create a new directory (in $HOME or $WORK) for cases you will run on derecho, to not confuse them with cases built on cheyenne (which will not run on derecho).  Derecho and cheyenne have separate $SCRATCH directories.

...

Get the optimized history-writing code for CLM with these commands: (Not sure this is still neeeded???)

Code Block
languagetext
> cd components/clm
> git fetch origin
> git checkout release-cesm2.2.01/hist_opt

...

NOTE: most changes in user_nl_* files do not require re-building. However, during a run (CONTINUE_RUN = TRUE) no changes can be made to history output (fincl lists). If you want to change history output, create a new or branch run.

Note
titleNote

Note, the The pe-layout of existing compsets in the new code base of CESM2 for derecho (cesm2.2.2) has not been adjusted to the new computer, and running out of the box can lead to large differences in computer costs compared to running on cheyenne. 

New PE-layouts are still being developed. We are working on updating these compsets. One way to increase performance is to use a namelist setting. To improve improve the performance of derecho, you can add the following to your user_nl_cam file.

Code Block
languagetext
phys_loadbalance = 1


...

While running, output is written to <run_dir>: /glade/derecho/scratch/<username>/$CASENAME/run

Section


Column
width65%

After the run completes successfully, output files are moved to the short term archive: /glade/derecho/scratch/<username>/archive/$CASENAME/atm/hist (similar directories exist for other model components: lnd, etc.).

Restart and initial conditions files are written to: /glade/derecho/scratch/<username>/archive/$CASENAME/rest

Note: long term archiving is currently not working

Tip
titleTip for New Users:

For more options regarding the submission and running options type: case.submit --help



Column
width35%


Tip
titleTip for New Users:

Default output is monthly, so if you run a test 5-day simulation with monthly output, you will not see any files in the atm/hist location. However, restart files will have been created.



...