Implement HOMME dycore

Motivation

The motivation of this project is to ensure that the CCSM is ready for the upcoming generation of petascale computing platforms, which are expected to have hundreds of thousands of processors. In the CCSM, one of the largest bottleneck to parallel scalability is due to the latitude-longitude grid based dycores in the atmospheric model. This grid clusters points at the poles, creating a potential "pole problem". There are highly effective techniques to deal with this pole problem, but the current techniques all adversely impact parallel scalability. This project aims to improve the scalability of the CCSM by the introduction of cubed-sphere based dycores (from HOMME) into CAM.  HOMME contains several cubed-sphere dycores. Our initial work will be with the conservative spectral element model.

• Conservative Spectral Element Dycore:
  • Locally conserves both mass and total energy. Zero dissipation of KE and IE except through explicitly added hyperviscosity. (mass weighted del^4 term)
  • Conservation obtained by "compatible" differencing, where the discrete operators satisfy discrete versions of key integral properties.
  • Physics is sub-cycled. Dynamics uses explicit time step
  • Advection:  3rd order accurate, non-oscillatory and sign-preserving
• Discontinious Galerken Dycore:
  • Locally conserves mass.
  • Conservation obtained by flux based methods, potentially allowing the use of limiters and non-oscillatory advection schemes.

Goals

• Phase 1: Produce validated simulations with the HOMME dycore using idealized physics forcings (Held-Suarez).
  • Baroclinic and Held-Suarez simulations are working with both explicit and semi-implicit time stepping. From the CAM wiki: http://wiki.ucar.edu/display/camdev/Verification+of+the+Homme+Dycore
• Phase 2: Produce validated simulation with the HOMME dycore using full CAM physics forcings.
  • APE w/ CAM 3.1 Physics Results
  • APE CAM/HOMME Performance Benchmarks
• Phase 3: Coupling with Land model, data ocean and data ice (AMIP runs)
  • Using CCSM tri-grid coupling system
  • Initial 2 degree CAM-HOMME AMIP Simulations

Outstanding Issues

• Initial data
  • 3_5 physics: no know issues
  • 3_0 physics: AerosolMass for January reused for all years
• Restart files
  • Dynamics restart now working with PIO and PNETCDF
  • PIO-PNETCDF-Restart benchmarks
  • Physics restart still uses serial I/O
• History output
  • History otuput still uses serial I/O
  • History output on native cubed-sphere grid NETCDF files
  • History output is interpolated into lat/lon grid via HOMME post-processing utility. Native spectral element basis function interpolation or binlinear interpolation is supported.
• Tracers
  • Tracers currently running at the gravity wave CFL
  • Considering subcycling approach, or switching to HOMME's semi-implicit dynamics. Both of these approaches would allow tracers to run at the advective CFL limit.
    
    

Running CAM-HOMME