Welcome to MITgcm’s user manual

Authors

Alistair Adcroft, Jean-Michel Campin, Ed Doddridge, Stephanie Dutkiewicz, Constantinos Evangelinos, David Ferreira, Mick Follows, Gael Forget, Baylor Fox-Kemper, Patrick Heimbach, Chris Hill, Ed Hill, Helen Hill, Oliver Jahn, Jody Klymak, Martin Losch, John Marshall, Guillaume Maze, Matt Mazloff, Dimitris Menemenlis, Andrea Molod, and Jeff Scott

Contents:

• 1. Overview
  • 1.1. Introduction
  • 1.2. Illustrations of the model in action
    • 1.2.1. Global atmosphere: ‘Held-Suarez’ benchmark
    • 1.2.2. Ocean gyres
    • 1.2.3. Global ocean circulation
    • 1.2.4. Convection and mixing over topography
    • 1.2.5. Boundary forced internal waves
    • 1.2.6. Parameter sensitivity using the adjoint of MITgcm
    • 1.2.7. Global state estimation of the ocean
    • 1.2.8. Ocean biogeochemical cycles
    • 1.2.9. Simulations of laboratory experiments
  • 1.3. Continuous equations in ‘r’ coordinates
    • 1.3.1. Kinematic Boundary conditions
    • 1.3.2. Atmosphere
    • 1.3.3. Ocean
    • 1.3.4. Hydrostatic, Quasi-hydrostatic, Quasi-nonhydrostatic and Non-hydrostatic forms
    • 1.3.5. Solution strategy
    • 1.3.6. Finding the pressure field
    • 1.3.7. Forcing/dissipation
    • 1.3.8. Vector invariant form
    • 1.3.9. Adjoint
  • 1.4. Appendix ATMOSPHERE
    • 1.4.1. Hydrostatic Primitive Equations for the Atmosphere in Pressure Coordinates
  • 1.5. Appendix OCEAN
    • 1.5.1. Equations of Motion for the Ocean
  • 1.6. Appendix OPERATORS
    • 1.6.1. Coordinate systems
• 2. Discretization and Algorithm
  • 2.1. Notation
  • 2.2. Time-stepping
  • 2.3. Pressure method with rigid-lid
  • 2.4. Pressure method with implicit linear free-surface
  • 2.5. Explicit time-stepping: Adams-Bashforth
    • 2.5.1. Adams-Bashforth II
    • 2.5.2. Adams-Bashforth III
  • 2.6. Implicit time-stepping: backward method
  • 2.7. Synchronous time-stepping: variables co-located in time
  • 2.8. Staggered baroclinic time-stepping
  • 2.9. Non-hydrostatic formulation
  • 2.10. Variants on the Free Surface
    • 2.10.1. Crank-Nicolson barotropic time stepping
    • 2.10.2. Non-linear free-surface
  • 2.11. Spatial discretization of the dynamical equations
    • 2.11.1. The finite volume method: finite volumes versus finite difference
    • 2.11.2. C grid staggering of variables
    • 2.11.3. Grid initialization and data
    • 2.11.4. Horizontal grid
    • 2.11.5. Vertical grid
    • 2.11.6. Topography: partially filled cells
  • 2.12. Continuity and horizontal pressure gradient term
  • 2.13. Hydrostatic balance
  • 2.14. Flux-form momentum equations
    • 2.14.1. Advection of momentum
    • 2.14.2. Coriolis terms
    • 2.14.3. Curvature metric terms
    • 2.14.4. Non-hydrostatic metric terms
    • 2.14.5. Lateral dissipation
    • 2.14.6. Vertical dissipation
    • 2.14.7. Derivation of discrete energy conservation
    • 2.14.8. Mom Diagnostics
  • 2.15. Vector invariant momentum equations
    • 2.15.1. Relative vorticity
    • 2.15.2. Kinetic energy
    • 2.15.3. Coriolis terms
    • 2.15.4. Shear terms
    • 2.15.5. Gradient of Bernoulli function
    • 2.15.6. Horizontal divergence
    • 2.15.7. Horizontal dissipation
    • 2.15.8. Vertical dissipation
  • 2.16. Tracer equations
    • 2.16.1. Time-stepping of tracers: ABII
  • 2.17. Advection schemes
    • 2.17.1. Linear advection schemes
    • 2.17.2. Non-linear advection schemes
    • 2.17.3. Comparison of advection schemes
  • 2.18. Shapiro Filter
    • 2.18.1. SHAP Diagnostics
  • 2.19. Nonlinear Viscosities for Large Eddy Simulation
    • 2.19.1. Eddy Viscosity
    • 2.19.2. Mercator, Nondimensional Equations
• 3. Getting Started with MITgcm
  • 3.1. Where to find information
  • 3.2. Obtaining the code
    • 3.2.1. Method 1
    • 3.2.2. Method 2
  • 3.3. Updating the code
  • 3.4. Model and directory structure
  • 3.5. Building the model
    • 3.5.1. Quickstart Guide
    • 3.5.2. Generating a Makefile using genmake2
    • 3.5.3. make commands
    • 3.5.4. Building with MPI
    • 3.5.5. Building with OpenMP
  • 3.6. Running the model
    • 3.6.1. Running with MPI
    • 3.6.2. Running with OpenMP
    • 3.6.3. Output files
    • 3.6.4. Looking at the output
  • 3.7. Customizing the Model Configuration - Code Parameters and Compilation Options
    • 3.7.1. Model Array Dimensions
    • 3.7.2. C Preprocessor Options
    • 3.7.3. Preprocessor Execution Environment Options
  • 3.8. Customizing the Model Configuration - Runtime Parameters
    • 3.8.1. Parameters: Configuration, Computational Domain, Geometry, and Time-Discretization
    • 3.8.2. Parameters: Main Algorithmic Parameters
    • 3.8.3. Parameters: Equation of State
    • 3.8.4. Parameters: Momentum Equations
    • 3.8.5. Parameters: Tracer Equations
    • 3.8.6. Parameters: Model Forcing
    • 3.8.7. Parameters: Simulation Controls
    • 3.8.8. Parameters Used In Optional Packages
    • 3.8.9. Execution Environment Parameters
  • 3.9. MITgcm Input Data File Format
• 4. MITgcm Tutorial Example Experiments
  • 4.1. Barotropic Ocean Gyre
    • 4.1.1. Equations Solved
    • 4.1.2. Discrete Numerical Configuration
    • 4.1.3. Configuration
    • 4.1.4. Building and running the model
    • 4.1.5. Model Solution
  • 4.2. Baroclinic Ocean Gyre
    • 4.2.1. Equations solved
    • 4.2.2. Discrete Numerical Configuration
    • 4.2.3. Configuration
    • 4.2.4. Building and running the model
    • 4.2.5. Running with MPI
    • 4.2.6. Running with OpenMP
    • 4.2.7. Model solution
  • 4.3. Southern Ocean Reentrant Channel Example
    • 4.3.1. Equations Solved
    • 4.3.2. Discrete Numerical Configuration
    • 4.3.3. Configuration
    • 4.3.4. Building and running the model
    • 4.3.5. Model Solution
  • 4.4. Ocean Gyre Advection Schemes
    • 4.4.1. Advection and tracer transport
    • 4.4.2. Introducing a tracer into the flow
    • 4.4.3. Selecting an advection scheme
    • 4.4.4. Comparison of different advection schemes
  • 4.5. Global Ocean Simulation
    • 4.5.1. Overview
    • 4.5.2. Discrete Numerical Configuration
    • 4.5.3. Experiment Configuration
  • 4.6. Global Ocean Simulation in Pressure Coordinates
    • 4.6.1. Overview
    • 4.6.2. Discrete Numerical Configuration
    • 4.6.3. Experiment Configuration
  • 4.7. Held-Suarez Atmosphere
    • 4.7.1. Overview
    • 4.7.2. Forcing
    • 4.7.3. Set-up description
    • 4.7.4. Experiment Configuration
  • 4.8. Deep Convection
    • 4.8.1. Overview
    • 4.8.2. Equations solved
    • 4.8.3. Discrete numerical configuration
    • 4.8.4. Numerical stability criteria and other considerations
    • 4.8.5. Experiment configuration
  • 4.9. Gravity Plume On a Continental Slope
    • 4.9.1. Configuration
    • 4.9.2. Binary input data
    • 4.9.3. Code configuration
    • 4.9.4. Model parameters
  • 4.10. Biogeochemistry Simulation
    • 4.10.1. Overview
    • 4.10.2. Equations Solved
    • 4.10.3. Code configuration
    • 4.10.4. Running the example
  • 4.11. Global Ocean State Estimation
    • 4.11.1. Overview
    • 4.11.2. Implementation of the control variable and the cost function
    • 4.11.3. Code Configuration
    • 4.11.4. Compiling
    • 4.11.5. Running the estimation
  • 4.12. Adjoint Sensitivity Analysis for Tracer Injection
    • 4.12.1. Overview of the experiment
    • 4.12.2. Code configuration
    • 4.12.3. Compiling the model and its adjoint
  • 4.13. Offline Experiments
    • 4.13.1. Overview
    • 4.13.2. Time-stepping of tracers
    • 4.13.3. Code Configuration
    • 4.13.4. Running the Experiment
    • 4.13.5. A more complicated example
  • 4.14. Rotating Tank
    • 4.14.1. Equations Solved
    • 4.14.2. Discrete Numerical Configuration
    • 4.14.3. Code Configuration
  • 4.15. Additional Example Experiments: Forward Model Setups
  • 4.16. Additional Example Experiments: Adjoint Model Setups
• 5. Contributing to the MITgcm
  • 5.1. Bugs and feature requests
  • 5.2. Using Git and Github
    • 5.2.1. Quickstart Guide
    • 5.2.2. Detailed guide for those less familiar with Git and GitHub
  • 5.3. Coding style guide
  • 5.4. Creating MITgcm packages
    • 5.4.1. Package structure
    • 5.4.2. Package boot sequence
    • 5.4.3. Package S/R calls
    • 5.4.4. Package “mypackage”
  • 5.5. MITgcm code testing protocols
    • 5.5.1. Test-experiment directory content
    • 5.5.2. The testreport utility
    • 5.5.3. The do_tst_2+2 utility
    • 5.5.4. Daily Testing of MITgcm
    • 5.5.5. Required Testing for MITgcm Code Contributors
  • 5.6. Contributing to the manual
    • 5.6.1. Section headings
    • 5.6.2. Internal document references
    • 5.6.3. Citations
    • 5.6.4. Other embedded links
    • 5.6.5. Symbolic Notation
    • 5.6.6. Figures
    • 5.6.7. Tables
    • 5.6.8. Other text blocks
    • 5.6.9. Other style conventions
    • 5.6.10. Building the manual
  • 5.7. Reviewing pull requests
• 6. Software Architecture
  • 6.1. Overall architectural goals
  • 6.2. WRAPPER
    • 6.2.1. Target hardware
    • 6.2.2. Supporting hardware neutrality
    • 6.2.3. WRAPPER machine model
    • 6.2.4. Machine model parallelism
    • 6.2.5. Communication mechanisms
    • 6.2.6. Communication primitives
    • 6.2.7. Memory architecture
    • 6.2.8. Summary
  • 6.3. Using the WRAPPER
    • 6.3.1. Specifying a domain decomposition
    • 6.3.2. Starting the code
    • 6.3.3. Controlling communication
  • 6.4. MITgcm execution under WRAPPER
    • 6.4.1. Annotated call tree for MITgcm and WRAPPER
    • 6.4.2. Measuring and Characterizing Performance
    • 6.4.3. Estimating Resource Requirements
• 7. Automatic Differentiation
  • 7.1. Some basic algebra
    • 7.1.1. Forward or direct sensitivity
    • 7.1.2. Reverse or adjoint sensitivity
    • 7.1.3. Storing vs. recomputation in reverse mode
  • 7.2. TLM and ADM generation in general
    • 7.2.1. General setup
    • 7.2.2. Building the AD code using TAF
    • 7.2.3. The AD build process in detail
    • 7.2.4. The cost function (dependent variable)
    • 7.2.5. The control variables (independent variables)
  • 7.3. The gradient check package
    • 7.3.1. Code description
    • 7.3.2. Code configuration
  • 7.4. Adjoint dump & restart – divided adjoint (DIVA)
    • 7.4.1. Introduction
    • 7.4.2. Recipe for divided adjoint code generation
    • 7.4.3. Special considerations for multi processor (MPI) runs
  • 7.5. Adjoint code generation using OpenAD
    • 7.5.1. Introduction
    • 7.5.2. Downloading and installing OpenAD
    • 7.5.3. Building MITgcm adjoint with OpenAD
    • 7.5.4. Building the MITgcm adjoint using an OpenAD Singularity container
  • 7.6. Adjoint code generation using Tapenade
    • 7.6.1. Introduction
    • 7.6.2. Downloading and installing Tapenade
    • 7.6.3. Prerequisites for Linux or Mac OS
    • 7.6.4. Steps for Mac OS
    • 7.6.5. Steps for Linux
    • 7.6.6. Prerequisites for Windows
    • 7.6.7. Steps for Windows
    • 7.6.8. Prerequisites for Tapenade setup
    • 7.6.9. Building MITgcm TLM with Tapenade
    • 7.6.10. Building MITgcm adjoint with Tapenade
• 8. Packages I - Physical Parameterizations
  • 8.1. Overview
    • 8.1.1. Using MITgcm Packages
  • 8.2. Packages Related to Hydrodynamical Kernel
    • 8.2.1. Generic Advection/Diffusion
    • 8.2.2. Momentum Packages
    • 8.2.3. Shapiro Filter
    • 8.2.4. FFT Filtering Code
    • 8.2.5. exch2: Extended Cubed Sphere Topology
    • 8.2.6. Gridalt - Alternate Grid Package
  • 8.3. General purpose numerical infrastructure packages
    • 8.3.1. OBCS: Open boundary conditions for regional modeling
    • 8.3.2. RBCS Package
    • 8.3.3. PTRACERS Package
  • 8.4. Ocean Packages
    • 8.4.1. GMREDI: Gent-McWilliams/Redi Eddy Parameterization
    • 8.4.2. KPP: Nonlocal K-Profile Parameterization for Vertical Mixing
    • 8.4.3. GGL90: a TKE vertical mixing scheme
    • 8.4.4. OPPS: Ocean Penetrative Plume Scheme
    • 8.4.5. KL10: Vertical Mixing Due to Breaking Internal Waves
    • 8.4.6. BULK_FORCE: Bulk Formula Package
    • 8.4.7. EXF: The external forcing package
    • 8.4.8. CAL: The calendar package
  • 8.5. Atmosphere Packages
    • 8.5.1. Atmospheric Intermediate Physics: AIM
    • 8.5.2. Land package
    • 8.5.3. Fizhi: High-end Atmospheric Physics
  • 8.6. Ice and Sea Ice Packages
    • 8.6.1. THSICE: The Thermodynamic Sea Ice Package
    • 8.6.2. SEAICE Package
    • 8.6.3. SHELFICE Package
    • 8.6.4. SHELFICE Remeshing
    • 8.6.5. STREAMICE Package
  • 8.7. Biogeochemistry Packages
    • 8.7.1. GCHEM Package
    • 8.7.2. DIC Package
• 9. Packages II - Diagnostics and I/O
  • 9.1. pkg/diagnostics – A Flexible Infrastructure
    • 9.1.1. Introduction
    • 9.1.2. Equations
    • 9.1.3. Key Subroutines and Parameters
    • 9.1.4. Usage Notes
  • 9.2. Fortran Native I/O: pkg/mdsio and pkg/rw
    • 9.2.1. pkg/mdsio
    • 9.2.2. pkg/rw basic binary I/O utilities
  • 9.3. NetCDF I/O: pkg/mnc
    • 9.3.1. Using pkg/mnc
    • 9.3.2. pkg/mnc Troubleshooting
    • 9.3.3. pkg/mnc Internals
  • 9.4. Monitor: Simulation State Monitoring Toolkit
    • 9.4.1. Introduction
    • 9.4.2. Using pkg/monitor
  • 9.5. Grid Generation
    • 9.5.1. Using SPGrid
    • 9.5.2. Example Grids
  • 9.6. Pre– and Post–Processing Scripts and Utilities
    • 9.6.1. Utilities Supplied With the Model
    • 9.6.2. Pre-Processing Software
  • 9.7. Potential Vorticity Matlab Toolbox
    • 9.7.1. Introduction
    • 9.7.2. Equations
    • 9.7.3. Key routines
    • 9.7.4. Technical details
    • 9.7.5. Notes on the flux form of the PV equation and vertical PV fluxes
  • 9.8. pkg/flt – Simulation of float / parcel displacements
    • 9.8.1. Introduction
    • 9.8.2. Compile-time options in FLT_OPTIONS.h
    • 9.8.3. Compile-time parameters in FLT_SIZE.h include:
    • 9.8.4. Run-time options in data.flt include:
    • 9.8.5. Input Files
    • 9.8.6. Output Files
    • 9.8.7. Verification Experiment
    • 9.8.8. Algorithm details
• 10. Ocean State Estimation Packages
  • 10.1. ECCO: model-data comparisons using gridded data sets
    • 10.1.1. Generic Cost Function
    • 10.1.2. Generic Integral Function
    • 10.1.3. Custom Cost Functions
    • 10.1.4. Key Routines
    • 10.1.5. Compile Options
  • 10.2. PROFILES: model-data comparisons at observed locations
  • 10.3. CTRL: Model Parameter Adjustment Capability
    • 10.3.1. Generic Control Parameters
    • 10.3.2. Shelfice Control Parameters
    • 10.3.3. Logarithmic Control Parameters
  • 10.4. SMOOTH: Smoothing And Covariance Model
  • 10.5. The line search optimisation algorithm
    • 10.5.1. General features
    • 10.5.2. The online vs. offline version
    • 10.5.3. Number of iterations vs. number of simulations
    • 10.5.4. Alternative code to optim and lsopt
  • 10.6. Test Cases For Estimation Package Capabilities
• 11. Utilities
  • 11.1. MITgcmutils
    • 11.1.1. mds
    • 11.1.2. mnc
    • 11.1.3. diagnostics
    • 11.1.4. ptracers
    • 11.1.5. jmd95
    • 11.1.6. mdjwf
    • 11.1.7. cs
    • 11.1.8. llc
    • 11.1.9. gluemncbig
• 12. Related Projects and Highlighted Papers
  • 12.1. Projects Related to MITgcm
    • 12.1.1. Estimating the Circulation and Climate of the Ocean (ECCO)
    • 12.1.2. Gcmfaces: Gridded Earth Variables In Matlab And Octave
    • 12.1.3. MITprof: In-Situ Ocean Data In Matlab And Octave
    • 12.1.4. OceanParcels - Lagrangian Particle Tracker
    • 12.1.5. Southern Ocean State Estimation (SOSE)
    • 12.1.6. Xgcm: General Circulation Model Postprocessing with xarray
    • 12.1.7. Xmitgcm
  • 12.2. Highlighted Papers
• 13. References