Regression tests

There are essentially 3 steps in system testing of AGCM model:

• checkout BASELINE version, run '1 day' and '1 step' experiments (one time operation)
• checkout CURRENT version, run '1 day' and '1 step' experiments (daily operation)
• compare outputs of BASELINE and CURRENT verions. Compare the restart and NetCDF-4 files, (TOTAL) times taken by each GridComp and memory usages as reported by MAPL timers.

The idea here is to checkout a "correct" version of the code, run an experiment and consider the resulting output to be the BASELINE version. Next, checkout the UNSTABLE tag, build it, run the experiment and compare the resulting output (CURRENT version) with the BASELINE version.

Once the BASELINE version has been run, running the CURRENT version and comparing outputs can be combined in a script and run nightly.

Test scripts

$ cvs co systest
$ cd systest

Running GCM_Run.py without any (command line) options provided prints out the usage information:

[discover15:systest]$ ./GCM_Run.py 
Usage: GCM_Run.py [options]

Options:
  -h, --help            show this help message and exit
  --exp_typ=EXP_TYP     [RQD] experiment type (AGCM/AOGCM)
  --tag=CVSTAG          [RQD] cvs tag
  --mod=CVSMOD          [RQD] cvs module
  --home_dir=HOMDIR     [RQD] Where we create bld/, run/, log/ (full path)
  --gcm_opts=GCMOPTS    [RQD] input file for gcm_setup (full path)
  --rst_dir=RSTDIR      [RQD] dir containing restart files (full path)
  --how_long=HOWLONG    [RQD] options: 1day/1step
  --exstng_bld=EXSTNGBLD
                        [OPT] existing build dir (full path)
  --spcl_tag=SPCLTAG    [OPT] special tag to update to
  --pbld                [OPT] parallel build (default = False)
  --timer               [OPT] turn MAPL timers ON (default = OFF)
  --memutils            [OPT] turn MAPL memutils ON (dafault = OFF)
  --lt_test             [OPT] layout/transparency test (default = False)

Input file

The script GCM_Run.py requires an input file passed to it through the flag '--gcm_opts'. This is a template file specifying the input options to gcm_setup. Two example input files are provided

input/gcm_setup_AGCM_b_FVqq_NO_O1_NO.input
input/gcm_setup_AGCM_b_FV3_NO_O1_NO.input

The contents of gcm_setup_AGCM_b_FV_NO_O1_NO.input are:

Experiment ID                     = TO_BE_IGNORED
Experiment Description            = TO_BE_IGNORED
Atmospheric Horizontal Resolution = b
Dynamical Core                    = FV
COUPLED                           = NO
Data_ocean Horizontal Resolution  = o1
GOCART                            = NO
HISTORY template file             = HISTORY.AGCM.rc.tmpl
Home Dir                          = TO_BE_IGNORED
Experiment Dir                    = TO_BE_IGNORED
Build Dir                         = TO_BE_IGNORED
GROUP ID                          = g0620

The value TO_BE_IGNORED is (as the name suggests) ignored. The values for these fields are generated by the scripts depending on the experiment duration. For a '1 step' run, Experiment ID is set to '1step' and Experiment Description is set to '1step_run'. The location of Home/Experiment/Build directories are discussed below.

The remaining fields require a valid value to be specified.

• 'Atmospheric Horizontal Resolution' has to be one of b, c, d and e.
• 'Dynamical Core' is one of FV, FV3
• 'COUPLED' is NO for AGCM runs.
• 'Data_ocean Horizontal Resolution' is one of o1, o8.
• 'GOCART' is NO (GOCART is not currently supported).
• 'GROUP ID' is the user's group id.