Checking Out and Running a One-Day Run

Method for Checking Out GEOS-5 GCM

Set up CVSROOT

First, set up your CVSROOT environment variable using the scheme provided NCCS's CVSACL webpage (requires NCCS login) where:

On Discover and NAS: CVSROOT=:ext:$USER@cvsacldirect:/cvsroot/esma

Elsewhere:           CVSROOT=:ext:$USER@ctunnel:/cvsroot/esma

Checking out the model

Make the directory in which you wish to checkout the model:

$ mkdir Ganymed-2_0_UNSTABLE
$ cd Ganymed-2_0_UNSTABLE/

And do the actual checkout using:

$ cvs co -r Ganymed-2_0_UNSTABLE Ganymed

In general:

$ cvs co -r <Tag Name> <Module Name>

where <Tag Name> is the tag for the model to check out (e.g., Ganymed-2_0_UNSTABLE, Fortuna-2_5_p6) and <Module Name> is the module (e.g., Ganymed, Fortuna).

Build and install the model

Go into the src/ directory of your model. Following above:

$ cd Ganymed-2_0_UNSTABLE/GEOSagcm/src

Setup the environment by sourcing the g5_modules file:

$ source g5_modules

To build the model, you have one of two choices. First, you can the parallel_build.csh script to submit a PBS job that compiles the model:

$ ./parallel_build.csh

or you can interactively build the model using:

$ gmake install

To capture the install log, we recommend tee'ing the output to a file:

$ gmake install |& tee make.install.log (on tcsh)
$ gmake install 2>&1 | tee make.install.log (on bash)

Note you can also build in parallel interactively with:

$ gmake -jN install |& tee make.install.log (on tcsh)

where N is the number of parallel processes. For best performance, N should be, say, 2 less than the number of cores. So, on a Westmere node, use 10. For the sake of others, do this on an interactive node.

Run the model

Setting up an experiment

Go into the model application directory and run the setup script:

$ cd Ganymed-2_0_UNSTABLE/GEOSagcm/src/Applications/GEOSgcm_App
$ ./gcm_setup

and provide required answers.

MAT: Add a sample run here

Run a one-day test

To run the job, first change to the experiment directory you specified in the above gcm_setup. Then, copy a set of restarts to model directory. Sample restarts are provided on Discover (at /discover/nobackup/mathomp4/Restarts-G10) and Pleiades (at /nobackup/gmao_SIteam/ModelData/Restarts-G10/).

Then, edit CAP.rc:

JOB_SGMT should be 1 day (00000001 000000)
NUM_SGMT should be 1

and gcm_run.j by inserting "exit" after

$RUN_CMD $NPES ./GEOSgcm.x
set rc =  $status
echo       Status = $rc 

A script that can automate these processes is:

#!/bin/bash

sed -i'.orig' -e '/^JOB_SGMT:/ s/000000[0-9][0-9]/00000001/' \
              -e '/^NUM_SGMT:/ s/[0-9][0-9]*/1/' \
              -e '/^MAPL_ENABLE_TIMERS:/ s/NO/YES/' CAP.rc

sed -i'.orig' -e '/^echo       Sta/ a exit' \
              -e '/^#PBS -j oe/ a #PBS -m abe\n#PBS -M email@myserver.com' \
              -e '/^  if(-e $EXPDIR\/$rst ) \/bin/ s/cp/ln -s/' gcm_run.j

Note: This script also sets up email notification as well as linking restarts to the scratch/ directory rather than copying (which is nice if running at high-res).

Compare to a "correct" run by using cmpscratch

a. If all success, runs are bit-identical b. If not => "What is correct?"