Running ADAS

Select DAS run and corresponding build

First, we choose a DAS run to simulate. For a list of runs

$ ls ~dao_ops 

on NCCS discover. As an example, we choose the run e572p5_fp

Look for ATAG in file ~dao_ops/e572p5_fp/run/FVDAS_Run_Config

$ grep ATAG ~dao_ops/e572p5_fp/run/FVDAS_Run_Config

    setenv ATAG    GEOSadas-5_7_2_p5_m1
    setenv ATAG    $GTAG
    setenv FVARCH   /archive/u/gmao_ops/GEOS-5.7.2/${ATAG}/ #include trailing colon

This gives us the tag and location of the restart files.

Older build

For an older build (as in the present case), it is best to rebuild the model.

$ mkdir -p $NOBACKUP/G572p5m1
$ cd $NOBACKUP/G572p5m1
$ cvs co -r GEOSadas-5_7_2_p5_m1 GEOSadas   # [checkout model]
$ cd GEOSadas/src                           # [source directory]
$ cvs up -r Ganymed-2_0_UNSTABLE g5_modules # [update to its latest version]
$ source g5_modules                         # [setup environment]
$ ./parallel_build.csh                      # [build]
$ cd ../Linux/bin                           # [move to bin directory]

Newer build

Move to the bin directory (~dao_ops/GEOSadas-5_7_2_p5_m1/GEOSadas/Linux/bin) to setup experiment.

Setup experiment

Run fvsetup from the bin directory with the following options (default value is chosen for the ones not mentioned):

Resolution? [b72]
> e72

EXPID? [u000_e72]
> g572p5m1

EXPDSC? [g572p5m1,GEOSadas-5_7_2_p5_m1_UNSTABLE,1st_try]
> g572p5m1,GEOSadas-5_7_2_p5_m1_UNSTABLE,1st_try

FVHOME? [/discover/nobackup/pchakrab/g572p5m1]
> /discover/nobackup/pchakrab/runs/ADAS/$expid

FVICS?
> /archive/u/gmao_ops/GEOS-5.7.2/GEOSadas-5_7_2_p5_m1/e572p5_fp/rs/Y2012/M05

Old documentation

1. Checkout and build the model:

Check with Joe Stassi for model tag and module name.

 $ cvs co -r GEOSadas-5_7_2_p5_m1 GEOSadas-5_7
 $ cd GEOSadas/src
 $ source g5_modules
 $ gmake install

Or, one can checkout the corresponding AGCM model Fortuna-2_5_p6 (module Fortuna) and build it.

Note: For GEOSadas-5_7_2_p5_m1 in the example above, g5_modules will need to be edited in order to compile successfully on discover (due to SP1). You should change the discover section to be (where the bold items are changes, and mod5 is removed):

  set basedir = /discover/nobackup/projects/gmao/share/gmao_ops/Baselibs/v3.2.0_buildSLES11SP1
  set mod1 = comp/intel-11.0.083
  set mod2 = mpi/impi-3.2.2.006
  set mod3 = lib/mkl-10.0.3.020
  set mod4 = other/SIVO-PyD/spd_1.6.0_gcc-4.3.4-sp1
  
  set mods = ( $mod1 $mod2 $mod3 $mod4 )
  set modinit = /usr/share/modules/init/csh
  set loadmodules = 0 

2. Experiment setup:

To simulate a forecast run we can use gcm_setup to setup our experiment (instead of fvsetup). Run ./gcm_setup and provide the required answers

 $ cd Applications/GEOSgcm_App/
 $ ./gcm_setup

Cubed-sphere was not fully implemented in this model. Choose Lat/Lon with resolution 1152x721x72 for the 'e'-run. The other options are

 GOCART: yes
 GOCART Emission Files to use: OPS
 AERO_PROVIDER: GOCART
 HISTORY: Current

3. AGCM.rc, CAP.rc, HISTORY.rc

Edit these files to match with the corresponding .tmpl files in

 ~dao_ops/e572p5_fp/run

Then, change the restart reading behavior to pbinary:

 $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc

4. gcm_run.j

Ensure that the correct boundary datasets are linked to. Compare the links in this file to that of

 ~dao_ops/e572p5_fp/run/lnbcs

Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).

5. RC files

Copy the .rc files over

 $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;

6. restart files

Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config

 $ grep ATAG ~dao_ops/e572p5_fp/run/FVDAS_Run_Config

   setenv ATAG    GEOSadas-5_7_2_p5_m1
   setenv ATAG    $GTAG
   setenv FVARCH   /archive/u/gmao_ops/GEOS-5.7.2/${ATAG}/ #include trailing colon

Pick a date and time: say 2012/05/15, 21z

 $ cd /archive/u/gmao_ops/GEOS-5.7.2/GEOSadas-5_7_2_p5_m1/e572p5_fp/rs/Y2012/M05
 $ ls *_rst.20120515_21z.bin

Copy these files over from ARCHIVE to experiment directory and the rename them:

 $ cd /path/to/experiment
 $ rename _rst.20120515_21z.bin _rst *_rst.20120515_21z.bin
 $ rename e572p5_fp. '' *rst

Also, create a cap_restart file:

 $ echo '20120515 210000' > cap_restart

7. Run job

 $ qsub gcm_run.j