Running ADAS

Select DAS run and corresponding build

First, we choose a DAS run to simulate. For a list of runs

$ ls ~dao_ops 

on NCCS discover. As an example, we choose the run d591_fpit

Look for ATAG in file ~dao_ops/d591_fpit/run/FVDAS_Run_Config

$ grep ATAG ~dao_ops/d591_fpit/run/FVDAS_Run_Config

    setenv ATAG    GEOSadas-5_9_1_p4
    setenv FVARCH   /archive/u/gmao_ops/GEOS-5.9.1/${ATAG}/ #include trailing colon

This gives us the tag and location of the restart files [for an older build, it is best to checkout the model tag (given by ATAG) and rebuild the model].

Move to the bin directory (of the existing build) to setup experiment.

$ cd ~dao_ops/GEOSadas-5_9_1_p4/GEOSadas/Linux/bin

Setup experiment

Run fvsetup from the bin directory with the following options (default value is chosen for the ones not mentioned):

AGCM Resolution? [b72]
> d72

OGCM Resolution? [c34]
> 

EXPID? [u000_d72]
> g591p4

FVICS? [/archive/u/jstassi/restarts/591]
> /archive/u/gmao_ops/GEOS-5.9.1/GEOSadas-5_9_1_p4/d591_fpit/rs/Y2012/M09

Starting year-month-day? [20120315]
> 20120901

Ending year-month-day? [20121003]
> 20120902

Length of FORECAST run segments (in hours)? [123]
> 24

Be sure to save these inputs.

Run data assimilation experiment

Move to the experiment directory

$ cd $NOBACKUP/g591p4/run

At this point, one can edit CAP.rc to run 1-day experiment. Also, one can turn on MAPL timers and memutils

JOB_SGMT:     1 000000
NUM_SGMT:     1

MAPL_ENABLE_TIMERS: YES
MAPL_ENABLE_MEMUTILS: YES

Run the experiment

$ qsub g5das.j

Old documentation

1. Checkout and build the model:

Check with Joe Stassi for model tag and module name.

 $ cvs co -r GEOSadas-5_7_2_p5_m1 GEOSadas-5_7
 $ cd GEOSadas/src
 $ source g5_modules
 $ gmake install

Or, one can checkout the corresponding AGCM model Fortuna-2_5_p6 (module Fortuna) and build it.

Note: For GEOSadas-5_7_2_p5_m1 in the example above, g5_modules will need to be edited in order to compile successfully on discover (due to SP1). You should change the discover section to be (where the bold items are changes, and mod5 is removed):

  set basedir = /discover/nobackup/projects/gmao/share/gmao_ops/Baselibs/v3.2.0_buildSLES11SP1
  set mod1 = comp/intel-11.0.083
  set mod2 = mpi/impi-3.2.2.006
  set mod3 = lib/mkl-10.0.3.020
  set mod4 = other/SIVO-PyD/spd_1.6.0_gcc-4.3.4-sp1
  
  set mods = ( $mod1 $mod2 $mod3 $mod4 )
  set modinit = /usr/share/modules/init/csh
  set loadmodules = 0 

2. Experiment setup:

To simulate a forecast run we can use gcm_setup to setup our experiment (instead of fvsetup). Run ./gcm_setup and provide the required answers

 $ cd Applications/GEOSgcm_App/
 $ ./gcm_setup

Cubed-sphere was not fully implemented in this model. Choose Lat/Lon with resolution 1152x721x72 for the 'e'-run. The other options are

 GOCART: yes
 GOCART Emission Files to use: OPS
 AERO_PROVIDER: GOCART
 HISTORY: Current

3. AGCM.rc, CAP.rc, HISTORY.rc

Edit these files to match with the corresponding .tmpl files in

 ~dao_ops/e572p5_fp/run

Then, change the restart reading behavior to pbinary:

 $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc

4. gcm_run.j

Ensure that the correct boundary datasets are linked to. Compare the links in this file to that of

 ~dao_ops/e572p5_fp/run/lnbcs

Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).

5. RC files

Copy the .rc files over

 $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;

6. restart files

Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config

 $ grep ATAG ~dao_ops/e572p5_fp/run/FVDAS_Run_Config

   setenv ATAG    GEOSadas-5_7_2_p5_m1
   setenv ATAG    $GTAG
   setenv FVARCH   /archive/u/gmao_ops/GEOS-5.7.2/${ATAG}/ #include trailing colon

Pick a date and time: say 2012/05/15, 21z

 $ cd /archive/u/gmao_ops/GEOS-5.7.2/GEOSadas-5_7_2_p5_m1/e572p5_fp/rs/Y2012/M05
 $ ls *_rst.20120515_21z.bin

Copy these files over from ARCHIVE to experiment directory and the rename them:

 $ cd /path/to/experiment
 $ rename _rst.20120515_21z.bin _rst *_rst.20120515_21z.bin
 $ rename e572p5_fp. '' *rst

Also, create a cap_restart file:

 $ echo '20120515 210000' > cap_restart

7. Run job

 $ qsub gcm_run.j