Running ADAS

DAS assimilation and forecast runs

Select DAS run and corresponding build

First, we choose a DAS run to simulate. For a list of runs

$ ls ~dao_ops 

on NCCS discover. As an example, we choose the run d591_fpit (for more about GMAO products, please see https://gmao.gsfc.nasa.gov/products)

Look for ATAG in file ~dao_ops/d591_fpit/run/FVDAS_Run_Config

$ grep ATAG ~dao_ops/d591_fpit/run/FVDAS_Run_Config

 setenv ATAG    GEOSadas-5_9_1_p8
 setenv FVARCH   /archive/u/gmao_ops/GEOS-5.9.1/${ATAG}/ #include trailing colon

This gives us the tag and location of the restart files.

Move to the bin directory (of the existing build) to setup experiment.

$ cd ~dao_ops/GEOSadas-5_9_1_p4/GEOSadas/Linux/bin

Setup experiment

Run fvsetup from the bin directory with the following options (default value is chosen for the ones not mentioned): [For now we choose the default value for FVICS as well]

AGCM Resolution? [b72]
> d72

EXPID? [u000_d72]
> g591p4_d72

OBSERVING SYSTEM CLASSES?
> "ncep_prep_bufr"

CHECKING OBSYSTEM? [2]
> 3

After saving these inputs, move to the experiment directory

$ cd $NOBACKUP/g591p4_d72/run

At this point, one can edit CAP.rc.tmpl to run 12-hour experiment. Also, one can turn on MAPL timers and memutils

JOB_SGMT:     0 120000
NUM_SGMT:     1

MAPL_ENABLE_TIMERS: YES
MAPL_ENABLE_MEMUTILS: YES

To run one segment each of GSIsa.x and GEOSgcm.x add 'exit' before 'PART IV - Next PBS Job Segment' in g5das.j.

Data Assimilation experiment

$ cd $NOBACKUP/g591p4_d72/run
$ qsub g5das.j

g5das creates a scratch directory $NOBACKUP/fvwork.xxxx and runs the experiment there. After the initial setup, GSIsa.x is run with output written to ana.log followed by GEOSgcm.x which writes its output to stdout and fvpsas.log.

Forecast experiment

Later.

Simulating DAS forecast run

The DAS forecast run can be simulated using the corresponding AGCM tag.

Checkout and build the model

Grab the corresponding AGCM tag (Check out GEOSadas-5_9_1_p4 and look for GCM in the file src/Applications/GEOSdas_App/ChangeLog).

 $ cvs co -r Ganymed-1_0_p3 Ganymed
 $ cd GEOSagcm/src
 $ source g5_modules
 $ ./parallel_build.csh

Experiment setup

To simulate a forecast run we can use gcm_setup to setup our experiment (instead of fvsetup). Run ./gcm_setup and provide the required answers

 $ cd Applications/GEOSgcm_App/
 $ ./gcm_setup
        GOCART: yes
        GOCART Emission Files to use: OPS
        AERO_PROVIDER: GOCART
        HISTORY: Current

Edit files

AGCM.rc, CAP.rc, HISTORY.rc

Edit these files to match with the corresponding .tmpl files in

 ~dao_ops/e572p5_fp/run

Then, change the restart reading behavior to pbinary:

 $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc

gcm_run.j

Ensure that the correct boundary datasets are linked to. Compare the links in this file to that of

 ~dao_ops/e572p5_fp/run/lnbcs

Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).

RC files

Copy the .rc files over

 $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;

Restart files

Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config

 $ grep ATAG ~dao_ops/e572p5_fp/run/FVDAS_Run_Config

   setenv ATAG    GEOSadas-5_7_2_p5_m1
   setenv ATAG    $GTAG
   setenv FVARCH   /archive/u/gmao_ops/GEOS-5.7.2/${ATAG}/ #include trailing colon

Pick a date and time: say 2012/05/15, 21z

 $ cd /archive/u/gmao_ops/GEOS-5.7.2/GEOSadas-5_7_2_p5_m1/e572p5_fp/rs/Y2012/M05
 $ ls *_rst.20120515_21z.bin

Copy these files over from ARCHIVE to experiment directory and the rename them:

 $ cd /path/to/experiment
 $ rename _rst.20120515_21z.bin _rst *_rst.20120515_21z.bin
 $ rename e572p5_fp. '' *rst

Also, create a cap_restart file:

 $ echo '20120515 210000' > cap_restart

Run job

 $ qsub gcm_run.j