Running ADAS: Difference between revisions

From GEOS-5
Jump to navigation Jump to search
← Older edit
Pchakrab (talk | contribs)
1,048 edits
Pchakrab (talk | contribs)
1,048 edits
 
(84 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Select DAS run and corresponding build==
==ADAS run==


First, we choose a DAS run to simulate. For a list of runs
===Build model===
$ ls ~dao_ops
First, choose a model tag to build. For this example, we choose GEOSadas-5_9_1_p7.
on NCCS discover. As an example, we choose the run <tt>d591_fpit</tt>


Look for ATAG in file <tt>~dao_ops/d591_fpit/run/FVDAS_Run_Config</tt>
  $ cvs co -P -r GEOSadas-5_9_1_p7 -d g591p8 GEOSadas-5_9
  $ grep ATAG ~dao_ops/d591_fpit/run/FVDAS_Run_Config
    setenv ATAG    GEOSadas-5_9_1_p4
    setenv FVARCH  /archive/u/gmao_ops/GEOS-5.9.1/${ATAG}/ #include trailing colon


This gives us the tag and location of the restart files [for an older build, it is best to checkout the model tag (given by ATAG) and rebuild the model].
Not the module name <tt>GEOSadas-5_9</tt>. Unlike AGCM, all ADAS tags have a corresponding module.


Move to the bin directory (of the existing build) to setup experiment.
  $ cd g591p8/src
  $ cd ~dao_ops/GEOSadas-5_9_1_p4/GEOSadas/Linux/bin
$ ./parallel_build.csh


==Setup experiment==
===Setup experiment (Run <tt>fvsetup</tt>)===


Run <tt>fvsetup</tt> from the bin directory with the following options (default value is chosen for the ones not mentioned):
Run <tt>fvsetup</tt> from the bin directory (<tt>g591p8/Linux/bin</tt>) with the following options (default value chosen for ones not mentioned):
[we choose the default value for the initial conditions (FVICS) as well]


  AGCM Resolution? [b72]
  AGCM Resolution? [b72]
  > d72
  > C180
   
   
  OGCM Resolution? [c34]
  OGCM Resolution? [c34]
  >  
  > f34
   
   
  EXPID? [u000_d72]
  EXPID? [u000_C180]
  > g591p4
  > g591p8
   
   
  FVICS? [/archive/u/jstassi/restarts/591]
  OBSERVING SYSTEM CLASSES?
  > /archive/u/gmao_ops/GEOS-5.9.1/GEOSadas-5_9_1_p4/d591_fpit/rs/Y2012/M10
  > "ncep_prep_bufr"
<!--
> "ncep_1bamua_bufr,ncep_1bamub_bufr,ncep_1bhrs2_bufr,ncep_1bhrs3_bufr,
    ncep_1bmsu_bufr,ncep_osbuv_bufr,ncep_prep_bufr,ncep_sptrmm_bufr,
    disc_airs_bufr,disc_amsua_bufr,ncep_mhs_bufr,ncep_1bhrs4_bufr,
    ncep_amsre_bufr,ncep_goesfv_bufr,ncep_mtiasi_bufr,ncep_gpsro_bufr,
    aura_omi_bufr,ncep_satwnd_bufr,ncep_atms_bufr,ncep_sevcsr_bufr"
-->
   
   
  Starting year-month-day? [20120315]
  Do Aerosol Analysis (y/n)? [y]
  > 20121001
  > n
   
 
  Ending year-month-day? [20121003]
One can, of course, turn on Aerosol Analysis and AOD assimilation here.
  > 20121002
 
   
A pdf of an example <tt>fvsetup</tt> session is provided here: [[Media:Fvsetup_session.pdf|fvsetup_session]].
  CHECKING OBSYSTEM? [2]
 
  > 3
====FVHOME====
FVHOME is the home directory for fvDAS experiment. Resource files, restarts and system output will be stored
under this directory. Usually it contains subdiretories:
 
  ana/        first guess/analysis output
  anasa/      job script to run stand-alone analysis
daotovs/    daotovs field output
diag/      diagnostic field output
etc/        listings and other odds & ends
fcst/      forecast run directory
fvInput/    process required inputs
obs/        post-analysis ODS files
  prog/      prognostic field output
  recycle/    latest restart files
  rs/        restart files
  run/        resource files


Be sure to save these inputs.
<tt>$FVHOME/.FVROOT</tt> gives the location of the installation <tt>bin</tt> directory.


==Run data assimilation experiment==
Some of these directories (<tt>ana/</tt>, <tt>chem/</tt>, <tt>diag/</tt>, <tt>etc/</tt>, <tt>obs/</tt>, <tt>rs/</tt>) are stored in the archive. After completing the experiment, look in <tt>$ARCHIVE/g591p8</tt> (where <tt>g591p8</tt> is the experiment ID).


Move to the experiment directory
====Run directory====
  $ cd $NOBACKUP/g591p4/run
  $ cd $FVHOME/run


At this point, one can edit CAP.rc to run 1-day experiment. Also, one can turn on MAPL timers and memutils
<tt>g5das.j</tt> is the job submission script. The number of cpus etc. can be changed here (if they haven't already been changed during setup). Also, if you want to run a single cycle of <tt>GSIsa.x</tt> & <tt>GEOSgcm.x</tt> add <tt>exit</tt> right before 'PART IV'.
<!--
To simulate the OPS run d591_fpit, we make a few changes:
# NX, NY in AGCM.rc.tmpl and AGCM.BOOTSTRAP.rc.tmpl have the same values (8, 48) as in the OPS run
# Copy HISTORY.rc.tmpl over from the OPS run
# Copy obsys.rc over from the OPT run
# NX, NY in GSI_GridComp.rc.tmpl should match those from the OPS run (6, 36)
# Set NUM_SGMT to 1 in CAP.rc.tmpl
# <tt>g5das.j</tt>
## PBS -l select=32
## <tt>setenv NCPUS    384</tt>
## <tt>setenv NCPUS_GSI 216</tt>
## '''One cycle''': To run one segment each of GSIsa.x and GEOSgcm.x add '<tt>exit</tt>' before 'PART IV'
-->


JOB_SGMT:    1 000000
===Data Assimilation===
NUM_SGMT:    1
MAPL_ENABLE_TIMERS: YES
MAPL_ENABLE_MEMUTILS: YES


Run the experiment
$ cd $FVHOME/run
  $ qsub g5das.j
  $ qsub g5das.j


==Old documentation==
<tt>g5das</tt> creates a scratch directory <tt>$NOBACKUP/fvwork.xxxx</tt> and runs the experiment there. After the initial setup, <tt>GSIsa.x</tt> is run with output written to <tt>ana.log</tt> followed by <tt>GEOSgcm.x</tt> which writes its output to <tt>stdout</tt> and <tt>fvpsas.log</tt>.
1. Checkout and build the model:


Check with Joe Stassi for model tag and module name.
===Forecast===


  $ cvs co -r GEOSadas-5_7_2_p5_m1 GEOSadas-5_7
==Simulating OPS run==
  $ cd GEOSadas/src
<!--EVERYTHING BELOW IS COMMENTED OUT
  $ source g5_modules
 
  $ gmake install
==Simulating DAS forecast run==
 
The DAS forecast run can be simulated using the corresponding AGCM tag.


Or, one can checkout the corresponding AGCM model Fortuna-2_5_p6 (module
====Checkout and build the model====
Fortuna) and build it.


Note: For GEOSadas-5_7_2_p5_m1 in the example above, g5_modules will need to be edited in order to compile successfully on discover (due to SP1). You should change the discover section to be (where the '''bold''' items are changes, and mod5 is removed):
Grab the corresponding AGCM tag (Check out <tt>GEOSadas-5_9_1_p4</tt> and look for <tt>GCM</tt> in the file <tt>src/Applications/GEOSdas_App/ChangeLog</tt>).


  '''set basedir = /discover/nobackup/projects/gmao/share/gmao_ops/Baselibs/v3.2.0_buildSLES11SP1'''
  $ cvs co -r Ganymed-1_0_p3 Ganymed
  set mod1 = comp/intel-11.0.083
  $ cd GEOSagcm/src
  set mod2 = mpi/impi-3.2.2.006
  $ source g5_modules
  set mod3 = lib/mkl-10.0.3.020
  $ ./parallel_build.csh
  '''set mod4 = other/SIVO-PyD/spd_1.6.0_gcc-4.3.4-sp1'''
 
  '''set mods = ( $mod1 $mod2 $mod3 $mod4''' )
  set modinit = /usr/share/modules/init/csh
  set loadmodules = 0


2. Experiment setup:
====Experiment setup====


To simulate a forecast run we can use gcm_setup to setup our experiment
To simulate a forecast run we can use gcm_setup to setup our experiment
Line 91: Line 113:
   $ cd Applications/GEOSgcm_App/
   $ cd Applications/GEOSgcm_App/
   $ ./gcm_setup
   $ ./gcm_setup
        GOCART: yes
        GOCART Emission Files to use: OPS
        AERO_PROVIDER: GOCART
        HISTORY: Current


Cubed-sphere was not fully implemented in this model. Choose Lat/Lon with
=====Edit files=====
resolution 1152x721x72 for the 'e'-run. The other options are
======AGCM.rc, CAP.rc, HISTORY.rc======
 
  GOCART: yes
  GOCART Emission Files to use: OPS
  AERO_PROVIDER: GOCART
  HISTORY: Current
 
3. AGCM.rc, CAP.rc, HISTORY.rc


Edit these files to match with the corresponding .tmpl files in
Edit these files to match with the corresponding .tmpl files in
Line 109: Line 128:
   $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc
   $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc


4. gcm_run.j
======gcm_run.j======


Ensure that the correct boundary datasets are linked to. Compare the links in
Ensure that the correct boundary datasets are linked to. Compare the links in
Line 118: Line 137:
Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).
Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).


5. RC files
=====RC files=====


Copy the .rc files over
Copy the .rc files over
Line 124: Line 143:
   $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;
   $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;


6. restart files
=====Restart files=====


Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config
Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config
Line 149: Line 168:
   $ echo '20120515 210000' > cap_restart
   $ echo '20120515 210000' > cap_restart


7. Run job
====Run job====
    
    
   $ qsub gcm_run.j
   $ qsub gcm_run.j
-->


[[Category:SI Team]]
[[Category:SI Team]]
[[Category:Running the Model]]
[[Category:Running the Model]]

Latest revision as of 09:38, 29 March 2013

ADAS run

Build model

First, choose a model tag to build. For this example, we choose GEOSadas-5_9_1_p7. 

$ cvs co -P -r GEOSadas-5_9_1_p7 -d g591p8 GEOSadas-5_9

Not the module name GEOSadas-5_9. Unlike AGCM, all ADAS tags have a corresponding module. 

$ cd g591p8/src
$ ./parallel_build.csh

Setup experiment (Run fvsetup)

Run fvsetup from the bin directory (g591p8/Linux/bin) with the following options (default value chosen for ones not mentioned): [we choose the default value for the initial conditions (FVICS) as well] 

AGCM Resolution? [b72]
> C180

OGCM Resolution? [c34]
> f34

EXPID? [u000_C180]
> g591p8

OBSERVING SYSTEM CLASSES?
> "ncep_prep_bufr"

Do Aerosol Analysis (y/n)? [y]
> n

One can, of course, turn on Aerosol Analysis and AOD assimilation here.

A pdf of an example fvsetup session is provided here: fvsetup_session.

FVHOME

FVHOME is the home directory for fvDAS experiment. Resource files, restarts and system output will be stored under this directory. Usually it contains subdiretories: 

ana/        first guess/analysis output
anasa/      job script to run stand-alone analysis
daotovs/    daotovs field output
diag/       diagnostic field output
etc/        listings and other odds & ends
fcst/       forecast run directory
fvInput/    process required inputs
obs/        post-analysis ODS files
prog/       prognostic field output
recycle/    latest restart files
rs/         restart files
run/        resource files

$FVHOME/.FVROOT gives the location of the installation bin directory.

Some of these directories (ana/, chem/, diag/, etc/, obs/, rs/) are stored in the archive. After completing the experiment, look in $ARCHIVE/g591p8 (where g591p8 is the experiment ID).

Run directory

$ cd $FVHOME/run

g5das.j is the job submission script. The number of cpus etc. can be changed here (if they haven't already been changed during setup). Also, if you want to run a single cycle of GSIsa.x & GEOSgcm.x add exit right before 'PART IV'.

Data Assimilation

$ cd $FVHOME/run
$ qsub g5das.j

g5das creates a scratch directory $NOBACKUP/fvwork.xxxx and runs the experiment there. After the initial setup, GSIsa.x is run with output written to ana.log followed by GEOSgcm.x which writes its output to stdout and fvpsas.log.

Forecast

Simulating OPS run

Retrieved from "https://geos5.org/wiki/index.php?title=Running_ADAS&oldid=1939"