Running ADAS: Difference between revisions

From GEOS-5
Jump to navigation Jump to search
← Older edit
Pchakrab (talk | contribs)
1,048 edits
Pchakrab (talk | contribs)
1,048 edits
 
(75 intermediate revisions by the same user not shown)
Line 1: Line 1:
==DAS assimilation and forecast runs==
==ADAS run==
====Select DAS run and corresponding build====


First, we choose a DAS run to simulate. For a list of runs
===Build model===
$ ls ~dao_ops
First, choose a model tag to build. For this example, we choose GEOSadas-5_9_1_p7.
on NCCS discover. As an example, we choose the run <tt>d591_fpit</tt>


Look for ATAG in file <tt>~dao_ops/d591_fpit/run/FVDAS_Run_Config</tt>
$ cvs co -P -r GEOSadas-5_9_1_p7 -d g591p8 GEOSadas-5_9
$ grep ATAG ~dao_ops/d591_fpit/run/FVDAS_Run_Config
 
Not the module name <tt>GEOSadas-5_9</tt>. Unlike AGCM, all ADAS tags have a corresponding module.
    setenv ATAG    GEOSadas-5_9_1_p4
    setenv FVARCH  /archive/u/gmao_ops/GEOS-5.9.1/${ATAG}/ #include trailing colon


This gives us the tag and location of the restart files [for an older build, it is best to checkout the model tag (given by ATAG) and rebuild the model].
$ cd g591p8/src
$ ./parallel_build.csh


Move to the bin directory (of the existing build) to setup experiment.
===Setup experiment (Run <tt>fvsetup</tt>)===
$ cd ~dao_ops/GEOSadas-5_9_1_p4/GEOSadas/Linux/bin


====Setup experiment====
Run <tt>fvsetup</tt> from the bin directory (<tt>g591p8/Linux/bin</tt>) with the following options (default value chosen for ones not mentioned):
[we choose the default value for the initial conditions (FVICS) as well]


Run <tt>fvsetup</tt> from the bin directory with the following options (default value is chosen for the ones not mentioned):
[For now we choose the default value for FVICS as well]
  AGCM Resolution? [b72]
  AGCM Resolution? [b72]
  > d72
  > C180
   
   
  OGCM Resolution? [c34]
  OGCM Resolution? [c34]
  >  
  > f34
   
   
  EXPID? [u000_d72]
  EXPID? [u000_C180]
  > g591p4
  > g591p8
   
   
  OBSERVING SYSTEM CLASSES?
  OBSERVING SYSTEM CLASSES?
  > "ncep_prep_bufr"
  > "ncep_prep_bufr"
<!--
> "ncep_1bamua_bufr,ncep_1bamub_bufr,ncep_1bhrs2_bufr,ncep_1bhrs3_bufr,
    ncep_1bmsu_bufr,ncep_osbuv_bufr,ncep_prep_bufr,ncep_sptrmm_bufr,
    disc_airs_bufr,disc_amsua_bufr,ncep_mhs_bufr,ncep_1bhrs4_bufr,
    ncep_amsre_bufr,ncep_goesfv_bufr,ncep_mtiasi_bufr,ncep_gpsro_bufr,
    aura_omi_bufr,ncep_satwnd_bufr,ncep_atms_bufr,ncep_sevcsr_bufr"
-->
   
   
  CHECKING OBSYSTEM? [2]
  Do Aerosol Analysis (y/n)? [y]
  > 3
  > n


After saving these inputs, move to the experiment directory
One can, of course, turn on Aerosol Analysis and AOD assimilation here.
$ cd $NOBACKUP/g591p4/run


At this point, one can edit CAP.rc to run 12-hour experiment. Also, one can turn on MAPL timers and memutils
A pdf of an example <tt>fvsetup</tt> session is provided here: [[Media:Fvsetup_session.pdf|fvsetup_session]].


  JOB_SGMT:    0 120000
====FVHOME====
  NUM_SGMT:    1
FVHOME is the home directory for fvDAS experiment. Resource files, restarts and system output will be stored
   
under this directory. Usually it contains subdiretories:
  MAPL_ENABLE_TIMERS: YES
 
MAPL_ENABLE_MEMUTILS: YES
ana/        first guess/analysis output
anasa/      job script to run stand-alone analysis
daotovs/    daotovs field output
  diag/      diagnostic field output
etc/        listings and other odds & ends
fcst/      forecast run directory
fvInput/    process required inputs
obs/        post-analysis ODS files
prog/      prognostic field output
recycle/    latest restart files
  rs/        restart files
  run/        resource files
 
<tt>$FVHOME/.FVROOT</tt> gives the location of the installation <tt>bin</tt> directory.
 
Some of these directories (<tt>ana/</tt>, <tt>chem/</tt>, <tt>diag/</tt>, <tt>etc/</tt>, <tt>obs/</tt>, <tt>rs/</tt>) are stored in the archive. After completing the experiment, look in <tt>$ARCHIVE/g591p8</tt> (where <tt>g591p8</tt> is the experiment ID).
 
====Run directory====
  $ cd $FVHOME/run
 
<tt>g5das.j</tt> is the job submission script. The number of cpus etc. can be changed here (if they haven't already been changed during setup). Also, if you want to run a single cycle of <tt>GSIsa.x</tt> & <tt>GEOSgcm.x</tt> add <tt>exit</tt> right before 'PART IV'.
<!--
To simulate the OPS run d591_fpit, we make a few changes:
# NX, NY in AGCM.rc.tmpl and AGCM.BOOTSTRAP.rc.tmpl have the same values (8, 48) as in the OPS run
# Copy HISTORY.rc.tmpl over from the OPS run
# Copy obsys.rc over from the OPT run
# NX, NY in GSI_GridComp.rc.tmpl should match those from the OPS run (6, 36)
# Set NUM_SGMT to 1 in CAP.rc.tmpl
# <tt>g5das.j</tt>
## PBS -l select=32
## <tt>setenv NCPUS    384</tt>
## <tt>setenv NCPUS_GSI 216</tt>
## '''One cycle''': To run one segment each of GSIsa.x and GEOSgcm.x add '<tt>exit</tt>' before 'PART IV'
-->


====Data Assimilation experiment====
===Data Assimilation===


  $ cd $NOBACKUP/g591p4_run/run
  $ cd $FVHOME/run
  $ qsub g5das.j
  $ qsub g5das.j


<tt>g5das</tt> creates a scratch directory <tt>fvwork.xxxx</tt> under <tt>$NOBACKUP</tt> and runs the experiment there. After the initial setup, <tt>GSIsa.x</tt> is run with output written to <tt>ana.log</tt> followed by <tt>GEOSgcm.x</tt> which writes its output to <tt>stdout</tt> and <tt>fvpsas.log</tt>.
<tt>g5das</tt> creates a scratch directory <tt>$NOBACKUP/fvwork.xxxx</tt> and runs the experiment there. After the initial setup, <tt>GSIsa.x</tt> is run with output written to <tt>ana.log</tt> followed by <tt>GEOSgcm.x</tt> which writes its output to <tt>stdout</tt> and <tt>fvpsas.log</tt>.


====Forecast experiment====
===Forecast===
Later.
 
==Simulating OPS run==
<!--EVERYTHING BELOW IS COMMENTED OUT


==Simulating DAS forecast run==
==Simulating DAS forecast run==
Line 65: Line 101:
Grab the corresponding AGCM tag (Check out <tt>GEOSadas-5_9_1_p4</tt> and look for <tt>GCM</tt> in the file <tt>src/Applications/GEOSdas_App/ChangeLog</tt>).
Grab the corresponding AGCM tag (Check out <tt>GEOSadas-5_9_1_p4</tt> and look for <tt>GCM</tt> in the file <tt>src/Applications/GEOSdas_App/ChangeLog</tt>).


   $ cvs co -r GEOSadas-5_7_2_p5_m1 GEOSadas-5_7
   $ cvs co -r Ganymed-1_0_p3 Ganymed
   $ cd GEOSadas/src
   $ cd GEOSagcm/src
   $ source g5_modules
   $ source g5_modules
   $ gmake install
   $ ./parallel_build.csh


Or, one can checkout the corresponding AGCM model Fortuna-2_5_p6 (module
====Experiment setup====
Fortuna) and build it.
 
Note: For GEOSadas-5_7_2_p5_m1 in the example above, g5_modules will need to be edited in order to compile successfully on discover (due to SP1). You should change the discover section to be (where the '''bold''' items are changes, and mod5 is removed):
 
  '''set basedir = /discover/nobackup/projects/gmao/share/gmao_ops/Baselibs/v3.2.0_buildSLES11SP1'''
  set mod1 = comp/intel-11.0.083
  set mod2 = mpi/impi-3.2.2.006
  set mod3 = lib/mkl-10.0.3.020
  '''set mod4 = other/SIVO-PyD/spd_1.6.0_gcc-4.3.4-sp1'''
 
  '''set mods = ( $mod1 $mod2 $mod3 $mod4''' )
  set modinit = /usr/share/modules/init/csh
  set loadmodules = 0
 
2. Experiment setup:


To simulate a forecast run we can use gcm_setup to setup our experiment
To simulate a forecast run we can use gcm_setup to setup our experiment
Line 92: Line 113:
   $ cd Applications/GEOSgcm_App/
   $ cd Applications/GEOSgcm_App/
   $ ./gcm_setup
   $ ./gcm_setup
        GOCART: yes
        GOCART Emission Files to use: OPS
        AERO_PROVIDER: GOCART
        HISTORY: Current


Cubed-sphere was not fully implemented in this model. Choose Lat/Lon with
=====Edit files=====
resolution 1152x721x72 for the 'e'-run. The other options are
======AGCM.rc, CAP.rc, HISTORY.rc======
 
  GOCART: yes
  GOCART Emission Files to use: OPS
  AERO_PROVIDER: GOCART
  HISTORY: Current
 
3. AGCM.rc, CAP.rc, HISTORY.rc


Edit these files to match with the corresponding .tmpl files in
Edit these files to match with the corresponding .tmpl files in
Line 110: Line 128:
   $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc
   $ sed -i -e '/RESTART_TYPE:/ s/binary/pbinary/' AGCM.rc


4. gcm_run.j
======gcm_run.j======


Ensure that the correct boundary datasets are linked to. Compare the links in
Ensure that the correct boundary datasets are linked to. Compare the links in
Line 119: Line 137:
Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).
Also, be sure to add an "exit" after you run the executable (if, say, you want to run one segment).


5. RC files
=====RC files=====


Copy the .rc files over
Copy the .rc files over
Line 125: Line 143:
   $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;
   $ find ~dao_ops/e572p5_fp/run -name "*.rc" -exec cp {} /path/to/home/dir/RC \;


6. restart files
=====Restart files=====


Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config
Locate the dir containing restarts by querying FVARCH in FVDAS_Run_Config
Line 150: Line 168:
   $ echo '20120515 210000' > cap_restart
   $ echo '20120515 210000' > cap_restart


7. Run job
====Run job====
    
    
   $ qsub gcm_run.j
   $ qsub gcm_run.j
-->


[[Category:SI Team]]
[[Category:SI Team]]
[[Category:Running the Model]]
[[Category:Running the Model]]

Latest revision as of 09:38, 29 March 2013

ADAS run

Build model

First, choose a model tag to build. For this example, we choose GEOSadas-5_9_1_p7. 

$ cvs co -P -r GEOSadas-5_9_1_p7 -d g591p8 GEOSadas-5_9

Not the module name GEOSadas-5_9. Unlike AGCM, all ADAS tags have a corresponding module. 

$ cd g591p8/src
$ ./parallel_build.csh

Setup experiment (Run fvsetup)

Run fvsetup from the bin directory (g591p8/Linux/bin) with the following options (default value chosen for ones not mentioned): [we choose the default value for the initial conditions (FVICS) as well] 

AGCM Resolution? [b72]
> C180

OGCM Resolution? [c34]
> f34

EXPID? [u000_C180]
> g591p8

OBSERVING SYSTEM CLASSES?
> "ncep_prep_bufr"

Do Aerosol Analysis (y/n)? [y]
> n

One can, of course, turn on Aerosol Analysis and AOD assimilation here.

A pdf of an example fvsetup session is provided here: fvsetup_session.

FVHOME

FVHOME is the home directory for fvDAS experiment. Resource files, restarts and system output will be stored under this directory. Usually it contains subdiretories: 

ana/        first guess/analysis output
anasa/      job script to run stand-alone analysis
daotovs/    daotovs field output
diag/       diagnostic field output
etc/        listings and other odds & ends
fcst/       forecast run directory
fvInput/    process required inputs
obs/        post-analysis ODS files
prog/       prognostic field output
recycle/    latest restart files
rs/         restart files
run/        resource files

$FVHOME/.FVROOT gives the location of the installation bin directory.

Some of these directories (ana/, chem/, diag/, etc/, obs/, rs/) are stored in the archive. After completing the experiment, look in $ARCHIVE/g591p8 (where g591p8 is the experiment ID).

Run directory

$ cd $FVHOME/run

g5das.j is the job submission script. The number of cpus etc. can be changed here (if they haven't already been changed during setup). Also, if you want to run a single cycle of GSIsa.x & GEOSgcm.x add exit right before 'PART IV'.

Data Assimilation

$ cd $FVHOME/run
$ qsub g5das.j

g5das creates a scratch directory $NOBACKUP/fvwork.xxxx and runs the experiment there. After the initial setup, GSIsa.x is run with output written to ana.log followed by GEOSgcm.x which writes its output to stdout and fvpsas.log.

Forecast

Simulating OPS run

Retrieved from "https://geos5.org/wiki/index.php?title=Running_ADAS&oldid=1939"