View source for Setting up S2Sv3 model

= Coupled Model Build Instructions =


This document describes the basic step to obtain, compile and execute the GEOS couple Ocean model.

=== Obtain the Code ===

The code is in a CVS repository. To obtain it, issue the command:

    cvs co -P -r tag_name GEOSodas_m7
    
where <code>tag_name</code> is the name of the CVS tag of interest. Current S2Sv3 tag is <code>yuri-s2s3-unstable-FullModel</code>.

You will get a directory named <code>GEOSodas</code>.

=== Compile the Code ===

To compile the code, issue the following commands:

     cd GEOSodas/src
     source g5_modules
     ./parallel_build.csh
     
In the last command, you might be asked to provide a <code>sponsor code id</code> is if you have more than one on the platform you are using.



A directory named <code>Linux/</code> will be created inside <code>GEOSodas/</code>. The <code>Linux/</code> directory contains temporary files, libraries and executables created during the compilation process. To make sure that the code was successfully compiled, check that the file <code>GEOSgcm.x</code> exists in the directory <code>GEOSodas/src/Applications/GEOSgcm_App/</code>.

'''Note:''' To compile the model on Rome nodes at NAS, use <code>g5_modules_4ROME</code> modules file:

    cp g5_modules_4ROME g5_modules
    ./parallel_buid.csh

'''Note:''' To compile the model on Milan nodes at NCCS, use <code>yuri-s2s3-unstable-milan</code> tag and run the <code>parallel_build.csh</code> from Milan nodes:
    
    ssh discover-mil
    ./parallel_build.csh

or request an interactive SLURM session with <code>--constraint=mil</code> option and follow steps described above. 

=== Create the Experiment Directory ===

After building the executable, we need to create an experiment directory. The code comes with a tool that allows you to do that. We recommend that you always go through this process whenever to start a new experiment.

The first step is to go to the directory <code>GEOSodas/src/Applications/GEOSgcm_App/</code>. Then check that the following files are available:

   AGCM.rc.tmpl GAAS_GridComp.rc.tmpl GEOSgcm.x gcm_run.j gcm_setup ocean_das_config.tmpl

There are many other files in the directory but the above ones play an important role in setting the experiment directory and driving the model.

The script <code>gcm_setup</code>, when executed, asks a series of questions to determine a set of parameters and input files needed to run the model. 

Issue the command:

      ./gcm_setup
      
and provide the information requested. When you are asked if:

* You want to run the COUPLED Ocean/Sea-Ice Model, answer <code>YES</code>.
* You want to to run the REPLAY DUAL OCEAN Model,
** Answer <code>YES</code> if you want to have the option of running <code>ODAS</code>.


The available resolutions are:

 | Atmosphere | Ocean                           |
 | ---------- | ------------------------------- |
 | c90x72     | 360x200x50                      |
 | c180x72    | 720x410x40                      | 
 | c180x72    | 1440x1080x50 (S2Sv3 resolution) |

When you are asked which history template do you want to use, you may choose <code>HISTORY.AOGCM.rc.tmpl</code> or <code>HISTORY.S2S.rc.tmpl</code>. All other parameters may be left default. If the `ODAS` configuration is selected, the entire <code>GEOSodas/src/Applications/UMD_Etc/</code> directory will be copied into the <code>ocean_das</code> directory located in the experiment directory.

When you are asked which history template do you want to use, you may choose <code>HISTORY.AOGCM.rc.tmpl</code> or <code>HISTORY.S2S.rc.tmpl</code>. All other parameters may be left default. If the <code>ODAS</code> configuration is selected, the entire <code>GEOSodas/src/Applications/UMD_Etc/</code> directory will be copied into the <code>ocean_das</code> directory located in the experiment directory.

Here, we provide a sample dialogue:
  
     Enter the Experiment ID:
          test_odas
     Enter a 1-line Experiment Description:
          Test ODAS
     Do you wish to CLONE an old experiment? (Default: NO or FALSE)
          NO
     Enter the Atmospheric Horizontal Resolution code:
     -----------------------------------------------------------
     Lat/Lon                     Cubed-Sphere
     -----------------------------------------------------------
        b --  2  deg                c48  --  2   deg 
        c --  1  deg                c90  --  1   deg 
        d -- 1/2 deg                c180 -- 1/2  deg (56-km) 
        e -- 1/4 deg (35-km)        c360 -- 1/4  deg (28-km)  
                                    c720 -- 1/8  deg (14-km) 
                                    c1440 - 1/16 deg ( 7-km) 
                                              DYAMOND Grids                 
                                    c768 -- 1/8  deg (12-km) 
                                    c1536 - 1/16 deg ( 6-km) 
                                    c3072 - 1/32 deg ( 3-km) 
          
          c180
     Enter the Atmospheric Model Vertical Resolution: LM (Default: 72)
          72
     Do you wish to run the COUPLED Ocean/Sea-Ice Model? (Default: NO or FALSE)
          Yes
      
     Enter the Ocean Lat/Lon Horizontal Resolution: IM JM (Default: 360 200)
          1440 1080
     Enter the Ocean Model Vertical Resolution: LM (Default: 50)
          50
     Do you wish to run the REPLAY DUAL OCEAN Model? (Default: NO or FALSE)
          YES
     
     Do you wish to run the AODAS Model? (Default: NO or FALSE)
          YES

     Enter the choice of  Restart Record Parameters: Frequency (hhmmss): (Default: 240000)
          240000
     Enter the choice of  Restart Record Parameters: Reference Date (YYYYMMDD): (Default: 19800101)
          19800101
     Enter the choice of  Restart Record Parameters: Reference Time (hhmmss): (Default: 210000)
          210000
     Enter the choice of  Land Surface Model: use 1 (Default: Catchment Model), 2 (CatchmentCN Model)
          1
     
     Do you wish to run  the Runoff Routing Model? (Default: NO or FALSE)
          NO
    
     Do you wish to run GOCART with Actual or Climatological Aerosols? (Enter: A (Default) or C)
          A
     
     Enter the GOCART Emission Files to use: MERRA2 (Default), PIESA, CMIP, NR, MERRA2-DD or OPS:
          MERRA2
      
     Enter the tag or directory (/filename) of the HISTORY.AGCM.rc.tmpl to use
     (To use HISTORY.AGCM.rc.tmpl from current build, Type:  Current         )
     -------------------------------------------------------------------------
     Hit ENTER to use Default Tag/Location: (/discover/nobackup/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/GEOSodas/src/Applications/GEOSgcm_App)
       
     Enter Desired Location for the HOME Directory (to contain scripts and RC files)
     Hit ENTER to use Default Location:
     ----------------------------------
     Default:  /gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas
     
     Enter Desired Location for the EXPERIMENT Directory (to contain model output and restart files)
     Hit ENTER to use Default Location:
     ----------------------------------
     Default:  /gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas
     
     Enter Location for Build directory containing:  src/ Linux/ etc...
     Hit ENTER to use Default Location:
     ----------------------------------
     Default:  /discover/nobackup/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/GEOSodas
    
     Current GROUPS: k3002 a930b s0942 s1043 m2val j1008
     Enter your GROUP ID for Current EXP: (Default: s1043)
     -----------------------------------
     
     sending incremental file list
     GEOSgcm.x
     
     sent 140,652,369 bytes  received 35 bytes  93,768,269.33 bytes/sec
     total size is 140,617,928  speedup is 1.00
      
     Creating gcm_run.j for Experiment: test_odas 
     Creating gcm_archive.j for Experiment: test_odas 
     Creating gcm_regress.j for Experiment: test_odas 
     Creating gcm_convert.j for Experiment: test_odas 
     Creating gcm_plot.tmpl for Experiment: test_odas 
     Creating gcm_quickplot.csh for Experiment: test_odas 
     Creating gcm_moveplot.j for Experiment: test_odas 
     Creating gcm_stats.j for Experiment: test_odas 
     Creating gcm_forecast.tmpl for Experiment: test_odas 
     Creating gcm_forecast.setup for Experiment: test_odas 
     Creating gcm_emip.setup for Experiment: test_odas 
     Creating CAP.rc.tmpl for Experiment: test_odas 
     Creating AGCM.rc.tmpl for Experiment: test_odas 
     Creating GAAS_GridComp.rc.tmpl for Experiment: test_odas 
     Creating ocean_das_config.tmpl for Experiment: test_odas 
     Creating HISTORY.rc.tmpl for Experiment: test_odas 
     Creating input.nml for Experiment: test_odas 
     Creating diag_table for Experiment: test_odas 
     Creating field_table for Experiment: test_odas 
     Creating plotocn.j for Experiment: test_odas 
     Creating __init__.py for Experiment: test_odas 
     Creating fvcore_layout.rc for Experiment: test_odas 
      
     /bin/mv: '/gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas/GAAS_GridComp.rc.tmpl' and '/gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas/GAAS_GridComp.rc.tmpl' are the same file
     Done!
     -----
      
     Build Directory: /discover/nobackup/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/GEOSodas
     ----------------
      
      
     The following executable has been placed in your Experiment Directory:
     ----------------------------------------------------------------------
     /discover/nobackup/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/GEOSodas/Linux/bin/GEOSgcm.x
     
      
     You must now copy your Initial Conditions into: 
     ----------------------------------------------- 
     /gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas
     
     Copying Sandbox Source Code (including non-committed files) into /gpfsm/dnb32/jkouatch/GEOS_OCEAN/yuri-s2s3-unstable-SLES12/test_odas/src/GEOSagcm/src ...
      
     Enter CVS Tag to COMMIT this Experiment (Default: test_odas__jkouatch)
     Enter q or quit to QUIT or SKIP the CVS COMMIT:
          q

=== Running the Code ===

After running the <code>gcm_setup</code> script, an experiment directory will be created. 
Depending on your settings, you will have (among others) the following files (in the experiment directory) that are critical in running the <code>ODAS</code> configuration:

    AGCM.rc.with_RECORD_OFF 
    AGCM.rc.with_RECORD_ON
    GAAS_GridComp.rc.tmpl
    ocean_das_config

You do not need to edit any of the above files unless you change the processor decomposition. In that case, the files <code>AGCM.rc.with_RECORD_OFF</code> and <code>AGCM.rc.with_RECORD_ON</code> need to be updated.

To run the code, you will need to create the <code>cap_restart</code> file. You may also need to edit the following files:
* <code>CAP.rc</code>: to set the end date.
* <code>HISTORY.rc</code>: to specify the collections you are interested in.
* <code>gcm_run.j</code>: to change the requested wallclock time (<code>-l walltime</code>) and the number of tasks (<code>--ntasks</code>).
* <code>input.nml</code>: to modify the <code>layout</code> settings in the <code>ocean_model_nml</code> section. This is tied to the settings in <code>AGCM.rc.with_RECORD_OFF</code> and <code>AGCM.rc.with_RECORD_ON</code>.

---------

---------


To check out the model:
 cvs co -r tag_name GEOSodas_m7,
where 
 tag_name = yuri-s2s3-unstable for SLES11
 tag_name = yuri-s2s3-unstable-SLES12 for SLES12
Note: You can use either tag on pfe. They are identical besides different modules.
 
To build the model:
 cd GEOSodas/src
 source g5_modules
 setenv ESMADIR path_to_sandbox (GEOSodas)
 make install

To build with debug options:
 make BOPT=g install

== To set up experiment ==
 cd Applications/GEOSgcm_App
 ./gcm_setup

When you are asked if you want to run a coupled model, answer 'yes'. Available resolutions are:
 c90x72 atmosphere, 360x200x50 ocean;
 c180x72 atmosphere, 720x410x40 ocean;
 c180x72 atmosphere, 1440x1080x50 ocean. This is S2Sv3 resolution.
When you are asked which history template do you want to use, you may choose <code>HISTORY.AOGCM.rc.tmpl</code> or <code>HISTORY.S2S.rc.tmpl</code>. All other parameters may be left default.