Running Coupled Ocean

Steps to run AOGCM

Checkout

As of this writing the stable tag is 'yuri-coupled-Fortuna-2_5_obio' and module is 'Fortuna'. Check with Yury Vikhliaev for the latest stable tag and module.

$ cvs co -r yuri-coupled-Fortuna-2_5_obio Fortuna

Compile

$ cd GEOSagcm/src
$ cvs up -r Ganymed-2_0_UNSTABLE g5_modules
$ source g5_modules

Compile time specification of resolution/layout

The seaice component has the resolution and layout specified in the GNUmakefile in:

$ESMADIR/src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/
   GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSCICEThermo_GridComp/cice/GNUmakefile

Inside this are lines:

NXGLOB = 360
NYGLOB = 200
BLCKX = 45
BLCKY = 20   
MXBLCKS = 1

where (NXGLOB, NYGLOB) define the ocean resolution. It is the user's responsibility to ensure that BLCKX (BLCKY) is a divisor of NXGLOB (NYGLOB) and the quotient defines the number of processors OGCM_NX (OGCM_NY) in the X (Y) direction, i.e.

OGCM_NX = NXGLOB/BLCKX (= 8  in our example)
OGCM_NY = NYGLOB/BLCKY (= 10 in our example)

[I had tried with NXGLOB = 360, BLCKX = 40 => OGCM_NX = 9. It didn't work. Apparently, odd numbered processes don't. Update: I could get it to work on 1x1 layout.]

Since the resolution/layout are specified at compile time (really!), the model needs to be re-compiled for a new resolution/layout combination.

Finally, compile:

$ make install

Setup experiment

gcm_setup

$ cd Applications/GEOSgcm_App

Set correct values of OGCM_NX and OGCM_NY in gcm_setup (if they are not already set):

set OGCM_NX = 8
set OGCM_NY = 10   

Then run gcm_setup:

$ ./gcm_setup
       Experiment ID:                                     Choose
       Experiement Description:                           Choose
       Atmospheric Horizontal Resolution:                 144 91
       Atmospheric Vertical Resolution:                   Default
       COUPLED Ocean/Sea-Ice Model:                       YES
       Ocean Lat/Lon Horizontal Resolution:               Default
       Ocean Model Vertical Resolution:                   Default
       GOCART:                                            Default
       HISTORY template:                                  HISTORY_COUPLED.rc.tmpl
       HOME directory:                                    Choose
       EXPERIMENT directory:                              Same as HOME
       BUILD directory:                                   Default
       GROUP ID:                                          Default

Edit some files

$ cd /path/to/exp/dir

CAP.rc

To run a 1 day experiment etc. Also timers and memutils can be enabled here.

AGCM.rc

Need to ensure:

OGCM_NX = NX = 8
OGCM_NY = NY = 10
CICE_NPROCS = 80

input.nml

layout = 8,10,

gcm_run.j

#PBS -l select=7:ncpus=12:mpiprocs=12

(need 80 cpus, hence 7 nodes) Also, if running a 1 day experiment, comment out qsub lines.

HISTORY.rc

Comment out the line

'RUNOFF'    , 'SURFACE'       ,

in geosgcm_surf.fields (model crashes with this option).

Copy restarts

Example restarts can be found in ~yvikhlia/geos5/test180/expdir. Several new restarts are needed but these can be bootstrapped (according to Yury). The new ones are orad_import, seaice_import, and seaice_internal. [The required restarts are fvcore_internal_rst, lake_internal_rst, landice_internal_rst, catch_internal_rst, moist_internal_rst, saltwater_internal_rst, seaice_internal_rst, seaice_import_rst, orad_import_rst. If using GOCART, then gocart_internal_rst, carma_internal_rst, stratchem_internal_rst, gmichem_internal_rst as well.]

$ cp ~yvikhlia/geos5/test180/expdir/*_rst .
$ cp ~yvikhlia/geos5/test180/expdir/cap_restart .

cap_restart may need to edited as well to specify a particular year (no changes in month, day or time).

When the experiment directory is created (by gcm_setup) a new directory is now there called RESTART. Inside are restart files, the only one of which is required is ocean_temp_salt.res.nc. The rest can be bootstrapped.

$ cp ~yvikhlia/geos5/test180/expdir/RESTART/* RESTART/

Boundary conditions

We need ocean grid_spec, ice grid, KPAR data and river routing file during a run. They are all in directory pointed to by $GRIDDIR in the runscript (gcm_run.j).

Run job

$ qsub gcm_run.j