Regridding Restarts for Fortuna 2.5

Here are basic instructions on how to regrid restart (initial condition) files, based on a README by Larry Takacs.

Back to GEOS-5 Documentation for Fortuna 2.5

The Easiest Way: Using Restarts from Earlier Versions of Fortuna

Restart files from Fortuna 2.4 may be used for Fortuna 2.5 with the exception of solar_internal_rst, which must be bootstrapped. Currently, this can be accomplished simply by deleting that restart file and running the model for a day -- the full set of restarts will be created before the model returns. Restarts from Fortuna 2.3 or Fortuna 2.2 can be used by bootstrapping solar_internal_rst and irrad_internal_rst. The above assumes that the restarts have the same resolution, otherwise regridding is necessary. Restarts from versions earlier than Fortuna 2.2 must be regridded to use with Fortuna 2.5.

The Next Easiest Way: Using the regrid Script

Restarts can be easily created from MERRA data using the regrid script in src/GMAO_Shared/GEOS_Util/post. The same script can regrid restarts generated by different versions of Fortuna to the Fortuna 2.5 at any resolution.

Running regrid (after running source g5_modules) without arguments tells the story:

GEOS_Util/post> regrid 
 
Usage -- To regrid MERRA restarts:
----------------------------------
regrid   -nymd  YYYYMMDD  
         -im    IM        
         -jm    JM        
         -out   DIRECTORY 
 
     where:  YYYYMMDD  is the desired Date from the MERRA Re-Analysis
                   IM  is the desired Zonal      Dimension for ReGridding
                   JM  is the desired Meridional Dimension for ReGridding
             DIRECTORY is the full pathname of the desired OUTPUT location
 
   (  Note: A regrid.rc file will be automatically generated )
 
 
Usage -- To regrid Non-MERRA restarts:
--------------------------------------
regrid   -rc    REGRIDRC  
         -out   DIRECTORY 
 
     where:     REGRIDRC  is the full pathname of a user-defined regrid.rc file
                DIRECTORY is the full pathname of the desired OUTPUT location
 
GEOS_Util/post>

So the following will create restarts in 0.25 degree resolution from MERRA on the first day of 1990 and put them in /discover/nobackup/aeichman/regrid

regrid -nymd 19900101 -im 1152 -jm 721 -out /discover/nobackup/aeichman/regrid

For sources other than MERRA, you will have to specify the location of a regrid resource file that contains the model version, resolution and location of the source restarts, and the resolution of the output restarts. A sample regrid resource file is regrid.rc in the same directory as the regrid script. Note that regrid only works for creating restarts for its model tag, and does not work at all for making restarts for tags before Fortuna 2.5. Restarts for earlier model tags have to be created "manually".

Renaming Restart Files

At this point you will have restart files with a fair bit of cruft in the filenames. As described above, they will have to be renamed for the model to recognize them. There are a number of scripts with different techniques for completing this tedious chore; here is one example:

#!/usr/bin/env python

import re
import os


# Open file for reading
dirlist = os.listdir(".")

for file in dirlist:

   if( re.search(".*rst.*",file) ):
      print file
# the concept is to match ??????.something_rst.??????? OR something_rst.????????
# and extract "something_rst"
      newfile = re.sub(r'(^.+\.|^)([a-z_]+rst)\..+$',r'\2',file)
      print newfile
      if cmp(newfile,file):
         os.symlink(file,newfile)

It will grab anything in the current working directory resembling the common forms of restart filename generated by the regridding scripts and other utilities and create symbolic links to them in the same directory with the superfluous parts of the filenames stripped. In an experiment directory the links will be overwritten by the model with "fresh" restart files at the end of a run segment. Alternatively, you can easily copy the restart files with cp *rst

Required Restarts

A few of the restart files are critical and need to be present for the model to function, while others can be "bootstrapped": the model is started without them and then it generates the restarts in order to save state for the next restart.

The following restart files are critical and need to be regridded:

• fvcore_internal_rst (atmosphere)
• moist_internal_rst (atmosphere)
• catch_internal_rst (surface)

The following surface restarts should be regridded, but can be bootstrapped:

• vegdyn_internal_rst
• landice_internal_rst
• lake_internal_rst
• saltwater_internal_rst

Optional (boot-strappable) restarts:

• pchem_internal_rst
• gocart_internal_rst
• turb_internal_rst
• saltwater_import_rst
• surf_import_rst
• turb_import_rst
• moist_import_rst

For DAS runs, you may also interpolate:

• agcm_import_rst

The restarts irrad_internal_rst and solar_internal_rst should not be regridded from earlier versions to Fortuna 2.5, as it must be bootstrapped.

Note: if the grid dimensions are the same SOME restarts from earlier model versions may be used , while other may not.

• fvcore_internal_rst and moist_internal_rst may be used directly from earlier versions.
• catch_internal_rst, lake_internal_rst, landice_internal_rst, and saltwater_internal_rst must be re-gridded due to new land surface tile data
• All other restarts from earlier versions should be boot-strapped.

Where to Find MERRA Restarts

If you lack a set of restarts for the appropriate date for an older version of Fortuna, you can derive yours from MERRA. MERRA restarts are available on the NCCS discover cluster under the following directories:

/archive/g_proj5/production/GEOSdas-2_1_4/d5_merra_jan79/rs/
/archive/g_proj5/production/GEOSdas-2_1_4/d5_merra_jan89/rs/
/archive/g_proj5/production/GEOSdas-2_1_4/d5_merra_jan98/rs/

Restarts are in subdirectories according to their respective year and month. The are in the form (as an example) d5_merra_jan79.catch_internal_rst.19910128_21z.bin, which you would otherwise know as catch_internal_rst (with the appropriate cap_restart). Be aware that a cp from /archive may appear to hang initially -- sometimes for hours -- while the appropriate tape is queued for mounting. Alternatively, run dmget on the files before copying them: this will retrieve the files and have them ready for copying once the command returns.

Regridding Restarts Manually

If you need restarts for a model tag before Fortuna 2.5, you will have to regrid them manually. This section shows you how.

Regridding Upper-Air Related Restarts

Within your build, go to the directory:

 src/GMAO_Shared/GEOS_Util/post

Set up the build environment by running source g5_modules and setting the evironment variable ESMADIR to the directory path above the src directory. Then run make rs_hinterp.x to produce the executable:

rs_hinterp.x

Running rs_hinterp.x shows the usage:

> ./rs_hinterp.x
Usage:

rs_hinterp_$ARCH.x -dyn      DYNRST
                   -moist    MOISTRST
                   -other    OtherRST1 OtherRST2 OtherRST3 ...
                   -topo_old TOPO_OLD
                   -topo_new TOPO_NEW
                   -im       IM_OUT
                   -jm       JM_OUT

where:

  -dyn         DYNRST:  Filename for DYNAMICS_INTERNAL_RESTART
  -moist     MOISTRST:  Filename for    MOIST_INTERNAL_RESTART
  -other     OtherRST:  Filename for Other Flat Binary_RESTART
  -topo_old  TOPO_OLD:  Filename for OLD Topography (associated with INPUT  resolution)
  -topo_new  TOPO_NEW:  Filename for NEW Topography (associated with OUTPUT resolution)
  -im          IM_OUT:  IM Dimension for Output
  -jm          JM_OUT:  JM Dimension for Output

So, as an example, to regrid from an existing 2x2.5-deg set of restarts to 1x1.25-deg, you have:

> rs_hinterp.x -dyn fvcore_internal_rst -moist moist_internal_rst -other pchem_internal_rst  gocart_internal_rst -topo_old TOPO_OLD 
-topo_new TOPO_NEW -im 288 -jm 181

where TOPO_OLD is linked to:

/discover/nobackup/ltakacs/bcs/Fortuna-2_1/144x91/topo_DYN_ave_144x91_DC.data

and TOPO_NEW is linked to:

/discover/nobackup/ltakacs/bcs/Fortuna-2_1/288x181/topo_DYN_ave_288x181_DC.data

Topography files for different Fortuna versions can be found under /discover/nobackup/ltakacs/bcs/. For regridding from MERRA use Fortuna 1.4 540x361 topography files.

Running the above will produce:

fvcore_internal_rst.0288x0181
moist_internal_rst.0288x0181
pchem_internal_rst.0288x0181
gocart_internal_rst.0288x0181

These are the finished restarts. They will have to be renamed -- truncated after rst -- in order to be recognized by the model in its standard configuration.

Note that even if you have no restarts that fall under the -other option, you still have to include that option (with nothing after it) in the command.

Regridding Land-Surface Related Restarts

Within your build, go to:

src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/ ...
GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts

At this point it is best to update your directory to the HEAD of CVS to be sure you have the latest datasets:

cvs upd -A

Under the mk_restarts directory you have the directories:

InData
OutData

The basic idea is:

1. Put your existing land-related restarts (catch_internal_rst, vegdyn_internal_rst, landice_internal_rst, lake_internal_rst, and saltwater_internal_rst) in the InData directory
2. Put the appropriate tile file for your current resolution in the InData directory
3. Put your target resolution tile file and appropriate clsm directory in the OutData directory

Consider soft linking files instead of copying, as they can be large. Where to find the appropriate tile file depends on the Fortuna versions you are regridding from and to, and is detailed below.

Run mk_Restarts. This will create the executables (if needed; remember to source g5_modules) and run them. If the executables are run without the proper setup, they will crash and create a zero-length file in OutData with the name *.til (with the asterisk in the filename); you will need to delete it before running again.

Your regridded restarts will be produced in the OutData directory.

Old and New Tile Files

There are two possible scenarios for regridding land-surface related restarts:

1. Converting existing restarts to resolutions: 144x91, 288x181, 540x361, 1080x721 (old tile files for Fortuna 1.4 and earlier)
2. Converting existing restarts to resolutions: 144x91, 288x181, 576x361, 1152x721 (new tile files for Fortuna 1.5 and later)

The old tile files are currently archived in:

/discover/nobackup/ltakacs/bcs/Fortuna-1_4/144x91/FV_144x91_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-1_4/288x181/FV_288x181_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-1_4/540x361/FV_540x361_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-1_4/1080x721/FV_1080x721_DC_360x180_DE.til

The new tile files and clsm directories are located in:

/discover/nobackup/ltakacs/bcs/Fortuna-2_1/144x91/FV_144x91_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-2_1/288x181/FV_288x181_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-2_1/576x361/FV_576x361_DC_360x180_DE.til
/discover/nobackup/ltakacs/bcs/Fortuna-2_1/1152x721/FV_1152x721_DC_360x180_DE.til

(These work for Fortuna 2.4 as well.)

Depending on which tile file you are regridding to, there are two procedures:

Target tile file is from OLD set (model tags Fortuna-1_4 and earlier)

Since your target tile file is OLD (and not consistent with the new mosaic_veg_typs_fracs data), you have to remove the mosaic_veg_typs_fracs file from the InData direcory (or simply rename it, eg: mosaic_veg_typs_fracs.hold). In its place you have to copy the OLD nirdf.dat, visdf.dat, and lai_grn_clim boundary condition datasets into the InData directory. These can be found in the corresponding /discover/nobackup/ltakacs/bcs/Fortuna-1_4 directories. Note, the names that the Fortran program is looking for are not the same names in our bcs directory. Therefore you need to link the old datasets to the proper names. For example, regridding FROM the old 540x361 resolution (eg. MERRA), you should have:

nirdf.dat -> /discover/nobackup/ltakacs/bcs/Fortuna-1_4/540x361/nirdf_540x361_DC.dat

visdf.dat -> /discover/nobackup/ltakacs/bcs/Fortuna-1_4/540x361/visdf_540x361_DC.dat

lai_grn_clim -> /discover/nobackup/ltakacs/bcs/Fortuna-1_4/540x361/lai_green_clim_540x361_DC.data

Target tile file is from NEW set (model tags Fortuna-1_5 and later)

In this case, the target tile files are consistent with the mosaic_veg_typs_fracs dataset which resides in the InData directory. Therefore you do not have link in the old nirdf.dat, visdf.dat, and lai_grn_clim datasets. For Fortuna 2.2 you do have to copy or link the clsm directory in directory with the appropriately-dimensioned tile files.

Bootstrapping for a Complete Set of Restarts

The above procedure produces all the necessary restart files for the model to run, but others are needed for proper function. The model can produce them starting from the minimal set created by the regridding procedure; this requires some modification of the AGCM.rc file in your experiment home directory.

In AGCM.rc, using turb_import_rst as an example, add a "-" to the beginning of the filename of the restarts to be bootstrapped:

TURBULENCE_IMPORT_RESTART_FILE:         turb_import_rst
TURBULENCE_IMPORT_RESTART_TYPE:         binary 
TURBULENCE_IMPORT_CHECKPOINT_FILE:      turb_import_checkpoint
TURBULENCE_IMPORT_CHECKPOINT_TYPE:      binary

TURBULENCE_IMPORT_RESTART_FILE:         -turb_import_rst
TURBULENCE_IMPORT_RESTART_TYPE:         binary
TURBULENCE_IMPORT_CHECKPOINT_FILE:      turb_import_checkpoint
TURBULENCE_IMPORT_CHECKPOINT_TYPE:      binary

This will have the effect that when the model starts, it will look for that restart file and use it if it finds it, and will otherwise bootstrap the restart file. Running the model for a model day using this modified AGCM.rc will produce a full set of restart files, which can be used with the default model configuration.

Back to GEOS-5 Documentation for Fortuna 2.5