Heracles 4.0 Quick Start: Difference between revisions

Created page with "This page describes the minimum steps required to build and run GEOS-5 Heracles 4.0 on NCCS discover and NAS pleiades. '''You should successfully complete the steps in these..."
 
 
(11 intermediate revisions by one other user not shown)
Line 28: Line 28:
where ''USERID'' is, of course, your repository username, which should be the same as your NASA and NCCS username.  Then, issue the command:
where ''USERID'' is, of course, your repository username, which should be the same as your NASA and NCCS username.  Then, issue the command:


  cvs co -r Ganymed-4_1 GEOSagcm
  cvs co -r Heracles-4_0 GEOSagcm


This should check out the latest stable version of the model from the repository and create a directory called <code>GEOSagcm</code>.
This should check out the latest stable version of the model from the repository and create a directory called <code>GEOSagcm</code>.
Line 36: Line 36:
First, you need to set <code>ESMADIR</code>. For example, if your <code>src/</code> directory is:
First, you need to set <code>ESMADIR</code>. For example, if your <code>src/</code> directory is:


  /discover/nobackup/mathomp4/Models/Ganymed-4_1/GEOSagcm/src
  /discover/nobackup/mathomp4/Models/Heracles-4_0/GEOSagcm/src


then you should set:
then you should set:


  setenv ESMADIR /discover/nobackup/mathomp4/Models/Ganymed-4_1/GEOSagcm
  setenv ESMADIR /discover/nobackup/mathomp4/Models/Heracles-4_0/GEOSagcm


Next, <code>cd</code> into <code>GEOSagcm/src</code> and <code>source</code> the file called <code>g5_modules</code>:
Next, <code>cd</code> into <code>GEOSagcm/src</code> and <code>source</code> the file called <code>g5_modules</code>:
Line 53: Line 53:


  Currently Loaded Modulefiles:
  Currently Loaded Modulefiles:
  1) comp/intel-13.1.2.183                      4) other/comp/gcc-4.6.3-sp1
  1) comp/intel-15.0.2.164
  2) other/mpi/mvapich2-1.8.1/intel-13.1.2.183  5) other/SIVO-PyD/spd_1.10.0_gcc-4.6.3-sp1
  2) mpi/impi-5.0.3.048
  3) lib/mkl-13.0.1.117
  3) lib/mkl-15.0.2.164
  4) other/comp/gcc-4.6.3-sp1
  5) other/SIVO-PyD/spd_1.20.0_gcc-4.6.3-sp1_mkl-15.0.0.090




Line 90: Line 92:
  Enter a 1-line Experiment Description:
  Enter a 1-line Experiment Description:


Spaces are ok here.   It will also ask you for the atmospheric model resolution, expecting the code for one of the displayed resolutions.  
Spaces are ok here. It will then ask for the source code version tag to associate with the model -- you should hit enter for the default:
Enter the Atmospheric Horizontal Resolution code:
 
Enter an Experiment Source Tag for History (Default: Heracles-4_0):
 
Hit enter for the default to the next question:
 
Do you wish to CLONE an old experiment? (Default: NO or FALSE)
 
It will also ask you for the atmospheric model resolution, expecting the code for one of the displayed resolutions.  
 
Enter the Atmospheric Horizontal Resolution code:
  -----------------------------------------------------------
  -----------------------------------------------------------
       Lat/Lon                    Cubed-Sphere
       Lat/Lon                    Cubed-Sphere
Line 136: Line 147:
  Hit ENTER to use Default Location:
  Hit ENTER to use Default Location:
  ----------------------------------
  ----------------------------------
  Default:  /discover/nobackup/''USER''/GEOSgcm
  Default:  /discover/nobackup/''USER''/GEOSagcm
   
   
After these it will ask you for a group ID -- the default for this writer is g0620 (GMAO modeling group).  Enter whatever is appropriate, as necessary.   
After these it will ask you for a group ID -- the default for this writer is g0620 (GMAO modeling group).  Enter whatever is appropriate, as necessary.   
Line 145: Line 156:




The script produces an experiment directory (''EXPDIR'') in your space as <code>/discover/nobackup/''USERID''/''EXPID''</code>, which contains the files and sub-directories:
The script will produce some messages and create an experiment directory (''EXPDIR'') in your space as <code>/discover/nobackup/''USERID''/''EXPID''</code>, which contains the files and sub-directories:


*<code>AGCM.rc</code> -- resource file with specifications of boundary conditions, initial conditions, parameters, etc.
*<code>AGCM.rc</code> -- resource file with specifications of boundary conditions, initial conditions, parameters, etc.
*<code>archive/</code> -- contains job script for archiving output
*<code>archive/</code> -- contains job script for archiving output
*<code>CAP.rc</code> -- resource file with run job parameters
*<code>CAP.rc</code> -- resource file with run job parameters
*<code>convert</code> -- contains job script that converts restarts (initial condition files) from older model versions
*<code>ExtData.rc</code> -- sample resource file for external data, not used
*<code>ExtData.rc</code> -- sample resource file for external data, not used
*<code>forecasts/</code> -- contains scripts used for data assimilation mode
*<code>forecasts/</code> -- contains scripts used for data assimilation mode
*<code>fvcore_layout.rc</code> --
*<code>gcm_run.j</code> -- run script
*<code>gcm_run.j</code> -- run script
*<code>GEOSgcm.x</code> -- model executable
*<code>GEOSgcm.x</code> -- model executable
Line 159: Line 172:
*<code>RC/</code> -- contains resource files for various components of the model
*<code>RC/</code> -- contains resource files for various components of the model
*<code>regress/</code> -- contains scripts for doing regression testing of model
*<code>regress/</code> -- contains scripts for doing regression testing of model
*<code>src</code> -- directory with a tarball of the model version's source code




Line 175: Line 189:
The <code>umask 022</code> is not strictly necessary, but it will make the various files readable to others, which will facilitate data sharing and user support.  Your home directory <code>~''USERID''</code> is also inaccessible to others by default; running <code>chmod 755 ~</code> is helpful.
The <code>umask 022</code> is not strictly necessary, but it will make the various files readable to others, which will facilitate data sharing and user support.  Your home directory <code>~''USERID''</code> is also inaccessible to others by default; running <code>chmod 755 ~</code> is helpful.


Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''.  You can get an arbitrary set of restarts by copying the contents of the directory <code>/archive/u/aeichman/restarts/Ganymed-4_1/c48/19800103/</code>, containing 2-degree cubed sphere restarts from Jan. 3, 1980, and their corresponding <code>cap_restart</code>.  If you are unfamiliar with the way that the <code>/archive</code> filesystem works, keep in mind that a <code>cp</code> from there might appear to stall while the tape is loaded -- see the NCCS documentation for details.   
Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''.  You can get an arbitrary set of restarts by copying the contents of the directory <code>/archive/u/aeichman/restarts/Heracles-4_0/c48/19800309</code>, containing 2-degree cubed sphere restarts from March 9, 1980, and their corresponding <code>cap_restart</code>.  If you are unfamiliar with the way that the <code>/archive</code> filesystem works, keep in mind that a <code>cp</code> from there might appear to stall while the tape is loaded -- see the NCCS documentation for details.   
 
The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''.  It should be ready to go as is.  The parameter END_DATE in <code>CAP.rc</code> can be set to the date you want the run to stop.  An alternative way to stop the run is by commenting out the line <code> if ( $capdate < $enddate ) qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted, or rename the script file, or kill the job (described below).


Submit the job with <code>qsub gcm_run.j</code>.  You can keep track of it with <code>qstat</code> or <code>qstat | grep ''USERID''</code>, or follow stdout with <code>tail -f ''EXPDIR''/slurm-''JOBID''.out</code>, ''JOBID'' being returned by <code>qsub</code> and displayed with <code>qstat</code>.  Jobs can be killed with <code>qdel ''JOBID''</code>.
The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''.  It should be ready to go as is.  The parameter END_DATE in <code>CAP.rc</code> can be set to the date you want the run to stop.  Submit the job with <code>sbatch gcm_run.j</code>.  You can keep track of it with <code>qstat</code> or <code>qstat | grep ''USERID''</code>, or follow stdout with <code>tail -f ''EXPDIR''/slurm-''JOBID''.out</code>, ''JOBID'' being returned by <code>sbatch</code> and displayed with <code>qstat</code>.  Jobs can be killed with <code>qdel ''JOBID''</code>.


== Output and Plots ==
== Output and Plots ==
Line 201: Line 213:
'''Back to [[GEOS-5 Documentation for Heracles 4.0]]'''
'''Back to [[GEOS-5 Documentation for Heracles 4.0]]'''


Contact Andrew Eichmann at GMAO with questions and comments
Contact Matt Thompson at GMAO with questions and comments