GEOS-5 Quick Start: Difference between revisions

This page describes the minimum steps required to build and run GEOS-5 Fortuna 2.0 on discover.  You should successfully complete the steps in these instructions before doing anything more complicated.  '''FORTUNA 2.0 IS NOW DEPRECATED AND MINIMALLY SUPPORTED, AS IS THIS DOCUMENTATION.  PLEASE SEE THE DOCUMENTATION FOR THE LATEST VERSION'''.

 

The following assumes that you know your way around Unix, have successfully logged into your NCCS account (presumably on the '''discover''' cluster) and have an account on the source code repository with the proper <code>ssh</code> configuration -- see the progress repository quick  start: https://progress.nccs.nasa.gov/trac/admin/wiki/CVSACL. 

The commands below assume that your shell is <code>csh</code>.  Since the scripts to build and run GEOS-5  tend to be written in the same, you shouldn't bother trying to import too much into an alternative shell. If you prefer a different shell, it is easiest just to open a <code>csh</code> process to build the model and your experiment.

Furthermore, model builds should be created in your space under <code>/discover/nobackup</code>, as creating them under your home directory will quickly wipe out your disk quota.

setenv CVSROOT :ext:''USERID''@progressdirect.nccs.nasa.gov:/cvsroot/esma

where ''USERID'' is, of course, your repository username, which should be the same as your NASA and NCCS username. Then, issue the command:

This should check out the latest stable version of the model from the repository and create a directory called <code>GEOSagcm</code>.  <code>cd</code> into <code>GEOSagcm/src</code> and <code>source</code> the file called <code>g5_modules</code>:

you should see:

Currently Loaded Modulefiles:

gmake install

This will build the model.  It will take about 40 minutes. If this works, it should create a directory under <code>GEOSagcm</code> called <code>Linux/bin</code>.  In here you should find the executable: <code>GEOSgcm.x</code> .

First of all, to run jobs on the cluster you will need to set up passwordless <code>ssh</code> (which operates within the cluster).  To do so, run the following from your '''discover''' home directory:

cd .ssh

cat id_dsa.pub >>  authorized_keys

Similarly, transferring the daily output files (in monthly tarballs) requires passwordless authentication from '''discover''' to '''palm'''.  While in <code>~/.ssh</code> on '''discover''', run

  ssh-keygen -t dsa

Then, log into '''palm''' and cut and paste the contents of the <code>id_rsa.pub</code> and <code>id_dsa.pub</code> files on '''discover''' into the <code>~/.ssh/authorized_keys</code> file on  '''palm'''.

To set the model up to run, in the  <code>GEOSagcm/src/Applications/GEOSgcm_App</code> directory we run:

The <code>gcm_setup</code> script asks you a few questions such as an experiment name (with no spaces, called ''EXPID'') and description (spaces ok). It will also ask you for the model resolution, expecting one of the available lat-lon domain sizes, the dimensions separated by a space.  For your first time out you will probably want to enter <code>144 91</code> (corresponding to ~2 degree resolution).  Towards the end it will ask you for a group ID -- the default is g0602 (GMAO modeling group).  Enter whatever is appropriate, as necessary. The rest of the questions provide defaults which will be suitable for now, so just press enter for these.

*<code>post</code>  (containing the post-processing job script)

*<code>archive</code>  (containing an incomplete archiving job script)

*<code>plot</code>  (containing an incomplete plotting job script)

The post-processing script will complete (i.e., add necessary commands to) the archiving and plotting scripts as it runs. The setup script that you ran also creates an experiment home directory (''HOMEDIR'') as <code>~''USERID''/geos5/''EXPID''</code>  containing the run scripts and GEOS resource (<code>.rc</code>) files.

The run scripts need some more environment variables -- here are the minimum contents of a <code>.cshrc</code>:

  limit stacksize unlimited

  source ~/.g5_modules

  set arch = `uname`

  setenv LD_LIBRARY_PATH ${LIBRARY_PATH}:${BASEDIR}/${arch}/lib

 

 

 

The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''.  It should be ready to go as is, though you may eventually want to tune JOB_SGMT (the number of days per segment, the internal between saving restarts) and NUM_SGMT (the number of segments attempted in a job) to maximize your run time.  Leave END_DATE alone in Fortuna 2.0 -- there is a bug that erroneously resubmits the script after this date.  You can stop the run by commenting out the <code>qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted.  Those and the PBS (batch system) parameters at the beginning are all that you will usually want to change in the script.

 

Submit the job with <code>qsub gcm_run.j</code>.  You can keep track of it with <code>qstat</code> or <code>qstat | grep ''USERID''</code>, or stdout with <code>tail -f /discover/pbs_spool/''JOBID''.OU</code>, ''JOBID'' being returned by <code>qsub</code> and displayed with <code>qstat</code>.  Jobs can be killed with <code>qdel ''JOBID''</code>.  The standard out and standard error will be delivered as files to the working directory at the time you submitted the job.

 

 

During a normal run, the <code>gcm_run.j</code> script will run the model for the segment length (current default is 8 days). The model creates output files (with an <code>nc4</code> extension), also called collections (of output variables), in  <code>''EXPDIR''/scratch</code> directory.  After each segment, the script moves the output to the <code>''EXPDIR''/holding</code> and spawns a post-processing batch job which partitions and moves the output files  within the <code>holding</code> directory to their own distinct collection directory, which is again partitioned into the appropriate year and month.  The  post processing script then checks to

see if  a full month of data is present.  If not, the post-processing job ends.  If there is a full month, the script will then run the time-averaging executable to produce a monthly mean file in <code>''EXPDIR''/geos_gcm_*</code>. The post-processing script then spawns a new batch job which will archive the data onto the mass-storage drives (<code>/archive/u/''USERID''/GEOS5.0/''EXPID''</code>).

plotting are:  1) if the month created was February or August,  AND

the seasons DJF or JJA will be issued.  The plots are created as gifs in <code>''EXPDIR''/plots</code>.

 

 

The contents of the output files (including which variables get saved) may be configured in the  <code>''HOMEDIR''/HISTORY/tmpl</code> -- a good description of this file may be found at http://modelingguru.nasa.gov/clearspace/docs/DOC-1190 .