Ganymed 1.0 Quick Start: Difference between revisions

(8 intermediate revisions by the same user not shown)

Line 22:

where ''USERID'' is, of course, your repository username, which should be the same as your NASA and NCCS username. Then, issue the command:

cvs co -r Ganymed-~~1_0~~ -d Ganymed-~~1_0~~ Ganymed

cvs co -r Ganymed-1_0_p7 -d Ganymed-1_0_p7 Ganymed

Line 140:

*<code>regress/</code> -- contains scripts for doing regression testing of model

The post-processing script will generate the archiving and plotting scripts as it runs. The setup script that you ran also creates an experiment home directory (''HOMEDIR'') as <code>~''USERID''/geos5/''EXPID''</code> containing the run scripts and GEOS resource (<code>.rc</code>) files. (You can also specify the experiment home directory to be the same as the experiment directory.)

== Running GEOS-5 ==

Line 153:

Line 155:

The <code>umask 022</code> is not strictly necessary, but it will make the various files readable to others, which will facilitate data sharing and user support. Your home directory <code>~''USERID''</code> is also inaccessible to others by default; running <code>chmod 755 ~</code> is helpful.

Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''. Keep the "originals" handy since if the job stumbles early in the run it might stop after having renamed them. The model expects restart filenames to end in "rst" but produces them with the date and time appended, so you may have to rename them back to ending in "rst". The <code>cap_restart</code> file is sometimes provided with a set of restarts, but if not you can create it: is simply one line of text with the date of the restart files in the format ''<code>YYYYMMDD HHMMSS</code>'' (with a space). The boundary conditions/forcings are provided by symbolic links created by the run script.

Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''. Keep the "originals" handy since if the job stumbles early in the run it might stop after having renamed them. The model expects restart filenames to end in "rst" but produces them with the date and time appended, so you may have to rename them back to ending in "rst". The <code>cap_restart</code> file is sometimes provided with a set of restarts, but if not you can create it: it is simply one line of text with the date of the restart files in the format ''<code>YYYYMMDD HHMMSS</code>'' (with a space). The boundary conditions/forcings are provided by symbolic links created by the run script. If you need an arbitrary set of restarts, you can copy them from <code>/archive/u/aeichman/restarts/Ganymed-1_0/</code>, where they are indexed by resolution and date. If you are unfamiliar with the way that the <code>/archive</code> filesystem works, keep in mind that a <code>cp</code> from there might appear to stall while the tape is loaded -- see the NCCS documentation for details.

If you need an arbitrary set of restarts, you can copy them from <code>/archive/u/aeichman/restarts/Ganymed-1_0/</code>, where they are indexed by resolution and date.

The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''. It should be ready to go as is. The parameter END_DATE in <code>CAP.rc</code> ~~(previously in <code>gcm_run.j</code>)~~ can be set to the date you want the run to stop. An alternative way to stop the run is by commenting out the line <code> if ( $capdate < $enddate ) qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted, or rename the script file~~. You may eventually want to tune parameters in the <code>CAP.rc</code> file JOB_SGMT (the number of days per segment~~, the ~~interval between saving restarts) and NUM_SGMT~~ (~~the number of segments attempted in a job~~) ~~to maximize your run time~~.

The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''. It should be ready to go as is. The parameter END_DATE in <code>CAP.rc</code> can be set to the date you want the run to stop. An alternative way to stop the run is by commenting out the line <code> if ( $capdate < $enddate ) qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted, or rename the script file, or kill the job (described below).

Submit the job with <code>qsub gcm_run.j</code>. You can keep track of it with <code>qstat</code> or <code>qstat | grep ''USERID''</code>, or follow stdout with <code>tail -f /discover/pbs_spool/''JOBID''.OU</code>, ''JOBID'' being returned by <code>qsub</code> and displayed with <code>qstat</code>. Jobs can be killed with <code>qdel ''JOBID''</code>. The standard out and standard error will be delivered as files to the working directory at the time you submitted the job.

Line 165:

Line 163:

== Output and Plots ==

During a normal run, the <code>gcm_run.j</code> script will run the model for the segment length (current default is 15 days). The model creates output files (with an <code>nc4</code> extension), also called collections (of output variables), in <code>''EXPDIR''/scratch</code> directory. After each segment, the script moves the output to the <code>''EXPDIR''/holding</code> and spawns a post-processing batch job which partitions and moves the output files within the <code>holding</code> directory to their own distinct collection directory, which is again partitioned into the appropriate year and month. The post processing script then checks to

During a normal run, the <code>gcm_run.j</code> script will run the model for the segment length (current default is 15 days in model time). The model creates output files (with an <code>nc4</code> extension), also called collections (of output variables), in <code>''EXPDIR''/scratch</code> directory. After each segment, the script moves the output to the <code>''EXPDIR''/holding</code> and spawns a post-processing batch job which partitions and moves the output files within the <code>holding</code> directory to their own distinct collection directory, which is again partitioned into the appropriate year and month. The post processing script then checks to

see if a full month of data is present. If not, the post-processing job ends. If there is a full month, the script will then run the time-averaging executable to produce a monthly mean file in <code>''EXPDIR''/geosgcm_*</code>. The post-processing script then spawns a new batch job which will archive the data onto the mass-storage drives (<code>/archive/u/''USERID''/GEOS5.0/''EXPID''</code>).

@@ Line 22: / Line 22: @@
 where ''USERID'' is, of course, your repository username, which should be the same as your NASA and NCCS username.  Then, issue the command:
-  cvs co -r Ganymed-1_0 -d Ganymed-1_0 Ganymed
+  cvs co -r Ganymed-1_0_p7 -d Ganymed-1_0_p7 Ganymed
@@ Line 140: / Line 140: @@
 *<code>regress/</code> -- contains scripts for doing regression testing of model
+The post-processing script will generate the archiving and plotting scripts as it runs.  The setup script that you ran also creates an experiment home directory (''HOMEDIR'') as <code>~''USERID''/geos5/''EXPID''</code>  containing the run scripts and GEOS resource (<code>.rc</code>) files.  (You can also specify the experiment home directory to be the same as the experiment directory.)
 == Running GEOS-5 ==
@@ Line 153: / Line 155: @@
 The <code>umask 022</code> is not strictly necessary, but it will make the various files readable to others, which will facilitate data sharing and user support.  Your home directory <code>~''USERID''</code> is also inaccessible to others by default; running <code>chmod 755 ~</code> is helpful.
-Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''.   Keep the "originals" handy since if the job stumbles early in the run it might stop after having renamed them.  The model expects restart filenames to end in "rst" but produces them with the date and time appended, so you may have to rename them back to ending in "rst".   The <code>cap_restart</code> file is sometimes provided with a set of restarts, but if not you can create it: is simply one line of text with the date of the restart files in the format ''<code>YYYYMMDD HHMMSS</code>'' (with a space).  The boundary conditions/forcings are provided by symbolic links created by the run script.
+Copy the restart (initial condition) files and associated <code>cap_restart</code> into ''EXPDIR''.   Keep the "originals" handy since if the job stumbles early in the run it might stop after having renamed them.  The model expects restart filenames to end in "rst" but produces them with the date and time appended, so you may have to rename them back to ending in "rst".   The <code>cap_restart</code> file is sometimes provided with a set of restarts, but if not you can create it: it is simply one line of text with the date of the restart files in the format ''<code>YYYYMMDD HHMMSS</code>'' (with a space).  The boundary conditions/forcings are provided by symbolic links created by the run script.  If you need an arbitrary set of restarts, you can copy them from <code>/archive/u/aeichman/restarts/Ganymed-1_0/</code>, where they are indexed by resolution and date.  If you are unfamiliar with the way that the <code>/archive</code> filesystem works, keep in mind that a <code>cp</code> from there might appear to stall while the tape is loaded -- see the NCCS documentation for details.
-If you need an arbitrary set of restarts, you can copy them from <code>/archive/u/aeichman/restarts/Ganymed-1_0/</code>, where they are indexed by resolution and date.
-The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''.  It should be ready to go as is.  The parameter END_DATE in <code>CAP.rc</code> (previously in <code>gcm_run.j</code>) can be set to the date you want the run to stop.   An alternative way to stop the run is by commenting out the line <code> if ( $capdate < $enddate ) qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted, or rename the script file.  You may eventually want to tune parameters in the <code>CAP.rc</code> file JOB_SGMT (the number of days per segment, the interval between saving restarts) and NUM_SGMT (the number of segments attempted in a job) to maximize your run time.
+The script you submit, <code>gcm_run.j</code>, is in ''HOMEDIR''.  It should be ready to go as is.  The parameter END_DATE in <code>CAP.rc</code> can be set to the date you want the run to stop.   An alternative way to stop the run is by commenting out the line <code> if ( $capdate < $enddate ) qsub $HOMDIR/gcm_run.j</code> at the end of the script, which will prevent the script from being resubmitted, or rename the script file, or kill the job (described below).
 Submit the job with <code>qsub gcm_run.j</code>.  You can keep track of it with <code>qstat</code> or <code>qstat | grep ''USERID''</code>, or follow stdout with <code>tail -f /discover/pbs_spool/''JOBID''.OU</code>, ''JOBID'' being returned by <code>qsub</code> and displayed with <code>qstat</code>.  Jobs can be killed with <code>qdel ''JOBID''</code>.  The standard out and standard error will be delivered as files to the working directory at the time you submitted the job.
@@ Line 165: / Line 163: @@
 == Output and Plots ==
-During a normal run, the <code>gcm_run.j</code> script will run the model for the segment length (current default is 15 days).  The model creates output files (with an <code>nc4</code> extension), also called collections (of output variables), in  <code>''EXPDIR''/scratch</code> directory.  After each segment, the script moves the output to the <code>''EXPDIR''/holding</code> and spawns a post-processing batch job which partitions and moves the output files  within the <code>holding</code> directory to their own distinct collection directory, which is again partitioned into the appropriate year and month.  The  post processing script then checks to
+During a normal run, the <code>gcm_run.j</code> script will run the model for the segment length (current default is 15 days in model time).  The model creates output files (with an <code>nc4</code> extension), also called collections (of output variables), in  <code>''EXPDIR''/scratch</code> directory.  After each segment, the script moves the output to the <code>''EXPDIR''/holding</code> and spawns a post-processing batch job which partitions and moves the output files  within the <code>holding</code> directory to their own distinct collection directory, which is again partitioned into the appropriate year and month.  The  post processing script then checks to
 see if  a full month of data is present.  If not, the post-processing job ends.  If there is a full month, the script will then run the time-averaging executable to produce a monthly mean file in <code>''EXPDIR''/geosgcm_*</code>.  The post-processing script then spawns a new batch job which will archive the data onto the mass-storage drives (<code>/archive/u/''USERID''/GEOS5.0/''EXPID''</code>).