Running the GEOS-5 SBU Benchmark: Difference between revisions

← Older edit

@@ Line 105: / Line 105: @@
 These effectively let you change whatever you want - useful for debugging, etc. For example, you can set your timers in ~/.esma_base.mk.
+==Test Build with One-Day Run==
+To make sure all works, we will first try setting up a simple one-day experiment.
+===Setting up a one-day experiment===
+Go into the model application directory and do a couple of preliminary commands:
+ $ cd $ESMADIR/src/Applications/GEOSgcm_App
+Now you can run <tt>create_expt.py</tt>:
+ <nowiki>$ ~mathomp4/bin/create_exp.py -h
+usage: create_expt.py [-h] [-v] [-q] [--expdsc EXPDSC] [--expdir EXPDIR]
+                      [--horz {a,b,c,d,e,c12,c24,c48,c90,c180,c360,c720,c1440,c2880}]
+                      [--vert {72,132}] [--ocean {o1,o2,o3}] [--land {1,2}]
+                      [--runoff {yes,no}] [--gocart {A,C}]
+                      [--emission {MERRA2,PIESA,CMIP,NR,MERRA2-DD,OPS}]
+                      [--history HISTORY] [--account ACCOUNT] [--gpu]
+                      expid
+Utility to quickly create experiment. At present, it creates an experiment in
+the same manner as gcm_setup would with home and experiment directories as
+usual.
+positional arguments:
+  expid                 Experiment ID
+optional arguments:
+  -h, --help            show this help message and exit
+  -v, --verbose         Verbose output
+  -q, --quiet           Quietly Setup Experiment (no printing)
+  --expdsc EXPDSC       Experiment Description (Default: same as expid)
+  --expdir EXPDIR       Experiment Directory Root *NOT CONTAINING EXPID*
+                        (Default is what is in ~/.EXPDIRroot:
+                        /discover/nobackup/mathomp4 )
+  --horz {a,b,c,d,e,c12,c24,c48,c90,c180,c360,c720,c1440,c2880}
+                        Horizontal Resolution (Default: c48 on clusters, c12
+                        on desktop)
+  --vert {72,132}       Vertical Resolution (Default: 72)
+  --ocean {o1,o2,o3}    Data Ocean Resolution (Default: o1)
+  --land {1,2}          Land Surface Model (Default: 1)
+  --runoff {yes,no}     Runoff Routing Model (Default: no)
+  --gocart {A,C}        GOCART aerosols: Actual (A) or Climatological (C)
+                        (Default: A on clusters, C on desktops)
+  --emission {MERRA2,PIESA,CMIP,NR,MERRA2-DD,OPS}
+                        GOCART Emissions to use (Default: MERRA2)
+  --history HISTORY     History Template (Default: Current)
+  --account ACCOUNT     Account Number to Use (Default: g0620 at NCCS, g26141
+                        at NAS)
+  --gpu                 Setup Experiment to use GPUs</nowiki>
+This is a script that attempts to ease setting up a GEOS-5 AGCM run. It has some "smarts" in that it will detect from an experiment ID some information, but you can be explicit (to the point you can specify impossible experiments too).
+Your first choice is where you'd like to run the experiment. $NOBACKUP or other disks with space are vital. For example, if you'd like to run the experiment in /discover/nobackup/username/experiment-name, then run the script with --expdir /discover/nobackup/username. To actually create the experiment, run the script and choose a C48 horizontal resolution:
+ $ ~mathomp4/bin/create_expt.py test-1day-expt --horz c48 --account <ACCOUNTID> --expdir /discover/nobackup/username
+ Horizontal resolution c48 passed in
+ Using c48 horizontal resolution
+ Assuming default vertical resolution of 72
+ Using 72 vertical resolution
+ Assuming default ocean resolution of o1
+ Using o1 ocean resolution
+ Using actual aerosols
+  Running gcm_setup...done.
+ Experiment is located in directory: /discover/nobackup/username/test-1day-expt
+If you don't pass in an account-id, you'll get the default of g0620 (the developer's account).
+===Run a one-day test===
+To run a test one-day run, go to the experiment directory:
+ <nowiki>$ cd /discover/nobackup/mathomp4/test-1day-expt
+total 96M
+-rwxr-xr-x 1 mathomp4 g0620 96M Feb 13 11:11 GEOSgcm.x*
+-rwxr-xr-x 1 mathomp4 g0620 573 Feb 13 14:47 CAP.rc*
+-rwxr-xr-x 1 mathomp4 g0620 364 Feb 13 14:47 fvcore_layout.rc*
+-rw-r--r-- 1 mathomp4 g0620 25K Feb 13 14:47 AGCM.rc
+-rwxr-xr-x 1 mathomp4 g0620 21K Feb 13 14:47 gcm_run.j*
+-rw-r--r-- 1 mathomp4 g0620 61K Feb 13 14:47 HISTORY.rc
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 post/
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 plot/
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 archive/
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 regress/
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 convert/
+drwxr-xr-x 2 mathomp4 g0620 512 Feb 13 14:47 forecasts/
+drwxr-xr-x 2 mathomp4 g0620 16K Feb 13 14:47 RC/
+Directory: /discover/nobackup/mathomp4/test-1day-expt</nowiki>
+Now you'll run another script called <tt>makeoneday.bash</tt> that edits some .rc files, edits run scripts, as well as linking in some example restarts and sets up the model to stop after one day (with no options). Note: these restarts are write-protected, so if you want to run a multi-year run, don't use this script!
+ <nowiki>$ ~mathomp4/bin/makeoneday.bash
+Using Heracles-5_0 directories
+Making one-day experiment
+Restoring AGCM.rc.save to AGCM.rc...
+Copying AGCM.rc to AGCM.rc.save...
+Restoring CAP.rc.save to CAP.rc...
+Copying CAP.rc to CAP.rc.save...
+Restoring HISTORY.rc.save to HISTORY.rc...
+Copying HISTORY.rc to HISTORY.rc.save...
+Restoring gcm_run.j.save to gcm_run.j...
+Copying gcm_run.j to gcm_run.j.save...
+Restoring regress/gcm_regress.j.save to regress/gcm_regress.j...
+Copying regress/gcm_regress.j to regress/gcm_regress.j.save...
+Restoring RC/GEOS_ChemGridComp.rc.save to RC/GEOS_ChemGridComp.rc...
+Copying RC/GEOS_ChemGridComp.rc to RC/GEOS_ChemGridComp.rc.save...
+DYN_INTERNAL_RESTART_TYPE not found. Assuming NC4
+Found fvcore_internal_rst. Assuming you have needed restarts!
+Changes made to CAP.rc:
+,10c9,10
+< JOB_SGMT:     00000015 000000
+< NUM_SGMT:     20
+---
+> JOB_SGMT:     00000001 000000
+> NUM_SGMT:     1
+c25
+< MAPL_ENABLE_TIMERS: NO
+---
+> MAPL_ENABLE_TIMERS: YES
+Changes made to gcm_run.j:
+c7
+< #PBS -l walltime=12:00:00
+---
+> #PBS -l walltime=0:15:00
+a12
+> #SBATCH --mail-type=ALL
+c261
+< if($numrs == 0) then
+---
+> if($numrs == 1) then
+a435
+> exit
+</nowiki>
+At this point, you should be able to <tt>sbatch gcm_run.j</tt> and the model should run a day.
 -->
@@ Line 130: / Line 276: @@
 To create the experiment, run create_expt.py and choose a C720 horizontal resolution with climatological GOCART:
-  $ $PORTBCS/scripts/create_expt.py benchmark-GEOSadas-5_16-5-5day-c720-Portable --horz c720 --ocean o3 --gocart C --account <ACCOUNTID> --expdir <root-for-experiment>
+  $ $PORTBCS/scripts/create_expt.py benchmark-GEOSadas-5_16-5-5day-c720 --horz c720 --ocean o3 --gocart C --account <ACCOUNTID> --expdir <root-for-experiment>
   Found c720 horizontal resolution in experiment name
   Using c720 horizontal resolution
@@ Line 143: / Line 289: @@
    Running gcm_setup...done.
-  Experiment is located in directory: <root-for-experiment>/benchmark-GEOSadas-5_16-5-5day-c720-OpsHistory
+  Experiment is located in directory: <root-for-experiment>/benchmark-GEOSadas-5_16-5-5day-c720
 Again, if you don't pass in an account-id, you'll get the default of g0620 (the developer's account).