GEOS-5

Running the GEOS-5 SBU Benchmark: Difference between revisions

← Older edit
 
(6 intermediate revisions by the same user not shown)
Line 106: Line 106:
These effectively let you change whatever you want - useful for debugging, etc. For example, you can set your timers in ~/.esma_base.mk.
These effectively let you change whatever you want - useful for debugging, etc. For example, you can set your timers in ~/.esma_base.mk.


-->


==Test Build with One-Day Run==
==Test Build with One-Day Run==
Line 112: Line 111:
To make sure all works, we will first try setting up a simple one-day experiment.
To make sure all works, we will first try setting up a simple one-day experiment.


===Learn to love tcsh===
One preliminary note is that GEOS-5 is, in many ways, a collection of csh/tcsh scripts. If things start going wrong, the answer can often be "change your shell to tcsh and try". Yes, it's not bash/fish/zsh, but it is what it is. I don't think it's entirely necessary for just this automated work, but it could happen.


===Setting up a one-day experiment===
===Setting up a one-day experiment===
Line 255: Line 251:


At this point, you should be able to <tt>sbatch gcm_run.j</tt> and the model should run a day.
At this point, you should be able to <tt>sbatch gcm_run.j</tt> and the model should run a day.
-->


==Benchmark Run==
==Benchmark Run==


The full benchmark run is a run of 5-days using the GMAO Ops setup. This will use space and cores. For effective benchmarking of I/O, it's recommended to run on less congested than nobackup.
The full benchmark run is a run of 5-days using a portable version of the GEOS-5 boundary conditions. This will use space and cores. For effective benchmarking of I/O, it's recommended to run on less congested than nobackup.
 
===Learn to love tcsh===
 
One preliminary note is that GEOS-5 is, in many ways, a collection of csh/tcsh scripts. If things start going wrong, the answer can often be "change your shell to tcsh and try". Yes, it's not bash/fish/zsh, but it is what it is. I don't think it's entirely necessary for just this automated work, but it could happen.


===Setting up benchmark experiment===
===Setting up benchmark experiment===
Line 268: Line 270:
These echos set up some defaults. While they both don't have to be in the same location, it's highly recommended they are and the use of the script below assumes they are. Also, make sure they are in a nobackup directory.
These echos set up some defaults. While they both don't have to be in the same location, it's highly recommended they are and the use of the script below assumes they are. Also, make sure they are in a nobackup directory.


To actually create the experiment, run the create_expy.py and choose a C720 horizontal resolution with OPS emissions:
For the next few commands, you will need to know the location of the portable BCs directory used for this experiment, which is referred to below as <tt>$PORTBCS</tt>. On discover, a version will always be at:
 
/discover/nobackup/mathomp4/HugeBCs-H50


  $ ~mathomp4/bin/create_expt.py benchmark-GEOSadas-5_16-5-5day-c720-OpsHistory --horz c720 --ocean o3 --emission OPS --account <ACCOUNTID> --expdir <root-for-experiment>
To create the experiment, run create_expt.py and choose a C720 horizontal resolution with climatological GOCART:
 
  $ $PORTBCS/scripts/create_expt.py benchmark-GEOSadas-5_16-5-5day-c720 --horz c720 --ocean o3 --gocart C --account <ACCOUNTID> --expdir <root-for-experiment>
  Found c720 horizontal resolution in experiment name
  Found c720 horizontal resolution in experiment name
  Using c720 horizontal resolution
  Using c720 horizontal resolution
Line 280: Line 286:
  Using o3 ocean resolution
  Using o3 ocean resolution
   
   
  Using actual aerosols
  Using climatological aerosols
   Running gcm_setup...done.
   Running gcm_setup...done.
   
   
  Experiment is located in directory: <root-for-experiment>/benchmark-GEOSadas-5_16-5-5day-c720-OpsHistory
  Experiment is located in directory: <root-for-experiment>/benchmark-GEOSadas-5_16-5-5day-c720


Again, if you don't pass in an account-id, you'll get the default of g0620 (the developer's account).
Again, if you don't pass in an account-id, you'll get the default of g0620 (the developer's account).
Line 289: Line 295:
===Setup and Run Benchmark===
===Setup and Run Benchmark===


Now ''change to the experiment directory'' and run makeoneday.bash to create it with Ops like options:
Now ''change to the experiment directory'' and run MakeSBUBench.bash which will set up the experiment:


  $ ~mathomp4/bin/makeoneday.bash opsb
  $ $PORTBCS/scripts/MakeSBUBench.bash


The <tt>opsb</tt> option sets up many things. The run is set for 5 days using 5400 cores, and other flags are tripped to best emulate Ops. It also copies in the Ops HISTORY.rc setup.
The script sets the run up for 5 days using 5400 cores, and other flags are tripped to best emulate Ops.


'''NOTE''': This will also set the experiment to run in this SLURM environment:
'''NOTE''': This will also set the experiment to run in this SLURM environment:
Line 302: Line 308:
This is how the script's developer can run 5400-core jobs. Others might have different partition/qos to submit to. Please edit these before sbatch submission to a <tt>partition/qos</tt> that you have access to that can accept a 5400-core job.
This is how the script's developer can run 5400-core jobs. Others might have different partition/qos to submit to. Please edit these before sbatch submission to a <tt>partition/qos</tt> that you have access to that can accept a 5400-core job.


Finally:
Finally, submit the job:


  $ sbatch gcm_run.j
  $ sbatch gcm_run.j
Retrieved from "https://geos5.org/wiki/index.php?title=Running_the_GEOS-5_SBU_Benchmark"