GEOS-5 Run-Time Characteristics for Heracles 5.2
AGCM Performance at various resolutions
Results for a 10-day AMIP-style simulations at 72 levels using MERRA2 GOCART aerosols, MERRA-2 ocean, and running on Haswell CPUs. In this table, all were run with the same HISTORY collections, though at varying output resolutions.
| Cube Grid | Model Resolution | Equiv. Lat-Lon Output Resolution | CPUs (typical) | Timestep (sec) | Wall Clock (Hours) | Throughput (Days per day) | CPU Hours | Typical Data Volume (GB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| C12 | 800-km | 8° (48x25) | 24 | 900 | 0.16 | 1380 | 5 | 4 | Reynolds Ocean |
| C24 | 400-km | 4° (96x49) | 96 | 900 | 0.20 | 1340 | 23 | 6 | Reynolds Ocean |
| C48 | 200-km | 2° (180x91) | 96 | 900 | 0.30 | 745 | 38 | 13 | |
| C90 | 100-km | 1° (360x181) | 96 | 900 | 0.65 | 383 | 73 | 38 | |
| C180 | 50-km | ½° (720x361) | 216 | 450 | 1.67 | 150 | 360 | 140 | |
| C360 | 25-km | ¼° (1440x721) | 864 | 450 | 2.25 | 107 | 1980 | 530 | |
| C720 | 12.5-km | ⅛° (2880x1441) | 1536 | 300 | 2090 |
AGCM Scaling for Cubed-Sphere Version at C180 (~½°, 2-day run)
| Layout NX | Layout NY | Total PEs | Total Time (sec) | Dynamics Time (sec) | Physics Time (sec) | Dynamics Percent | Physics Percent |
|---|---|---|---|---|---|---|---|
| 6 | 36 | 216 | 1170 | 407 | 667 | 34.8% | 57.0% |
| 8 | 36 | 288 | 945 | 326 | 536 | 34.5% | 56.7% |
| 8 | 48 | 384 | 780 | 269 | 426 | 34.5% | 54.6% |
| 12 | 72 | 864 | 534 | 166 | 275 | 31.1% | 51.5% |
AOGCM Performance
Here we present two runs of Heracles 5.2 to compare coupled AOGCM to an AGCM run of the same atmospheric resolution but without coupled to MOM. The first line is the coupled AOGCM and the second is the pure AGCM. Results for a 10-day AMIP-style simulations at 72 atmospheric levels and 40 ocean levels using MERRA2 GOCART aerosols running on Haswell CPUs.
The large discrepancy in data volume is due to different HISTORY files being run, i.e., the coupled run output some ocean-specific collections the pure AGCM run did not. As always, data volume depends on the output requested by the user.
| Atmos Res | Ocean Res | CPUs | Timestep (sec) | Wall Clock (Hours) | Throughput (Days per day) | CPU Hours | Typical Data Volume (GB) | Notes |
|---|---|---|---|---|---|---|---|---|
| C90 (1°) | 720x410 (½°) | 180 | 900 | 0.92 | 261 | 205 | 85 | Data size depends on HISTORY used |
| C90 (1°) | MERRA2 dataocean | 180 | 900 | 0.67 | 374 | 130 | 38 | Data size depends on HISTORY used |
Doubly-Periodic Performance
These runs exercise the doubly-periodic model in Heracles 5.2. All runs are made with 81 CPUs and ran for 1 day. In the Dynamics DT column, H = hydrostatic and NH = non-hydrostatic.
| DP Grid | Resolution | Heartbeat DT | Dynamics DT | Wallclock (sec) | Throughput (Days per day) |
|---|---|---|---|---|---|
| 180x180 | 14-km | 300 s | 25 s (H) | 600 | 144 |
| 180x180 | 7-km | 300 s | 10 s (NH) | 960 | 90 |
| 180x180 | 3-km | 300 s | 5 s (NH) | 1320 | 65 |
| 180x180 | 1-km | 300 s | 2 s (NH) | 2448 | 35 |