GEOS-5 Run-Time Characteristics for Heracles 5.2

AGCM Performance at various resolutions

Results for a 10-day AMIP-style simulations at 72 levels using MERRA2 GOCART aerosols, MERRA-2 ocean, and running on Haswell CPUs. In this table, all were run with the same HISTORY collections, though at varying output resolutions.

Cube Grid	Model Resolution	Equiv. Lat-Lon Output Resolution	CPUs (typical)	Timestep (sec)	Wall Clock (Hours)	Throughput (Days per day)	CPU Hours	SBUs	Typical Data Volume (GB)	Notes
C12	800-km	8° (48x25)	24	900	0.16	1380	5	0.49	4	Reynolds Ocean
C24	400-km	4° (96x49)	96	900	0.20	1340	23	2.47	6	Reynolds Ocean
C48	200-km	2° (180x91)	96	900	0.30	745	38	3.71	13
C90	100-km	1° (360x181)	96	900	0.65	383	73	8.03	38
C180	50-km	½° (720x361)	216	450	1.67	150	360	41.28	140
C360	25-km	¼° (1440x721)	864	450	2.25	107	1980	215.53	530
C720	12.5-km	⅛° (2880x1441)	1536	300	7.08	34	10945	1203.25	2090

AGCM Scaling for Cubed-Sphere Version at C180 (~½°, 2-day run)

Layout NX x NY	Total PEs	Total Time (sec)	Dynamics Time (sec)	Physics Time (sec)	Dynamics Percent	Physics Percent
6 x 36	216	1170	407	667	34.8%	57.0%
8 x 36	288	945	326	536	34.5%	56.7%
8 x 48	384	780	269	426	34.5%	54.6%
12 x 72	864	534	166	275	31.1%	51.5%

GMI-GOCART Performance

Performance for a 15-day replay run using GMI Chemistry and GOCART with a sparse (but frequent) output stream

Atmos Res	CPUs	Timestep (sec)	Wall Clock (Hours)	Throughput (Days per day)	CPU Hours	SBUs	Typical Data Volume (GB)	Notes
C180 (½°)	840	450	0.92	261	205	22.74	85	Data size depends on HISTORY used

AOGCM Performance

Here we present two runs of Heracles 5.2 to compare coupled AOGCM to an AGCM run of the same atmospheric resolution but without coupled to MOM. The first line is the coupled AOGCM and the second is the pure AGCM. Results for a 10-day AMIP-style simulations at 72 atmospheric levels and 40 ocean levels using MERRA2 GOCART aerosols running on Haswell CPUs.

The large discrepancy in data volume is due to different HISTORY files being run, i.e., the coupled run output some ocean-specific collections the pure AGCM run did not. As always, data volume depends on the output requested by the user.

Atmos Res	Ocean Res	CPUs	Timestep (sec)	Wall Clock (Hours)	Throughput (Days per day)	CPU Hours	SBUs	Typical Data Volume (GB)	Notes
C90 (1°)	720x410 (½°)	180	900	0.92	261	205	22.74	85	Data size depends on HISTORY used
C90 (1°)	MERRA2 dataocean	180	900	0.67	374	130	16.56	38	Data size depends on HISTORY used

Doubly-Periodic Performance

These runs exercise the doubly-periodic model in Heracles 5.2. All runs are made with 81 CPUs and ran for 1 day. In the Dynamics DT column, H = hydrostatic and NH = non-hydrostatic.

DP Grid	Resolution	Heartbeat DT	Dynamics DT	Wallclock (sec)	Throughput (Days per day)
180x180	14-km	300 s	25 s (H)	600	144
180x180	7-km	300 s	10 s (NH)	960	90
180x180	3-km	300 s	5 s (NH)	1320	65
180x180	1-km	300 s	2 s (NH)	2448	35

GEOS-5 ADAS Performance

Note: Time greatly depends on the date of the assimilation due to differing numbers of observations.

GEOS 5.13.1 ADAS Forecasts

Results for GEOS 5.13.1 ADAS forecasts of various lengths. The values presented as C/E are refer to the Central v. Ensemble GCM. There were 32 members in the ensemble.

Cube Grid	Model Resolution	Equiv. Lat-Lon Output Resolution	CPUs	Timestep (sec)	Forecast Length	Wall Clock (Hours)	CPU Hours	SBUs	Notes
C360/C90	25-km/100-km	¼° (1152x721)	1152/48	450/450	33 hours	1.02	1383	151.29
					5 days	1.97	2638	292.19
					10 days	2.78	3751	412.33

MERRA2 ADAS

Results for a 1-day MERRA2 ADAS run at 72 levels using MERRA2 GOCART aerosols, MERRA-2 ocean, and running on Haswell CPUs.

Cube Grid	Model Resolution	Equiv. Lat-Lon Output Resolution	CPUs (typical)	Timestep (sec)	Wall Clock (Hours)	Throughput (Days per day)	CPU Hours	SBUs	Notes
C180	50-km	½° (576x361)	240	450	2.12	11.3	592	65.51	Model: GEOS 5.14.1 UNSTABLE (2016-Jun-07)

Previous Models

• GEOS-5 Run-Time Characteristics for Ganymed 1.0
• GEOS-5 Run-Time Characteristics for Ganymed 4.0