History Control: Difference between revisions

(21 intermediate revisions by the same user not shown)

Line 1:

This section provides a brief overview of the GEOS GCM output control. At its highest level, model output is controlled through a resource file named HISTORY.rc; many times, this file comes in templated format, which means it contains entries that are filled in at runtime, in such cases, the ~~files~~ is typically found ~~under the~~ name ~~HISTORY.rc.tmpl~~.

This section provides a brief overview of the GEOS GCM output control. At its highest level, model output is controlled through a resource file named HISTORY.rc; many times, this file comes in templated format, which means it contains entries that are filled in at runtime, in such cases, the file is typically found with an extension "tmpl" appended to its name.

The output of GEOS GCM is subset in files, referred to as COLLECTIONS. An example of a COLLECTION table is provided below:

COLLECTIONS: 'geosgcm_prog'

'geosgcm_surf'

COLLECTIONS: 'geosgcm_prog'

'geosgcm_moist'

'geosgcm_surf'

'geosgcm_turb'

'geosgcm_moist'

'geosgcm_gwd'

'geosgcm_turb'

'geosgcm_tend'

'geosgcm_gwd'

'geosgcm_budi'

'geosgcm_tend'

'geosgcm_buda'

'geosgcm_budi'

'geosgcm_landice'

'geosgcm_buda'

'geosgcm_meltwtr'

'geosgcm_landice'

'geosgcm_snowlayer'

'geosgcm_meltwtr'

'geosgcm_tracer'

'geosgcm_snowlayer'

>>>HIST_GOCART<<< 'tavg2d_aer_x'

'geosgcm_tracer'

>>>HIST_GOCART<<< 'tavg3d_aer_p'

>>>HIST_GOCART<<< 'tavg2d_aer_x'

# 'geosgcm_iau'

>>>HIST_GOCART<<< 'tavg3d_aer_p'

# 'geosgcm_conv'

# 'geosgcm_iau'

# 'goswim_catch'

# 'geosgcm_conv'

# 'goswim_land'

# 'goswim_catch'

# 'goswim_landice'

# 'goswim_land'

# 'geosgcm_lidar'

# 'goswim_landice'

# 'geosgcm_parasol'

# 'geosgcm_lidar'

# 'geosgcm_modis'

# 'geosgcm_parasol'

# 'geosgcm_radar'

# 'geosgcm_modis'

# 'geosgcm_isccp'

# 'geosgcm_radar'

# 'geosgcm_misr'

# 'geosgcm_isccp'

::

# 'geosgcm_misr'

::

The table follows MAPL's (and MPEU's) convention of starting with a name (COLLECTION) and editing with two colons (::). The particular example above would be found in a file with the extension "tmpl' to indicate it is a template file (meaning, it has entries in a format >>>ENTRY<<< which get replaced by something meaningful at runtime.

Users can control ~~the~~ freely the collections (or output streams) they want by simply commenting or uncommenting existing collections --- a "#" sign is a comment, which the reading software ignores.

Users can control freely the collections (or output streams) they want by simply commenting or uncommenting existing collections --- a "#" sign is a comment, which the reading software ignores. User can also create their own collections; existing ones can be used as examples for how to create new ones.

Each collection defines its own meta-information. For example, the output stream <<geosgcm_prog>> in the COLLECTION table above is associated with the following entries in the HISTORY resource file:

geosgcm_prog.template: '%y4%m2%d2_%h2%n2z.nc4',

geosgcm_prog.archive: '%c/Y%y4',

geosgcm_prog.format: 'CFIO',

geosgcm_prog.frequency: 060000,

geosgcm_prog.grid_label: PC@HIST_IMx@HIST_JM-DC

geosgcm_prog.vscale: 100.0,

geosgcm_prog.vunit: 'hPa',

geosgcm_prog.vvars: 'log(PLE)' , 'DYN' ,

geosgcm_prog.levels: 1000 975 950 925 900 875 850 825 800 775 750 725 700 650 600 550 500 450 400 350 300 250 200 150 100 70 50 40 30 20 10 7 5 4 3 2 1 0.7 0.5 0.4 0.3 0.2 0.1 0.07 0.05 0.04 0.03 0.02,

geosgcm_prog.fields: 'PHIS' , 'AGCM' ,

'SLP' , 'DYN' ,

'U;V' , 'DYN' ,

'T' , 'DYN' ,

'PS' , 'DYN' ,

'ZLE' , 'DYN' , 'H' ,

'OMEGA' , 'DYN' ,

'Q' , 'MOIST' , 'QV' ,

'QITOT' , 'AGCM' , 'QI' ,

'QLTOT' , 'AGCM' , 'QL' ,

'RH2' , 'MOIST' , 'RH' ,

'O3' , 'CHEMISTRY' ,

::

Combined with the stream name, the template entry defines the filename, that is, at 060000 UTC on 1 April 2010 a file will be written out with the name: geosgcm_prog.20100401_0600z.nc4; the archive entry specifies a directory where the file will land; the format makes it complaint with the CFIO standard in GEOS; the frequency specifies how often a particular collection will be written out; the vertical units are defined in pressure, as opposed to model index levels; the levels entry specifies desired levels for the model output, and the fields entry allows for specification of a list of fields to be written. The variable grid_label defines the resolution of the output: here the use of the symbol "@", as in @HIST_IM and @HIST_JM, means that these are variables to be defined at setup time (as opposed to runtime).

A particular menace in creating output from GEOS is sometimes overlooked. The simplicity of requesting output through HISTORY makes us forget that many times, they imply generation of massive amounts of data. Users using GEOS out-of-the-box, should give particular attention to this matter. Users should look into the default collections of GEOS GCM carefully, comment out collections carrying fields they do not expect to ever look at in their studies. This will avoid generation and, worse, storage of unnecessarily large amounts of data. Such considerations are even more fundamental for users running GEOS applications that involve GOCART and GEOSChem; the latter one, in particular, can carry hundreds of chemical constituents and lead to the request of a lot of data never to be looked at. Users running the GMAO Data Assimilation System (DAS) should also exercise similar caution, particularly, those users configuring the DAS to the configuration of one of GMAO's reanalyses. Commenting out a COLLECTION involves a single line change, please exercise caution when requesting output from GEOS GCM.

~~Each collection defines its own meta~~-information.

Considerably more detail about GEOS GCM output streams and conventions, specifically related to the MERRA-2, can be found under: https://gmao.gsfc.nasa.gov/pubs/docs/Bosilovich785.pdf. A host of extra information on GEOS file specs, in particular, can also be found under the Documentation section in the GMAO Products page: https://gmao.gsfc.nasa.gov/GMAO_products/NRT_products.php

@@ Line 1: / Line 1: @@
-This section provides a brief overview of the GEOS GCM output control. At its highest level, model output is controlled through a resource file named HISTORY.rc; many times, this file comes in templated format, which means it contains entries that are filled in at runtime, in such cases, the files is typically found under the name HISTORY.rc.tmpl.
+This section provides a brief overview of the GEOS GCM output control. At its highest level, model output is controlled through a resource file named HISTORY.rc; many times, this file comes in templated format, which means it contains entries that are filled in at runtime, in such cases, the file is typically found with an extension "tmpl" appended to its name.
 The output of GEOS GCM is subset in files, referred to as COLLECTIONS. An example of a COLLECTION table is provided below:
-COLLECTIONS: 'geosgcm_prog'
-             'geosgcm_surf'
+ COLLECTIONS: 'geosgcm_prog'
-             'geosgcm_moist'
+              'geosgcm_surf'
-             'geosgcm_turb'
+              'geosgcm_moist'
-             'geosgcm_gwd'
+              'geosgcm_turb'
-             'geosgcm_tend'
+              'geosgcm_gwd'
-             'geosgcm_budi'
+              'geosgcm_tend'
-             'geosgcm_buda'
+              'geosgcm_budi'
-             'geosgcm_landice'
+              'geosgcm_buda'
-             'geosgcm_meltwtr'
+              'geosgcm_landice'
-             'geosgcm_snowlayer'
+              'geosgcm_meltwtr'
-             'geosgcm_tracer'
+              'geosgcm_snowlayer'
->>>HIST_GOCART<<<             'tavg2d_aer_x'
+              'geosgcm_tracer'
->>>HIST_GOCART<<<             'tavg3d_aer_p'
+ >>>HIST_GOCART<<<     'tavg2d_aer_x'
-#             'geosgcm_iau'
+ >>>HIST_GOCART<<<     'tavg3d_aer_p'
-#             'geosgcm_conv'
+ #             'geosgcm_iau'
-#             'goswim_catch'
+ #             'geosgcm_conv'
-#             'goswim_land'
+ #             'goswim_catch'
-#             'goswim_landice'
+ #             'goswim_land'
-#             'geosgcm_lidar'
+ #             'goswim_landice'
-#             'geosgcm_parasol'
+ #             'geosgcm_lidar'
-#             'geosgcm_modis'
+ #             'geosgcm_parasol'
-#             'geosgcm_radar'
+ #             'geosgcm_modis'
-#             'geosgcm_isccp'
+ #             'geosgcm_radar'
-#             'geosgcm_misr'
+ #             'geosgcm_isccp'
-::
+ #             'geosgcm_misr'
+              ::
 The table follows MAPL's (and MPEU's) convention of starting with a name (COLLECTION) and editing with two colons (::). The particular example above would be found in a file with the extension "tmpl' to indicate it is a template file (meaning, it has entries in a format >>>ENTRY<<< which get replaced by something meaningful at runtime.
-Users can control the freely the collections (or output streams) they want by simply commenting or uncommenting existing collections --- a "#" sign is a comment, which the reading software ignores.
+Users can control freely the collections (or output streams) they want by simply commenting or uncommenting existing collections --- a "#" sign is a comment, which the reading software ignores. User can also create their own collections; existing ones can be used as examples for how to create new ones.
+Each collection defines its own meta-information. For example, the output stream <<geosgcm_prog>> in the COLLECTION table above is associated with the following entries in the HISTORY resource file:
+  geosgcm_prog.template:  '%y4%m2%d2_%h2%n2z.nc4',
+  geosgcm_prog.archive:   '%c/Y%y4',
+  geosgcm_prog.format:    'CFIO',
+  geosgcm_prog.frequency:  060000,
+  geosgcm_prog.grid_label: PC@HIST_IMx@HIST_JM-DC
+  geosgcm_prog.vscale:     100.0,
+  geosgcm_prog.vunit:     'hPa',
+  geosgcm_prog.vvars:     'log(PLE)' , 'DYN'          ,
+  geosgcm_prog.levels:     1000 975 950 925 900 875 850 825 800 775 750 725 700 650 600 550 500 450 400 350 300 250 200 150 100 70 50 40 30 20 10 7 5 4 3 2 1 0.7 0.5 0.4 0.3 0.2 0.1 0.07 0.05 0.04 0.03 0.02,
+  geosgcm_prog.fields:    'PHIS'     , 'AGCM'         ,
+                          'SLP'      , 'DYN'          ,
+                          'U;V'      , 'DYN'          ,
+                          'T'        , 'DYN'          ,
+                          'PS'       , 'DYN'          ,
+                          'ZLE'      , 'DYN'          , 'H'   ,
+                          'OMEGA'    , 'DYN'          ,
+                          'Q'        , 'MOIST'        , 'QV'  ,
+                          'QITOT'    , 'AGCM'         , 'QI'  ,
+                          'QLTOT'    , 'AGCM'         , 'QL'  ,
+                          'RH2'      , 'MOIST'        , 'RH'  ,
+                          'O3'       , 'CHEMISTRY'    ,
+                          ::
+Combined with the stream name, the template entry defines the filename, that is, at 060000 UTC on 1 April 2010 a file will be written out with the name: geosgcm_prog.20100401_0600z.nc4; the archive entry specifies a directory where the file will land; the format makes it complaint with the CFIO standard in GEOS; the frequency specifies how often a particular collection will be written out; the vertical units are defined in pressure, as opposed to model index levels; the levels entry specifies desired levels for the model output, and the fields entry allows for specification of a list of fields to be written. The variable grid_label defines the resolution of the output: here the use of the symbol "@", as in @HIST_IM and @HIST_JM, means that these are variables to be defined at setup time (as opposed to runtime).
+A particular menace in creating output from GEOS is sometimes overlooked. The simplicity of requesting output through HISTORY makes us forget that many times, they imply generation of massive amounts of data. Users using GEOS out-of-the-box, should give particular attention to this matter. Users should look into the default collections of GEOS GCM carefully, comment out collections carrying fields they do not expect to ever look at in their studies. This will avoid generation and, worse, storage of unnecessarily large amounts of data. Such considerations are even more fundamental for users running GEOS applications that involve GOCART and GEOSChem; the latter one, in particular, can carry hundreds of chemical constituents and lead to the request of a lot of data never to be looked at. Users running the GMAO Data Assimilation System (DAS) should also exercise similar caution, particularly, those users configuring the DAS to the configuration of one of GMAO's reanalyses. Commenting out a COLLECTION involves a single line change, please exercise caution when requesting output from GEOS GCM.
-Each collection defines its own meta-information.
+Considerably more detail about GEOS GCM output streams and conventions, specifically related to the MERRA-2, can be found under: https://gmao.gsfc.nasa.gov/pubs/docs/Bosilovich785.pdf. A host of extra information on GEOS file specs, in particular, can also be found under the Documentation section in the GMAO Products page: https://gmao.gsfc.nasa.gov/GMAO_products/NRT_products.php