Using the ExtData component: Difference between revisions

Line 55:

=== Primary Exports - Heracles-5_4 on ===

The first table the user can define is a PrimaryExports table. Below is an example of a PrimaryExport table. This may have as many entries as the user desires. Note that only the last entry in a line is optional.

<pre>

PrimaryExports%%

# ---------|---------|------|------------ |----------------------|--------|-------|------------------------------------|-----------------------------------------------------------|

# ---------|---------|--------------------|----------------------|--------|-------|----------|-------------------------|-----------------------------------------------------------|

ALBNF NA N N 0 0.0 1.0 ALBNF myfile.%y4%m2%d2_%h2z.nc4 2000-04-15T00:00:00P03:00

du001 NA N N 0 0.0 1.0 du001 /dev/null

%%

</pre>

The following is an explanation of each entry in a line.

:Export Name - This is the actual name of the import in the application this entry will satisfy.

:Units - Units not currently used, just a placeholder for now, hopefully someday it might be used for automatic unit conversion for example.

:Dimensions - xy for 2D or xyz for 3D

:Vertical location - c for center, e for edge, if 2d enter c or e but this will obviously not be used

:Clim - enter Y if the file is a 12 month climatology, if you specify it is a climatology ExtData the data can be on either one file or 12 files if they are templated appropriately with one per month. Enter N if the data is not a climatological data set. Enter a year (2008 for example) if you have a years worth of data you want to use as a climatology, but is not 12 months, for example you had multiple years worth of data but wanted to use 1 particular year worth as a climatology

:Conservative - Enter Y the data should be regridded in a mass conserving fashion through a tiling file. Enter N to use the non-conervative bilinear regridding. Enter V to regrid using the conservative voting option. Enter F;integer to regrid using the fractional conservative option.

:Refresh template - you have 3 choices

::1.) Enter '-'. In this case the field will only be updated once the first time ExtData runs

::2.) Enter a refresh template of the form %y4-%m2-%h2T%h2:%n2:00 to set the recurring time to update the file. The file will be updated when the evaluated template changes. For example a template of the form %y4-%m2-%d2T12:00:00 will cause the variable to be updated at the start of a new day. Note that ExtData will use the evaluated template as the working time for reading the file and will try to interpolate to that time. So in the example of %y4-%m2-%d2T12:00:00, when the clock hits 2007-08-02T00:00:00 it will update the variable but the time it will use for reading and interpolation is 2007-08-02T12:00:00

::3.) Enter '0' to update the variable at every step. ExtData will do a linear interpolation to the current time using the available data.

::4.) Enter Phr:mn where hr is a two digit hour and mn is a two digit minute or enter Pyear-mm-ddThr:mn where now year, mm, and dd are a year, month and day, this is an interval at which to update the variable from the start time of the clock used in the program.

:Offset - This is a factor the variable will be shifted by, if you enter "none", no shifting will be performed, [[If you do not want to shift do not put 0.0 as you will be wasting time since it will multiply by 0.0 instead of skipping the shifting]].

:Scale - This is a factor the variable will be scaled by, if you enter "none", no scaling will be performed, [[If you do not want to scale do not put 1.0 as you will be wasting time since it will multiply by 1.0 instead of skipping the scaling]].

:Variable - This is the name of the variable ON THE FILE. It need not be the same as the export name.

:File Template - this is a grads style template describing the time structure of your data if it is broken into multiple files. If there are no tokens in the template name ExtData will assume that all the data is on one file. Note that if the data on file is at a different resolution that the application grid, the underlying I/O library ExtData uses will regrid the data to the application grid. The user can enter /dev/null to simply fill the import with zero or enter /dev/null:300.0 to set the import to a non-zero constant value.

:File reference time and frequency - this keyword is optional. Also note if your data is on one file then it makes there is no point to this keyword. This entry is the time and time-interval that describes the start time and frequency of the file template you provided and has the form %y4-%m2-%d2T%h2%n2P%y4-%m2-%d2T%h2%n2 where the time before the P is a reference time and the time after the P is a time interval. Note that the year, month, and day can be left off as a unit in the time interval and it will assume that these are zero. This keyword says that the first time that the file template is good for is the reference time and there will be files using the supplied file template at the interval provided. This provides a direct way to specify if ExtData can not determine the file frequency from the template. For example if you have data every half hour to ingest, this can not be "guessed" from a file template that has a last token of a minute or you had files every 3 hours. For example, a valid entry would be 2012-01-01T21:00P03:00. This says you have a file every 3 hours from starting at 21z on 01/01/2012.

==== Refresh Template ====

@@ Line 55: / Line 55: @@
+=== Primary Exports - Heracles-5_4 on ===
+The first table the user can define is a PrimaryExports table. Below is an example of a PrimaryExport table. This may have as many entries as the user desires. Note that only the last entry in a line is optional.
+<pre>
+PrimaryExports%%
+# ---------|---------|------|------------ |----------------------|--------|-------|------------------------------------|-----------------------------------------------------------|
+#  Export  |         |      |             |_______ Refresh ______|____ Factors ___|________ External File _____________|______________________External File Time Data______________|
+#  Name    | Units   | Clim |Conservative |     Time Template    | Offset | Scale | Variable |      Template           |  Reference Time and frequency                             |
+# ---------|---------|--------------------|----------------------|--------|-------|----------|-------------------------|-----------------------------------------------------------|
+ALBNF           NA       N        N               0                0.0      1.0     ALBNF     myfile.%y4%m2%d2_%h2z.nc4   2000-04-15T00:00:00P03:00
+du001           NA       N        N               0                0.0      1.0     du001     /dev/null
+%%
+</pre>
+The following is an explanation of each entry in a line.
+:Export Name - This is the actual name of the import in the application this entry will satisfy.
+:Units - Units not currently used, just a placeholder for now, hopefully someday it might be used for automatic unit conversion for example.
+:Dimensions - xy for 2D or xyz for 3D
+:Vertical location - c for center, e for edge, if 2d enter c or e but this will obviously not be used
+:Clim - enter Y if the file is a 12 month climatology, if you specify it is a climatology ExtData the data can be on either one file or 12 files if they are templated appropriately with one per month. Enter N if the data is not a climatological data set. Enter a year (2008 for example) if you have a years worth of data you want to use as a climatology, but is not 12 months, for example you had multiple years worth of data but wanted to use 1 particular year worth as a climatology
+:Conservative - Enter Y the data should be regridded in a mass conserving fashion through a tiling file. Enter N to use the non-conervative bilinear regridding. Enter V to regrid using the conservative voting option. Enter F;integer to regrid using the fractional conservative option.
+:Refresh template - you have 3 choices
+::1.) Enter '-'. In this case the field will only be updated once the first time ExtData runs
+::2.) Enter a refresh template of the form %y4-%m2-%h2T%h2:%n2:00 to set the recurring time to update the file. The file will be updated when the evaluated template changes. For example a template of the form %y4-%m2-%d2T12:00:00 will cause the variable to be updated at the start of a new day. Note that ExtData will use the evaluated template as the working time for reading the file and will try to interpolate to that time. So in the example of %y4-%m2-%d2T12:00:00, when the clock hits 2007-08-02T00:00:00 it will update the variable but the time it will use for reading and interpolation is 2007-08-02T12:00:00
+::3.) Enter '0' to update the variable at every step. ExtData will do a linear interpolation to the current time using the available data.
+::4.) Enter Phr:mn where hr is a two digit hour and mn is a two digit minute or enter Pyear-mm-ddThr:mn where now year, mm, and dd are a year, month and day, this is an interval at which to update the variable from the start time of the clock used in the program.
+:Offset - This is a factor the variable will be shifted by, if you enter "none", no shifting will be performed, [[If you do not want to shift do not put 0.0 as you will be wasting time since it will multiply by 0.0 instead of skipping the shifting]].
+:Scale - This is a factor the variable will be scaled by, if you enter "none", no scaling will be performed, [[If you do not want to scale do not put 1.0 as you will be wasting time since it will multiply by 1.0 instead of skipping the scaling]].
+:Variable - This is the name of the variable ON THE FILE. It need not be the same as the export name.
+:File Template - this is a grads style template describing the time structure of your data if it is broken into multiple files. If there are no tokens in the template name ExtData will assume that all the data is on one file. Note that if the data on file is at a different resolution that the application grid, the underlying I/O library ExtData uses will regrid the data to the application grid. The user can enter /dev/null to simply fill the import with zero or enter /dev/null:300.0 to set the import to a non-zero constant value.
+:File reference time and frequency - this keyword is optional. Also note if your data is on one file then it makes there is no point to this keyword. This entry is the time and time-interval that describes the start time and frequency of the file template you provided and has the form %y4-%m2-%d2T%h2%n2P%y4-%m2-%d2T%h2%n2 where the time before the P is a reference time and the time after the P is a time interval. Note that the year, month, and day can be left off as a unit in the time interval and it will assume that these are zero. This keyword says that the first time that the file template is good for is the reference time and there will be files using the supplied file template at the interval provided. This provides a direct way to specify if ExtData can not determine the file frequency from the template. For example if you have data every half hour to ingest, this can not be "guessed" from a file template that has a last token of a minute or you had files every 3 hours. For example, a valid entry would be 2012-01-01T21:00P03:00. This says you have a file every 3 hours from starting at 21z on 01/01/2012.
 ==== Refresh Template ====