Using the ExtData component: Difference between revisions

Line 63:

The most complex case is if you enter '0' for the refresh template. In this case the field gets updated every step and ExtData will try to interpolate to that time using the data on file and can interpolate between data points in different files. In order to accomplish this if you select a refresh template of '0' ExtData tries to find two times that bracket the start time the first time ExtData runs. These get updated as time advances in your application. But how does it know what time to apply to the file template if the data is spread across multiple files? This is where the idea of a file frequency and reference time comes in. The user can specify a file frequency and a reference time either with the 3 optional key words to the primary export or by letting ExtData "guess" from the file template. The idea is that starting from a reference time and the given frequency ExtData tries to find the bracketing data on the file with the time closest to the current time. If it does not find the data there it checks the next time. The user can specify a reference time and the frequency. If these are not given it tries to guess this from the file template. For example if the last token in the template is an hour then the frequency is an hour and the reference time is whatever the current data is with anything after the hour set to zero. If the file template is myfile_%y4%m2%d2_%h2.nc4 and ExtData will assume the frequency is 1 hour and the reference time is the top of the hour on whatever date you start the application. The following picture illustrates another example.

[[File:Bracketing_data.png]]

Finally suppose you had data that was arrange in files like below with one time per file:

d591_fpit.tavg3_3d_nav_Ne.20130701_0130z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_0430z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_0730z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_1030z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_1330z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_1630z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_1930z.nc4

d591_fpit.tavg3_3d_nav_Ne.20130701_2230z.nc4

and so on . . .

If the variable PLE was contained in the files and you wanted to do continuous time interpolation you would add line like this to ExtData.rc:

PLE NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00 hours 3

=== Masks ===

=== Derived Exports ===

The user can also specify derived exports. The user specifies an import to satisfy, an expression, and a refresh template. When the import is updated it is updated with the result of the expression. The expression is evaluated user the MAPL parser component and can involve any fields in the primary export section. Although it is currently not checked, the refresh template on the derived export should be the same as the primary exports in the expression. Also, the primary exports in the expression need not actually be needed to fulfill an import but can still be used in a derived expression.

@@ Line 63: / Line 63: @@
 The most complex case is if you enter '0' for the refresh template. In this case the field gets updated every step and ExtData will try to interpolate to that time using the data on file and can interpolate between data points in different files. In order to accomplish this if you select a refresh template of '0' ExtData tries to find two times that bracket the start time the first time ExtData runs. These get updated as time advances in your application. But how does it know what time to apply to the file template if the data is spread across multiple files? This is where the idea of a file frequency and reference time comes in. The user can specify a file frequency and a reference time either with the 3 optional key words to the primary export or by letting ExtData "guess" from the file template. The idea is that starting from a reference time and the given frequency ExtData tries to find the bracketing data on the file with the time closest to the current time. If it does not find the data there it checks the next time. The user can specify a reference time and the frequency. If these are not given it tries to guess this from the file template. For example if the last token in the template is an hour then the frequency is an hour and the reference time is whatever the current data is with anything after the hour set to zero. If the file template is myfile_%y4%m2%d2_%h2.nc4 and ExtData will assume the frequency is 1 hour and the reference time is the top of the hour on whatever date you start the application. The following picture illustrates another example.
 [[File:Bracketing_data.png]]
+Finally suppose you had data that was arrange in files like below with one time per file:
+d591_fpit.tavg3_3d_nav_Ne.20130701_0130z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_0430z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_0730z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_1030z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_1330z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_1630z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_1930z.nc4
+d591_fpit.tavg3_3d_nav_Ne.20130701_2230z.nc4
+and so on . . .
+If the variable PLE was contained in the files and you wanted to do continuous time interpolation you would add line like this to ExtData.rc:
+PLE  NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00 hours 3
 === Masks ===
 === Derived Exports ===
 The user can also specify derived exports. The user specifies an import to satisfy, an expression, and a refresh template. When the import is updated it is updated with the result of the expression. The expression is evaluated user the MAPL parser component and can involve any fields in the primary export section. Although it is currently not checked, the refresh template on the derived export should be the same as the primary exports in the expression. Also, the primary exports in the expression need not actually be needed to fulfill an import but can still be used in a derived expression.