Using the ExtData component: Difference between revisions

Line 57:

==== Refresh Template ====

The refresh template allows quite a bit of flexibility in how the data can be distributed in the files and when the data is updated. [[In all cases, ExtData will try to time interpolate to the current model time or the time evaluated by applying the current application time to the ~~fresh~~ template if it is a grads style entry with tokens]].

The refresh template allows quite a bit of flexibility in how the data can be distributed in the files and when the data is updated. [[In all cases, ExtData will try to time interpolate to the current model time if it is 0 or the time evaluated by applying the current application time to the refresh template if it is a grads style entry with tokens]].

If you enter '-' the field will be updated only the first time ExtData runs. It will take the current model time and try to interpolate to that time using the available data.

Line 68:

~~These get~~ updated as time advances in your application. But how does it know ~~what time~~ to ~~apply to the file template~~ if the data is spread across multiple files? ~~This is where~~ the ~~idea of a file frequency and reference~~ time ~~comes~~ in. ~~The user can~~ specify a ~~file frequency and a reference time either with the 3 optional key words to the primary export or by letting ExtData "guess" from the~~ file template. ~~The idea is that starting from a reference time and the given frequency~~ ExtData tries to find the bracketing ~~data on the file with~~ the time ~~closest~~ to the current time. If it does not find the ~~data there~~ it ~~checks~~ the next ~~time~~. ~~The~~ user ~~can~~ specify a reference time and the frequency~~. If these are not given it tries to guess this from~~ the ~~file template~~. For example if the ~~last~~ token in the ~~template is an hour then~~ the ~~frequency is an hour~~ and the reference time ~~is whatever~~ the ~~current data is with anything~~ after ~~the hour set to zero. If~~ the file template ~~is myfile_%y4%m2%d2_%h2.nc4 and ExtData will assume~~ the ~~frequency is 1 hour~~ and the ~~reference~~ time is the ~~top of the hour on whatever date you start the application~~. The following picture illustrates ~~another~~ example.

This bracketing data gets updated as time advances in your application. But how does it know where to find this, especially if the data is spread across multiple files? If your file_template has no grads tokens to evaluate then all data you need to span the time span your application will run on needs to be in the file. More likely if you have multiple files you specify a grads style file template. When ExtData tries to find the bracketing times, it takes the time it is trying to interpolate to and applies that to the file template. It looks for data at two times that bracket the current time. If it does not find either the left or right bracket time it looks in the previous file (for the left) or the next file (for the right). How does it know what the next file is. If the user does not specify a reference time and frequency it looks at the right most token and assumes that is the frequency of the files. For example if the right most token is %d2 it assumes that files are daily so to find the right bracket time, it will advance the time by one day and try to look in that file to update the right bracket time. Now what if you have a file every 3 hours for example, this logic would not work. In this case you must specify a reference time and frequency using the optional keyword after the file template to tell it explicitly when the files start and how to advance the time to find the bracketing data.

The following picture illustrates a simple example of how this works.

[[File:Bracketing_data.png]]

Finally suppose you had data that was ~~arrange~~ in files like below with one time per file:

Finally suppose you had data that was arranged in files like below with one time per file:

d591_fpit.tavg3_3d_nav_Ne.20130701_0130z.nc4

Line 92:

Line 94:

and so on . . .

If the variable PLE was contained in the files and you wanted to do continuous time interpolation you would add line like this to ExtData.rc:

If the variable PLE was contained in the files and you wanted to do continuous time interpolation of PLE you would add line like this to ExtData.rc:

PLE NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00 ~~hours 3~~

PLE NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00P03:00

=== Special Options ===

@@ Line 57: / Line 57: @@
 ==== Refresh Template ====
-The refresh template allows quite a bit of flexibility in how the data can be distributed in the files and when the data is updated. [[In all cases, ExtData will try to time interpolate to the current model time or the time evaluated by applying the current application time to the fresh template if it is a grads style entry with tokens]].
+The refresh template allows quite a bit of flexibility in how the data can be distributed in the files and when the data is updated. [[In all cases, ExtData will try to time interpolate to the current model time if it is 0 or the time evaluated by applying the current application time to the refresh template if it is a grads style entry with tokens]].
 If you enter '-' the field will be updated only the first time ExtData runs. It will take the current model time and try to interpolate to that time using the available data.
@@ Line 68: / Line 68: @@
-These get updated as time advances in your application. But how does it know what time to apply to the file template if the data is spread across multiple files? This is where the idea of a file frequency and reference time comes in. The user can specify a file frequency and a reference time either with the 3 optional key words to the primary export or by letting ExtData "guess" from the file template. The idea is that starting from a reference time and the given frequency ExtData tries to find the bracketing data on the file with the time closest to the current time. If it does not find the data there it checks the next time. The user can specify a reference time and the frequency. If these are not given it tries to guess this from the file template. For example if the last token in the template is an hour then the frequency is an hour and the reference time is whatever the current data is with anything after the hour set to zero. If the file template is myfile_%y4%m2%d2_%h2.nc4 and ExtData will assume the frequency is 1 hour and the reference time is the top of the hour on whatever date you start the application. The following picture illustrates another example.
+This bracketing data gets updated as time advances in your application. But how does it know where to find this, especially if the data is spread across multiple files? If your file_template has no grads tokens to evaluate then all data you need to span the time span your application will run on needs to be in the file. More likely if you have multiple files you specify a grads style file template. When ExtData tries to find the bracketing times, it takes the time it is trying to interpolate to and applies that to the file template. It looks for data at two times that bracket the current time. If it does not find either the left or right bracket time it looks in the previous file (for the left) or the next file (for the right). How does it know what the next file is. If the user does not specify a reference time and frequency it looks at the right most token and assumes that is the frequency of the files. For example if the right most token is %d2 it assumes that files are daily so to find the right bracket time, it will advance the time by one day and try to look in that file to update the right bracket time. Now what if you have a file every 3 hours for example, this logic would not work. In this case you must specify a reference time and frequency using the optional keyword after the file template to tell it explicitly when the files start and how to advance the time to find the bracketing data.
+ The following picture illustrates a simple example of how this works.
 [[File:Bracketing_data.png]]
-Finally suppose you had data that was arrange in files like below with one time per file:
+Finally suppose you had data that was arranged in files like below with one time per file:
 d591_fpit.tavg3_3d_nav_Ne.20130701_0130z.nc4
@@ Line 92: / Line 94: @@
 and so on . . .
-If the variable PLE was contained in the files and you wanted to do continuous time interpolation you would add line like this to ExtData.rc:
+If the variable PLE was contained in the files and you wanted to do continuous time interpolation of PLE you would add line like this to ExtData.rc:
-PLE  NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00 hours 3
+PLE  NA xyz e N N 0 none none d591_fpit.tavg3_3d_nav_Ne.%y4%m2%d2_%h2%n2z.nc4 2013-07-01T01:30:00P03:00
 === Special Options ===