MERRA2 Data Lessons for next REANL: Difference between revisions

(6 intermediate revisions by 2 users not shown)

Line 6:

* Aircraft observations. Prior to 20130731/00z our reanalysis data uses a table incompatible with the new NRL aircraft quality control. There was also a problem in the way the height (derived from pressure) was calculated in the earlier MDCRS obs that would be flagged by NRLQC. One more issue is that our MDCRS data is being written with +/-3 hour window and the QC would benefit from having observations beyond these limits (just as Profiler and AMDAR windows are larger). For the next reanalysis the aircraft observations should be replaced, preferably processed with a recent version of PREPDATA so that TAMDAR observations get tail number IDs. (Even if we don't use them we may want to look at them.) Alternatively we could try to rewrite the data we have with corrected ACARS heights, updated BUFR table, and possibly wider time windowing.

** Code and a namelist switch 'ACARSH' was added to "gmao_prevents.x" to allow rewriting the incorrect geopotential height values that had been written in MERRA ACARS observations prior to the WCOSS update

** Modifications were added to the NRL aircraft quality control "prepacqc.f" so that the new profile aircraft observations can be written in the new BUFR format.

** These two changes allow using the NRL aircraft quality control with our current files to produce aircraft profile datasets to be used with the NCEP variational aircraft temperature bias correction. There would still be some benefit from getting reprocessed aircraft data to obtain a larger time window and to properly identify the TAMDAR data with non-missing tail numbers.

*A shift in RAOB T O-F bias @ 700 hPa was seen to occur in Nov 1991 and seemingly recover in later years. The only real signal to correlate was [https://gmao.gsfc.nasa.gov/intranet/science/gms/d5124_m2_jan91/TERRA_html/obs2html/Y1991/M11/conv/G48/d5124_m2_jan91.pseries.L1.1991110100-1991113018.png SSMI Wind Speeds] over that month, which showed a reduction in O-F bias. Otherwise, no real obvious signal was seen, but further investigation is necessary. It may be worthwhile to consider this transition as a period of testing in the next reanalysis.

Line 29:

Line 32:

* The Cheng-Zhi Zou recalibrated radiances show AMSU-A NOAA-17 data available from 28 Jun - 15 Jul 2002. The satinfo is set up to not assimilate data to 15 Jul, and I'm not sure when the N17 data shows up in our L1A files. Whether we go forward with recalibrated radiances or not, it may be worth attempting to recover this 2-3 weeks of data if it is stable. See data start date in Zou and Wang (2011)

* GOES-15 Sounder has short periods with large O-F standard deviation in fall and spring near the equinox. We should check to see whether the issue is significant enough to set the instrument (or certain channels) to passive during these periods in the satinfo for future historical/reanalysis experiments.

* GOES-15 Sounder has short periods with large O-F standard deviation in fall and spring near the equinox. We should check to see whether the issue is significant enough to set the instrument (or certain channels) to passive during these periods in the satinfo for future historical/reanalysis experiments. It also has problems with differences in bias values in a few channels at different times of day, not well characterized by the current radiance bias correction.

* AQUA-AMSUA recalibrated data have stable data available from May 24th 2002. However, the files seem to be stable from the end of August 2002.

* Ricardo said something at a recent GSI meeting about time-dependent rawinsonde observation errors. Another approach (maybe more appropriate) would be observation errors dependent on rawinsonde instrument type - providing a time-varying error with a physical basis. X. Su at NCEP investigated this and had a Trak ticket #628 with some results of an investigation. This could be extended further for the historic instrument types for reanalysis. Certainly other data types such as the satellite winds (original or reprocessed) should be considered for some kind of time-dependent observation errors.

* Another ticket in NCEP Trak involves aircraft data thinning which may be something they are implementing. Any instance of thinning should also be considered with respect to the time-varying aspects of observation spacing.

* During 2000/2001 there was a period of 4 months when unthinned QuikScat data were used in MERRA2. The QuikScat data volume went up from 50,000 to 400,000. Need to keep an eye on that period. Actually during the period we accidentally included both our own processed QuikScat and QuikScat from NCEP in the prepbufr file - there should be no problem in excluding the data from NCEP when/if we create new prepbufr files for this period.

* between 2018212-20180221 some surface ship obs were being written with pob=0 and other data missing. These obs were in both the operational GDAS files and the MERRA2 stream. Reading this data leads to a number of warning messages written in the ana.log file but it should be harmless.

@@ Line 6: / Line 6: @@
 * Aircraft observations.  Prior to 20130731/00z our reanalysis data uses a table incompatible with the new NRL aircraft quality control.  There was also a problem in the way the height (derived from pressure) was calculated in the earlier MDCRS obs that would be flagged by NRLQC.  One more issue is that our MDCRS data is being written with +/-3 hour window and the QC would benefit from having observations beyond these limits (just as Profiler and AMDAR windows are larger).  For the next reanalysis the aircraft observations should be replaced, preferably processed with a recent version of PREPDATA so that TAMDAR observations get tail number IDs. (Even if we don't use them we may want to look at them.)  Alternatively we could try to rewrite the data we have with corrected ACARS heights, updated BUFR table, and possibly wider time windowing.
+** Code and a namelist switch 'ACARSH' was added to "gmao_prevents.x" to allow rewriting the incorrect geopotential height values that had been written in MERRA ACARS observations prior to the WCOSS update
+** Modifications were added to the NRL aircraft quality control "prepacqc.f" so that the new profile aircraft observations can be written in the new BUFR format.
+** These two changes allow using the NRL aircraft quality control with our current files to produce aircraft profile datasets to be used with the NCEP variational aircraft temperature bias correction.  There would still be some benefit from getting reprocessed aircraft data to obtain a larger time window and to properly identify the TAMDAR data with non-missing tail numbers.
 *A shift in RAOB T O-F bias @ 700 hPa was seen to occur in Nov 1991 and seemingly recover in later years.  The only real signal to correlate was [https://gmao.gsfc.nasa.gov/intranet/science/gms/d5124_m2_jan91/TERRA_html/obs2html/Y1991/M11/conv/G48/d5124_m2_jan91.pseries.L1.1991110100-1991113018.png SSMI Wind Speeds] over that month, which showed a reduction in O-F bias.  Otherwise, no real obvious signal was seen, but further investigation is necessary.  It may be worthwhile to consider this transition as a period of testing in the next reanalysis.
@@ Line 29: / Line 32: @@
 * The Cheng-Zhi Zou recalibrated radiances show AMSU-A NOAA-17 data available from 28 Jun - 15 Jul 2002.  The satinfo is set up to not assimilate data to 15 Jul, and I'm not sure when the N17 data shows up in our L1A files.  Whether we go forward with recalibrated radiances or not, it may be worth attempting to recover this 2-3 weeks of data if it is stable.  See data start date in Zou and Wang (2011)
-* GOES-15 Sounder has short periods with large O-F standard deviation in fall and spring near the equinox.  We should check to see whether the issue is significant enough to set the instrument (or certain channels) to passive during these periods in the satinfo for future historical/reanalysis experiments.
+* GOES-15 Sounder has short periods with large O-F standard deviation in fall and spring near the equinox.  We should check to see whether the issue is significant enough to set the instrument (or certain channels) to passive during these periods in the satinfo for future historical/reanalysis experiments.  It also has problems with differences in bias values in a few channels at different times of day, not well characterized by the current radiance bias correction.
+* AQUA-AMSUA recalibrated data have stable data available from May 24th 2002. However, the files seem to be stable from the end of August 2002.
+* Ricardo said something at a recent GSI meeting about time-dependent rawinsonde observation errors.  Another approach (maybe more appropriate) would be observation errors dependent on rawinsonde instrument type - providing a time-varying error with a physical basis.  X. Su at NCEP investigated this and had a Trak ticket #628 with some results of an investigation.  This could be extended further for the historic instrument types for reanalysis.  Certainly other data types such as the satellite winds (original or reprocessed) should be considered for some kind of time-dependent observation errors.
+* Another ticket in NCEP Trak involves aircraft data thinning which may be something they are implementing.  Any instance of thinning should also be considered with respect to the time-varying aspects of observation spacing.
+* During 2000/2001 there was a period of 4 months when unthinned QuikScat data were used in MERRA2. The QuikScat data volume went up from 50,000 to 400,000. Need to keep an eye on that period.   Actually during the period we accidentally included both our own processed QuikScat and QuikScat from NCEP in the prepbufr file - there should be no problem in excluding the data from NCEP when/if we create new prepbufr files for this period.
+* between 2018212-20180221 some surface ship obs were being written with pob=0 and other data missing.  These obs were in both the operational GDAS files and the MERRA2 stream.  Reading this data leads to a number of warning messages written in the ana.log file but it should be harmless.