MERRA2 Data Lessons for next REANL

This is just a scratch place to put notes on obs as they are found in MERRA2.

• AMSU-A N19. The date of death in active channels database is 20091222 000000 for Channel 8. This isn't proper, however, as the channel is seemingly dead from the get-go of the Jan10 stream. Eventually, a true death date will need to be determined.

• RapidScat will come in at some point. Initial dev was w/ QuikSCAT slot but no code changes. NESDIS (Sid's team) passed developments to use the OSCAT slot. If OSCAT is considered in the future, both instruments will likely share the same slot. From my end (wm), the largest development was the implementation of a thinning mesh similar to ASCAT for RapidScat. This is probably ideal for QuikSCAT in the future no matter. I do not know offhand what thinning, if any, OSCAT used. It may also be worth considering thinning for ERS instruments.

• Aircraft observations. Prior to 20130731/00z our reanalysis data uses a table incompatible with the new NRL aircraft quality control. There was also a problem in the way the height (derived from pressure) was calculated in the earlier MDCRS obs that would be flagged by NRLQC. One more issue is that our MDCRS data is being written with +/-3 hour window and the QC would benefit from having observations beyond these limits (just as Profiler and AMDAR windows are larger). For the next reanalysis the aircraft observations should be replaced, preferably processed with a recent version of PREPDATA so that TAMDAR observations get tail number IDs. (Even if we don't use them we may want to look at them.) Alternatively we could try to rewrite the data we have with corrected ACARS heights, updated BUFR table, and possibly wider time windowing.
  • Code and a namelist switch 'ACARSH' was added to "gmao_prevents.x" to allow rewriting the incorrect geopotential height values that had been written in MERRA ACARS observations prior to the WCOSS update
  • Modifications were added to the NRL aircraft quality control "prepacqc.f" so that the new profile aircraft observations can be written in the new BUFR format.
  • These two changes allow using the NRL aircraft quality control with our current files to produce aircraft profile datasets to be used with the NCEP variational aircraft temperature bias correction. There would still be some benefit from getting reprocessed aircraft data to obtain a larger time window and to properly identify the TAMDAR data with non-missing tail numbers.

• A shift in RAOB T O-F bias @ 700 hPa was seen to occur in Nov 1991 and seemingly recover in later years. The only real signal to correlate was SSMI Wind Speeds over that month, which showed a reduction in O-F bias. Otherwise, no real obvious signal was seen, but further investigation is necessary. It may be worthwhile to consider this transition as a period of testing in the next reanalysis.
  • Media:Large.300_700_raobt_o-f_m2_vs_merra.png
  • Media:Large.300_700_raobt_o-f_m2_vs_merra.1991-1992.png
  • Media:Large.300_700_raobt_o-f_m2_vs_merra.2004-2008.png

• An issue was seen in the OSTIA SST field for Feb-Mar 2016. Refer to Gary's monitoring slides from 15 Apr 2016.
  • Media:gp_15apr2016-slide4.png

• Drifting buoys with WMO IDs 48513 and 48517 have been added to the MERRA2 and FP/FPIT blacklists to stop assimilating temperature since the observations appear to be bad. This blacklist has not been checked in to the repository since the buoy IDs were used in earlier periods and we're not sure we want to blacklist these for all time. We need to work on blacklisting (possibly time dependent) and/or improved quality control of buoy data for the next reanalysis.

• We will need to give some attention to radiosonde radiation correction and the Haimberger homogenization database for the next reanalysis. Our current Haimberger data runs through 2005 (I think) and there have been new radiosonde types introduced e.g. in the US since then.

• Metop-B AMSU-A/Channel 15 fix in FP system from NCEP (see comments in Talk:GEOS-5_ADAS_Radiance_Monitoring) may have undesirable affects on MSU, AMSU-A, HIRS, AMSU-B, MHS. Check changes, and be sure that we are not having adverse effects with this.

• AIRS/AQUA data count in MERRA-2 was noisy from 19 Jan 2017. Solution: The pre-processing airs job that pulls data via HTTP had been producing datasets that were incomplete, probably because it was set to run with a latency that was too short. The input data was re-processed and Radiance Monitoring plots were re-run to make sure that the obs-count look correct.

• MSG Meteosat-8 winds in NCEP prepbufr start 2/2007. EUMETSAT has reprocessed MSG winds (CF-015 Release 1) starting 2004/03/01 that could be considered for use in next reanalysis.

• When preparing for MERRA2 we used the radiosonde data that had been processed for MERRA, instead of using the observations from the newer prepbufr files that were provided by Jack Woollen. The MERRA raobs had Chris Redder's correction for the US Vaisala radiation correction problem, as well as corrections to the observation times for US sondes from data obtained by NCDC. (The more accurate obs times make the uncorrect-recorrect process more accurate.) There were some problems with the MERRA sonde data (e.g. missing surface pressure observations) that were corrected in MERRA2. It would probably be a good idea to apply the Vaisala correction to the newer raob data, so that can be used for the next reanalysis.

• The Cheng-Zhi Zou recalibrated radiances show AMSU-A NOAA-17 data available from 28 Jun - 15 Jul 2002. The satinfo is set up to not assimilate data to 15 Jul, and I'm not sure when the N17 data shows up in our L1A files. Whether we go forward with recalibrated radiances or not, it may be worth attempting to recover this 2-3 weeks of data if it is stable. See data start date in Zou and Wang (2011)

• GOES-15 Sounder has short periods with large O-F standard deviation in fall and spring near the equinox. We should check to see whether the issue is significant enough to set the instrument (or certain channels) to passive during these periods in the satinfo for future historical/reanalysis experiments. It also has problems with differences in bias values in a few channels at different times of day, not well characterized by the current radiance bias correction.

• AQUA-AMSUA recalibrated data have stable data available from May 24th 2002. However, the files seem to be stable from the end of August 2002.

• Ricardo said something at a recent GSI meeting about time-dependent rawinsonde observation errors. Another approach (maybe more appropriate) would be observation errors dependent on rawinsonde instrument type - providing a time-varying error with a physical basis. X. Su at NCEP investigated this and had a Trak ticket #628 with some results of an investigation. This could be extended further for the historic instrument types for reanalysis. Certainly other data types such as the satellite winds (original or reprocessed) should be considered for some kind of time-dependent observation errors.

• Another ticket in NCEP Trak involves aircraft data thinning which may be something they are implementing. Any instance of thinning should also be considered with respect to the time-varying aspects of observation spacing.

• During 2000/2001 there was a period of 4 months when unthinned QuikScat data were used in MERRA2. The QuikScat data volume went up from 50,000 to 400,000. Need to keep an eye on that period. Actually during the period we accidentally included both our own processed QuikScat and QuikScat from NCEP in the prepbufr file - there should be no problem in excluding the data from NCEP when/if we create new prepbufr files for this period.