MERRA2 Preparations: Difference between revisions

(53 intermediate revisions by 4 users not shown)

Line 1:

This is a page to keep tabs on MERRA2 tasks that need to be performed.

==Files to be updated==

==GSI Updates==

* ~~gmao_global_satinfo~~.rc

===Files to be updated/In Progress===

* ~~gmao_global_scaninfo~~.rc

* GSI_GridComp.rc.tmpl (WM)

* ~~gmao_global_pcpinfo~~.rc

**Initial version available on discover:

* gsi.rc.tmpl

***/gpfsm/dnb31/wrmccart/merra2/gsi/GSI_GridComp.rc.tmpl.merra2

* GSI_GridComp.rc.tmpl

**Issues/Questions:

***airs/mhs .blk suffix & consistency w/ obsys.rc

***Which data stream for EOS/Aqua AMSU-A

***TMI rainrate bufr: separate ocean/land sources or combined file?

***SSMI rain rates - unsure if we assimilated all SSMI's in MERRA

***Ensure prepbufr configured correctly

***Needs file structure for AVHRR winds - will need to be consistent w/ obsys.rc

***Entries needed for MLS - T & O3?

***Entry (entries?) needed for TOMS?

***Not updated for cbkg != NONE

* gsi.rc.tmpl (WM)

**Initial version available on discover at:

***/gpfsm/dnb31/wrmccart/merra2/gsi/gsi.rc.tmpl.merra2

**Issues/Questions:

***All changes above w/ GSI_GridComp.rc.tmpl need to be consistent with and probably to this file

***Reorganized based on Obs type (like GSI_GridComp.rc.tmpl) to ensure dvals/dthins are consistent

***Ozone not organized - defer to Kris/Larry to tidy this up/make dthin&dvals consistent - '''DONE!'''

***Single FOV GOES Sounder only exists from 2007-onward. GOES-08,10 need only sndr_gXX entry, GOES-13,14,15 need only sndrDN_gXX (N=1,4) entries, GOES-11,12 need both

***GOES-14 included w/ numbers fudged from GOES-13 in case future implementation is desired

* all *.arc

* satbang/satbias restart

* obsys.rc (DM: $NOBACKUP/merra2/run/obsys.rc))

* ~~obsys~~.rc

**GPS Startdate need to be confirmed/adjusted. We have data that preceeds the obsys.rc start date

* sac.nl.tmpl

**MetOp-A observations need to be adjusted.

***IASI files start rougly Sept 2008, but the AMSUA/MHS/HIRS streams may start earlier

* update sat-db:

**ozone - as soon as TOMS data are moved to the proper place - edit obsys.rc (Kris)

*** edit ozinfo-db 'active.tbl' to facilitate a switch from SBUV to MLS O3 in 2004 (Kris) - '''DONE''', file sent to RT

**sat

*** GOES Sounders need to be consistent w/ end of 5x5 prep stream (''gXX_prep'' entries in gsi.rc) and 1x1 (''gXX'' entries)

* update conv-db (presently being considered to control conventional obs)-added to tag (JG), will be used as additional tool to stream obs (diverting METEOSAT * AMV's , ERS data for Andrew huricane etc) (DM)

===Files Updated/hopefully completed===

* gmao_global_satinfo.rc (WM) '''DONE'''

** Template for use w/ satinfo.db: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_satinfo.rc.merra2

* gmao_global_scaninfo.rc (WM) '''DONE'''

** Updated at: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_scaninfo.rc.merra2

* <s>gmao_global_pcpinfo.rc</s> - no update needed

* gmao_global_ozinfo.rc - '''DONE!'''

** Available on discover at: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_ozinfo.rc.merra2

* satbang/satbias restart (WM)'''DONE'''

** Available at:

***/gpfsm/dnb31/wrmccart/merra2/gsi/satbang_rst.merra2-scout

***/gpfsm/dnb31/wrmccart/merra2/gsi/satbias_rst.merra2-scout

**Should be acceptable for scout runs. Sources of error values/bias coefficients/etc [https://gmao.gsfc.nasa.gov/intranet/personnel/wmccarty/20130708-MERRA2-satfile-sources.xlsx available here].

* sac.nl.tmpl '''DONE'''

** Available at: /gpfsm/dnb31/wrmccart/merra2/gsi/sac.nl.tmpl.merra2

==Observations to be Acquired and/or Processed==

* SSMI Radiances v7 (YK processing data-in progress, DM-testing)

* SSMI v7 wind speed (processed YK, tested DM) '''DONE!'''

* ERS2 scat. (DM-testing) '''DONE!'''

* AVHRR AMVs

** <s>Issue at hand for recent (post-2010) data stream. Infrastructure developed, but base observations may not actually exist (observations used for development may be from an antiquated algorithm).</s>

*** <s> WM - Awaiting further guidance from D. Santek</s>

*** WM - Will proceed forward - working with Rob to determine best way to acquire data

*** bufr files (2010-Oct2013) DONE (DM)

** prepbufr AVHRR winds(1982-2010) (reprocessing files for 1982-1990 due change in input data, DONE (DM)

* Consolidated prepbufr for 79-97 - ncep2013 (DM)

* GPS - preCOSMIc era ? (DM) (from Jack Woollen:gps data starting in 2002 is from the CHAMP instrument, reprocessed by the Boulder folks. Reprocessed COSMIC starts in around 2006)

* IASI (2008-now) (DONE)

* ASCAT (2009-now) DONE

* GOES 1x1 (2007-now) (Done)

* (Question from Mike B.) VAD data prior 2001? (DM) getting data from Jack Woolen (Rob)

**getting new prepbufr data from J. Woolen. DONE

* AIRCRFT data prior cdas prepbufr (2001) can we get reprocessed data? (DM) yes, DONE

* Preprocessed TOMS data need to be moved to a proper place on archive (Kris & Rob)(DONE)

* Get and process OMI and MLS v2.2 for 2012 - present (Kris). The same for MLS Temperature (Larry C.?) DONE

* Set up automatic processing of new MLS T, MLS O3 and OMI data

* tcvitals files (filling gap for 2005 and get some preop 2004 data?) (DM) -Jack Woolen provided 2000-2006 tcvitals (Rob has access to aquired data) DONE , getting additional data for 1980-1999.

==Issues to address==

* Infrastructure to bring in "spun-up" satbias/satbang files. Specific issues: (WM)

** Restart satbias/satbang files created for scout. Question remaining is how will values be adjusted for full M2 (if at all):

*** satbang coefficients & tlapmean values - spin up from scout diags

*** satbias coefficients - substitute in from "spun-up" scout satbias_rst (some delta-time in the future)

* (Question from Austin) How much of the data currently used as 'passive' do we need? E.g. SBUV N16 has been 'passive' in FP for some time; should it be just turned off in MERRA-2? It's bad data

@@ Line 1: / Line 1: @@
 This is a page to keep tabs on MERRA2 tasks that need to be performed.
-==Files to be updated==
+==GSI Updates==
-* gmao_global_satinfo.rc
+===Files to be updated/In Progress===
-* gmao_global_scaninfo.rc
+* GSI_GridComp.rc.tmpl (WM)
-* gmao_global_pcpinfo.rc
+**Initial version available on discover:
-* gsi.rc.tmpl
+***/gpfsm/dnb31/wrmccart/merra2/gsi/GSI_GridComp.rc.tmpl.merra2
-* GSI_GridComp.rc.tmpl
+**Issues/Questions:
+***airs/mhs .blk suffix & consistency w/ obsys.rc
+***Which data stream for EOS/Aqua AMSU-A
+***TMI rainrate bufr: separate ocean/land sources or combined file?
+***SSMI rain rates - unsure if we assimilated all SSMI's in MERRA
+***Ensure prepbufr configured correctly
+***Needs file structure for AVHRR winds - will need to be consistent w/ obsys.rc
+***Entries needed for MLS - T & O3?
+***Entry (entries?) needed for TOMS?
+***Not updated for cbkg != NONE
+* gsi.rc.tmpl (WM)
+**Initial version available on discover at:
+***/gpfsm/dnb31/wrmccart/merra2/gsi/gsi.rc.tmpl.merra2
+**Issues/Questions:
+***All changes above w/ GSI_GridComp.rc.tmpl need to be consistent with and probably to this file
+***Reorganized based on Obs type (like GSI_GridComp.rc.tmpl) to ensure dvals/dthins are consistent
+***Ozone not organized - defer to Kris/Larry to tidy this up/make dthin&dvals consistent - '''DONE!'''
+***Single FOV GOES Sounder only exists from 2007-onward.  GOES-08,10 need only sndr_gXX entry, GOES-13,14,15 need only sndrDN_gXX (N=1,4) entries, GOES-11,12 need both
+***GOES-14 included w/ numbers fudged from GOES-13 in case future implementation is desired
 * all *.arc
-* satbang/satbias restart
+* obsys.rc (DM: $NOBACKUP/merra2/run/obsys.rc))
-* obsys.rc
+**GPS Startdate need to be confirmed/adjusted.  We have data that preceeds the obsys.rc start date
-* sac.nl.tmpl
+**MetOp-A observations need to be adjusted.
+***IASI files start rougly Sept 2008, but the AMSUA/MHS/HIRS streams may start earlier
+* update sat-db:
+**ozone - as soon as TOMS data are moved to the proper place - edit obsys.rc (Kris)
+*** edit ozinfo-db 'active.tbl' to facilitate a switch from SBUV to MLS O3 in 2004 (Kris) - '''DONE''', file sent to RT
+**sat
+*** GOES Sounders need to be consistent w/ end of 5x5 prep stream (''gXX_prep'' entries in gsi.rc) and 1x1 (''gXX'' entries)
+* update conv-db (presently being considered to control conventional obs)-added to tag (JG), will be used as additional tool to stream obs (diverting METEOSAT * AMV's , ERS data for Andrew huricane etc) (DM)
+===Files Updated/hopefully completed===
+* gmao_global_satinfo.rc (WM) '''DONE'''
+** Template for use w/ satinfo.db: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_satinfo.rc.merra2
+* gmao_global_scaninfo.rc (WM) '''DONE'''
+** Updated at: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_scaninfo.rc.merra2
+* <s>gmao_global_pcpinfo.rc</s> - no update needed
+* gmao_global_ozinfo.rc - '''DONE!'''
+** Available on discover at: /gpfsm/dnb31/wrmccart/merra2/gsi/gmao_global_ozinfo.rc.merra2
+* satbang/satbias restart (WM)'''DONE'''
+** Available at:
+***/gpfsm/dnb31/wrmccart/merra2/gsi/satbang_rst.merra2-scout
+***/gpfsm/dnb31/wrmccart/merra2/gsi/satbias_rst.merra2-scout
+**Should be acceptable for scout runs.  Sources of error values/bias coefficients/etc [https://gmao.gsfc.nasa.gov/intranet/personnel/wmccarty/20130708-MERRA2-satfile-sources.xlsx available here].
+* sac.nl.tmpl '''DONE'''
+** Available at: /gpfsm/dnb31/wrmccart/merra2/gsi/sac.nl.tmpl.merra2
+==Observations to be Acquired and/or Processed==
+* SSMI Radiances v7 (YK processing data-in progress, DM-testing)
+* SSMI v7 wind speed (processed YK, tested DM) '''DONE!'''
+* ERS2 scat. (DM-testing) '''DONE!'''
+* AVHRR AMVs
+** <s>Issue at hand for recent (post-2010) data stream.  Infrastructure developed, but base observations may not actually exist (observations used for development may be from an antiquated algorithm).</s>
+*** <s> WM - Awaiting further guidance from D. Santek</s>
+*** WM - Will proceed forward - working with Rob to determine best way to acquire data
+*** bufr files (2010-Oct2013) DONE (DM)
+** prepbufr AVHRR winds(1982-2010) (reprocessing files for 1982-1990 due change in input data, DONE (DM)
+* Consolidated prepbufr for 79-97 - ncep2013 (DM)
+* GPS - preCOSMIc era ? (DM) (from Jack Woollen:gps data starting in 2002 is from the CHAMP instrument, reprocessed by the Boulder folks. Reprocessed COSMIC starts in around 2006)
+* IASI (2008-now) (DONE)
+* ASCAT (2009-now) DONE
+* GOES 1x1 (2007-now) (Done)
+* (Question from Mike B.) VAD data prior 2001? (DM) getting data from Jack Woolen (Rob)
+**getting new prepbufr data from J. Woolen. DONE
+* AIRCRFT data prior cdas prepbufr (2001) can we get reprocessed data? (DM) yes, DONE
+* Preprocessed TOMS data need to be moved to a proper place on archive (Kris & Rob)(DONE)
+* Get and process OMI and MLS v2.2 for 2012 - present (Kris). The same for MLS Temperature (Larry C.?) DONE
+* Set up automatic processing of new MLS T, MLS O3 and OMI data
+* tcvitals files (filling gap for 2005 and get some preop 2004 data?) (DM) -Jack Woolen provided 2000-2006 tcvitals (Rob has access to aquired data) DONE , getting additional data for 1980-1999.
+==Issues to address==
+* Infrastructure to bring in "spun-up" satbias/satbang files.  Specific issues: (WM)
+** Restart satbias/satbang files created for scout.  Question remaining is how will values be adjusted for full M2 (if at all):
+*** satbang coefficients & tlapmean values - spin up from scout diags
+*** satbias coefficients - substitute in from "spun-up" scout satbias_rst (some delta-time in the future)
+* (Question from Austin) How much of the data currently used as 'passive' do we need? E.g. SBUV N16 has been 'passive' in FP for some time; should it be just turned off in MERRA-2? It's bad data