Quick Fire Emission Dataset (QFED): Difference between revisions

(11 intermediate revisions by the same user not shown)

Line 1:

This page to contains information regarding QFED, mostly for developers.

This page to contains information regarding QFED, mostly for developers and NCCS users.

== How to Check Out and Build the Code ==

Line 7:

These are checkout and build instructions for QFED. We presume that the user has an account and appropriate permissions on '''progress''' to check code out, and access to the NCCS machines (e.g., '''discover''') to run code.

The name of the module in the cvs repository is '''QFED'''. ~~Currently the name of the development~~ tag is '''qfed-~~1_0_b4~~'''. ~~To checkout the~~ QFED source into the directory ''./qfed/src''~~, for example, use~~ the following command

The name of the module in the cvs repository is '''QFED'''. The curent stable tag is '''qfed-1_0_r5'''. The QFED source can be checked out into the directory ''${HOME}/projects/qfed/src'' with the following command

% cvs -d $CVSROOT co -d ./qfed/src -r qfed-~~1_0_b4~~ QFED

% cvs -d $CVSROOT co -d ${HOME}/projects/qfed/src -r qfed-1_0_r5 QFED

Once the code is checked out set the '''ESMADIR''' variable to point to the QFED ~~top~~ directory, e.g.:

Once the code is checked out set the '''ESMADIR''' variable to point to the parent QFED source directory, e.g.:

% setenv ESMADIR ${HOME}/projects/qfed

Make sure that you have a valid ${BASEDIR}. ~~QFED code is known to compile~~ against Baselibs-v3.1.7.

Make sure that you have a valid ${BASEDIR}. The qfed-1_0_r5 tag compiles successfully against Baselibs-v3.2.0 and Baselibs-v3.2.1.

To build the code navigate to the source directory where the main GNUMakefile resides:

% cd ./qfed/src

% cd ${HOME}/projects/qfed/src

The available options can be seen by running gmake or '''gmake esma_help''':

The available build options can be seen by running gmake or '''gmake esma_help''':

% gmake esma_help

Line 33:

Environment:

ESMADIR = ''.../qfed/src/''

BASEDIR = /opt/Baselibs/v3.1.7/

BASEDIR = /opt/Baselibs/v3.2.1/

ARCH = Linux

SITE = ''hostname''

Line 46:

% make PYTHON_INSTALL=python_install install

if you ~~wish~~ to ~~install~~ the ~~'''~~python~~''' modules~~ and ~~scripts as well~~.

It is safe to directly request building the QFED python modules skipping the intermediate build step.

To build and run QFED on discover you might need to change the default python interpreter with Python-2.5.4 for example. Additionally f2py and pyhdf are needed to build and run QFED. For details how to setup your environment on discover refer to section 4.

== QFED Vegetation Map ==

Line 69:

Line 72:

The available options and their description can be seen by running the script with '-h' or '--help' option.

== Level 3 ~~QFED~~ Products ==

== QFED Level 3 Products ==

There are two Level 3 (Global Gridded) QFED products:

# The daily fire radiative power (FRP) derived from MODIS Thermal Anomalies/Fire products (MOD14, MYD14). The name of the product is QFED Level 3a.

# The Level 3b product is derived from the Level 3a product. It contains the daily QFED fire emissions (BC, OC, CO, CO2, SO2).

# The Level 3b product is derived from the Level 3a product. It contains the daily QFED fire emissions (BC, OC, CO, CO2, SO2, PM25).

The QFED Level 3a product can be generated after obtaining the necessary MOD14 and/or MYD14 granules using the '''qfed_l3a.py''' script. Similarly, Level 3b can be generated using the '''qfed_l3b.py''' script. Typical usage of the scripts ~~will require~~ one to provide the year ~~and start and~~ end day of the year. For full list of options run the scripts with the '--help' option, e.g.

The QFED Level 3a product can be generated after obtaining the necessary MOD14 and/or MYD14 granules using the '''qfed_l3a.py''' script. Similarly, Level 3b can be generated using the '''qfed_l3b.py''' script. Typical usage of the scripts requires one to provide the date in terms of year a day of the year. Several consecutive days can be processed by additionally specifying end day of the year. For full list of options run the scripts with the '--help' option, e.g.

% qfed_l3a.py --help

== Running QFED ~~on discover~~ ==

== Running QFED ==

QFED ~~(tag qfed-1_0_a5)~~ has been successfully build and run on discover using Python-2.5.4, pyhdf-0.8.3, Baselibs-3.2.1 and Intel-11.0.083 compiler. First, lets set some environmental variables:

Here we will show how to setup the environment to build and run QFED on discover. QFED has been successfully build and run on discover using with Python-2.5.4, pyhdf-0.8.3, Baselibs-3.2.1 and Intel-11.0.083 compiler. First, lets set some environmental variables:

# load intel and MPI modules

module purge

module load comp/intel-11.0.083

module load other/mpi/mvapich2-1.4.1/intel-11.0.083

~~setenv~~ QFED ${HOME}/qfed/Linux # ~~QFED installation directory~~

# QFED installation directory

set QFED = ${HOME}/qfed/Linux/

# pyhdf - python interface to the HDF library

set PYHDF = '/usr/local/other/pyhdf/0.8.3/'

~~setenv PATH ${QFED}/bin/:${SHARE}/dasilva/bin/:${PATH}~~ # ~~QFED scripts and n4zip utility~~

# IGBP-INPE dataset

~~setenv PYTHONPATH ${QFED}/lib/Python/:${PYTHONPATH} # QFED modules~~

set QFED_IGBP = '/discover/nobackup/rgovinda/Emissions/Vegetation/GL_IGBP_INPE/'

~~setenv~~ QFED_IGBP '/discover/nobackup/rgovinda/Emissions/Vegetation/GL_IGBP_INPE' # ~~IGBP-INPE dataset~~

# MODIS Fire products

~~setenv~~ MODIS_FIRE '/discover/nobackup/dao_ops/intermediate/flk/modis/' # ~~MODIS Fire products~~

set MODIS_FIRE = '/discover/nobackup/dao_ops/intermediate/flk/modis/'

~~setenv~~ QFED_L3a ${HOME}/tmp/QFED-test ~~# QFED Level3a output directory~~

# QFED Level3a and Level3 output directories

~~setenv~~ QFED_L3b ${HOME}/tmp/QFED-test # QFED ~~Level3b output directory~~

set QFED_L3a = ${HOME}/tmp/QFED-test/

set QFED_L3b = ${HOME}/tmp/QFED-test/

# QFED scripts and n4zip utility

setenv PATH ${QFED}/bin/:${SHARE}/dasilva/bin/:${PATH}

# QFED and pyhdf python modules

setenv PYTHONPATH ${QFED}/lib/Python/:${PYHDF}/lib/python2.5/site-packages/:${PYTHONPATH}

To generate the QFED FRP (QFED-L3a) files for July 19, 2010 (DOY = 200) using both MODIS Aqua and Terra data, we will execute the qfed_l3a.py script, e.g.

qfed_l3a.py --mxd14=${MODIS_FIRE} --mxd03=${MODIS_FIRE} --igbp=${QFED_IGBP} --output=${QFED_L3a} 2010 200

Next we will generate the actual emissions files for the same day by running the qfed_l3b.py script

qfed_l3b --input=${QFED_L3a} --output=${QFED_L3b} --ndays=1 2010 200

qfed_l3b.py --input=${QFED_L3a} --output=${QFED_L3b} --ndays=1 2010 200

@@ Line 1: / Line 1: @@
 {{rightTOC}}
-This page to contains information regarding QFED, mostly for developers.
+This page to contains information regarding QFED, mostly for developers and NCCS users.
 == How to Check Out and Build the Code ==
@@ Line 7: / Line 7: @@
 These are checkout and build instructions for QFED. We presume that the user has an account and appropriate permissions on '''progress''' to check code out, and access to the NCCS machines (e.g., '''discover''') to run code.
-The name of the module in the cvs repository is '''QFED'''. Currently the name of the development tag is '''qfed-1_0_b4'''. To checkout the QFED source into the directory ''./qfed/src'', for example, use the following command
+The name of the module in the cvs repository is '''QFED'''. The curent stable tag is '''qfed-1_0_r5'''. The QFED source can be checked out into the directory ''${HOME}/projects/qfed/src'' with the following command
-  % cvs -d $CVSROOT co -d ./qfed/src -r qfed-1_0_b4 QFED
+  % cvs -d $CVSROOT co -d ${HOME}/projects/qfed/src -r qfed-1_0_r5 QFED
-Once the code is checked out set the '''ESMADIR''' variable to point to the QFED top directory, e.g.:
+Once the code is checked out set the '''ESMADIR''' variable to point to the parent QFED source directory, e.g.:
   % setenv ESMADIR ${HOME}/projects/qfed
-Make sure that you have a valid ${BASEDIR}. QFED code is known to compile against Baselibs-v3.1.7.
+Make sure that you have a valid ${BASEDIR}. The qfed-1_0_r5 tag compiles successfully against Baselibs-v3.2.0 and Baselibs-v3.2.1.
 To build the code navigate to the source directory where the main GNUMakefile resides:
-  % cd ./qfed/src
+  % cd ${HOME}/projects/qfed/src
-The available options can be seen by running gmake or '''gmake esma_help''':
+The available build options can be seen by running gmake or '''gmake esma_help''':
   % gmake esma_help
@@ Line 33: / Line 33: @@
     Environment:
        ESMADIR = ''.../qfed/src/''
-       BASEDIR = /opt/Baselibs/v3.1.7/
+       BASEDIR = /opt/Baselibs/v3.2.1/
           ARCH = Linux
           SITE = ''hostname''
@@ Line 46: / Line 46: @@
   % make PYTHON_INSTALL=python_install install
-if you wish to install the '''python''' modules and scripts as well.
+It is safe to directly request building the QFED python modules skipping the intermediate build step.
+To build and run QFED on discover you might need to change the default python interpreter with Python-2.5.4 for example. Additionally f2py and pyhdf are needed to build and run QFED. For details how to setup your environment on discover refer to section 4.
 == QFED Vegetation Map ==
@@ Line 69: / Line 72: @@
 The available options and their description can be seen by running the script with '-h' or '--help' option.
-== Level 3 QFED Products ==
+== QFED Level 3 Products ==
 There are two Level 3 (Global Gridded) QFED products:
 # The daily fire radiative power (FRP) derived from MODIS Thermal Anomalies/Fire products (MOD14, MYD14). The name of the product is QFED Level 3a.
-# The Level 3b product is derived from the Level 3a product. It contains the daily QFED fire emissions (BC, OC, CO, CO2, SO2).
+# The Level 3b product is derived from the Level 3a product. It contains the daily QFED fire emissions (BC, OC, CO, CO2, SO2, PM25).
-The QFED Level 3a product can be generated after obtaining the necessary MOD14 and/or MYD14 granules using the '''qfed_l3a.py''' script. Similarly, Level 3b can be generated using the '''qfed_l3b.py''' script. Typical usage of the scripts will require one to provide the year and start and end day of the year. For full list of options run the scripts with the '--help' option, e.g.
+The QFED Level 3a product can be generated after obtaining the necessary MOD14 and/or MYD14 granules using the '''qfed_l3a.py''' script. Similarly, Level 3b can be generated using the '''qfed_l3b.py''' script. Typical usage of the scripts requires one to provide the date in terms of year a day of the year. Several consecutive days can be processed by additionally specifying end day of the year. For full list of options run the scripts with the '--help' option, e.g.
   % qfed_l3a.py --help
-== Running QFED on discover ==
+== Running QFED ==
-QFED (tag qfed-1_0_a5) has been successfully build and run on discover using Python-2.5.4, pyhdf-0.8.3, Baselibs-3.2.1 and Intel-11.0.083 compiler. First, lets set some environmental variables:
+Here we will show how to setup the environment to build and run QFED on discover. QFED has been successfully build and run on discover using with Python-2.5.4, pyhdf-0.8.3, Baselibs-3.2.1 and Intel-11.0.083 compiler. First, lets set some environmental variables:
+ # load intel and MPI modules
+ module purge
+ module load comp/intel-11.0.083
+ module load other/mpi/mvapich2-1.4.1/intel-11.0.083
-  setenv QFED ${HOME}/qfed/Linux                                                   # QFED installation directory
+  # QFED installation directory
+ set QFED = ${HOME}/qfed/Linux/
+ # pyhdf - python interface to the HDF library
+ set PYHDF = '/usr/local/other/pyhdf/0.8.3/'
-  setenv PATH ${QFED}/bin/:${SHARE}/dasilva/bin/:${PATH}                           # QFED scripts and n4zip utility
+  # IGBP-INPE dataset
- setenv PYTHONPATH ${QFED}/lib/Python/:${PYTHONPATH}                              # QFED modules
+  set QFED_IGBP = '/discover/nobackup/rgovinda/Emissions/Vegetation/GL_IGBP_INPE/'
-  setenv QFED_IGBP '/discover/nobackup/rgovinda/Emissions/Vegetation/GL_IGBP_INPE' # IGBP-INPE dataset
+ # MODIS Fire products
-  setenv MODIS_FIRE '/discover/nobackup/dao_ops/intermediate/flk/modis/'           # MODIS Fire products
+  set MODIS_FIRE = '/discover/nobackup/dao_ops/intermediate/flk/modis/'
-  setenv QFED_L3a ${HOME}/tmp/QFED-test                                            # QFED Level3a output directory
+ # QFED Level3a and Level3 output directories
-  setenv QFED_L3b ${HOME}/tmp/QFED-test                                            # QFED Level3b output directory
+  set QFED_L3a = ${HOME}/tmp/QFED-test/
+  set QFED_L3b = ${HOME}/tmp/QFED-test/
+ # QFED scripts and n4zip utility
+ setenv PATH ${QFED}/bin/:${SHARE}/dasilva/bin/:${PATH}
+ # QFED and pyhdf python modules
+ setenv PYTHONPATH ${QFED}/lib/Python/:${PYHDF}/lib/python2.5/site-packages/:${PYTHONPATH}
 To generate the QFED FRP (QFED-L3a) files for July 19, 2010 (DOY = 200) using both MODIS Aqua and Terra data, we will execute the qfed_l3a.py script, e.g.
   qfed_l3a.py --mxd14=${MODIS_FIRE} --mxd03=${MODIS_FIRE} --igbp=${QFED_IGBP} --output=${QFED_L3a} 2010 200
 Next we will generate the actual emissions files for the same day by running the qfed_l3b.py script
-  qfed_l3b --input=${QFED_L3a} --output=${QFED_L3b} --ndays=1 2010 200
+  qfed_l3b.py --input=${QFED_L3a} --output=${QFED_L3b} --ndays=1 2010 200