G5NR Data Access Guide
For questions or comments please send an email to g5nr at lists dot nasa dot gov.
G5NR Model Config
G5NR File Spec
The G5NR data files are generated using the NetCDF-4 library [link] which uses HDF-5 [link] as the underlying format. For more details please see [link].
Download globaldata from FTP server
[NOTE: once data is made public, would we still have a username?]
The base url for G5NR data is
ftp://G5NR@:ftp.nccs.nasa.gov/c1440_NR/DATA.
At this location, the data is organized by resolution (0.5000_deg/0.0625_deg), type (const/inst/tavg/tdav), collection name, year, month and day as follows:
|-- resolution | |-- type | | |-- collection | | | |-- year | | | | |-- month | | | | | |-- day
[A couple of lines for how many files for inst, tavg, tdav etc.]
A web browser can be used to browse directories, read and retrieve files. To retrieve the collection inst01hr_3d_T_Cv for the day 2006-09-18, one needs to point to:
ftp://G5NR:@ftp.nccs.nasa.gov/c1440_NR/DATA/0.5000_deg/inst/inst01hr_3d_T_Cv/Y2006/M09/D18.
Alternately, one can use the command line tool wget to retrieve the same files as:
wget ftp://G5NR:@ftp.nccs.nasa.gov/c1440_NR/DATA/0.5000_deg/inst/inst01hr_3d_T_Cv/Y2006/M09/D18/*
The * at the end would retrieve all (24) files for the given day.
To download a file for a specific time, say 0900z, try
wget ftp://G5NR:@ftp.nccs.nasa.gov/c1440_NR/DATA/0.5000_deg/inst/inst01hr_3d_T_Cv/Y2006/M09/D18/c1440_NR.inst01hr_3d_T_Cv.20060918_0900z.nc4
Download tool (retrieve data subsets)
The download tool can be used for server-side subsetting of collections. This can be done either via the tool's web interface or the command line.
- Web interface - The base url for the download tool for G5NR data is
http://portal.nccs.nasa.gov/cgi-lats4d/opendap.cgi?&path=/OSSE/GEOS-5.12.
To download a subset of the collection inst01hr_3d_T_Cv, from the above url, one can follow links [0.5000_deg → inst → Download (for the appropriate collection)] to the collection's download page at http://portal.nccs.nasa.gov/cgi-lats4d/webform.cgi?&i=OSSE/GEOS-5.12/0.5000_deg/inst01hr_3d_T_Cv. Here one can select variables, the begin and end times, vertical levels (for 3D data), spatial subset (by drawing a box or specifying the lats/lons) and download the resulting data in a format of choice (compressed NetCDF-4, little endian binary etc.).
- Command line/Python script - There has to be a way to construct the POST request and then potentially use the Python requests module???
OPenDAP
OPeNDAP is a data server architecture that allows users to use data files that are stored on remote computers with their favorite analysis and visualization tools. Opening an OPeNDAP file is as easy replacing the file name in the client software by an OPeNDAP URL. All G5NR collections that are provided by ftp/download-tool are also available on the OPeNDAP server
http://opendap.nccs.nasa.gov/dods/OSSE/GEOS-5.12/BETA9.
The metadata of our example collection, inst01hr_3d_T_Cv, can be viewed by following the links [0.5000_deg → inst → info (for the appropriate collection)] to the info page http://opendap.nccs.nasa.gov/dods/OSSE/GEOS-5.12/BETA9/0.5000_deg/inst/inst01hr_3d_T_Cv.info.
For retrieving aggregated data from the OPeNDAP server using your favorite client, see Client access below.
Client access
In the following, we read the field 'T' (air temperature) from collection inst01hr_3d_T_Cv, compute its min/max and if applicable, plot it. We give an example for each of the two cases
- a file has been downloaded either via ftp or using the download tool
- using the OPenDAP server
For each case, we compute min/max for both
- global temperature
- temperature over North America
Programming
Fortran
For reading a downloaded file or accessing directly via OPeNDAP using Fortran, please see this page.
Free clients
In this section we read air temperature, compute it min/max (as with the 'programming' examples) and display the surface air temperature.
Python
netcdf4-python
If netcdf4-python module is available, the following script would read air temperature for the specified time, compute its min and max values and plot it.
#!/usr/bin/env python
import sys
import numpy as np
import netCDF4 as nc4
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
rootgrp = nc4.Dataset('http://opendap.nccs.nasa.gov:9090/dods/OSSE/GEOS-5.12/BETA9/0.5000_deg/inst/inst01hr_3d_T_Cv', 'r')
# read air temperature
print 'Reading T for time=37...',; sys.stdout.flush()
Ttime37 = rootgrp.variables['t'][36,:,:,:]
print 'done.'; sys.stdout.flush()
# min/max
print 'min(T):', np.min(Ttime37)
print 'max(T):', np.max(Ttime37)
# set up cylindrical map
m = Basemap(
projection='cyl',
llcrnrlat=-90, urcrnrlat=90,
llcrnrlon=-180, urcrnrlon=180,
resolution='c'
)
m.drawcoastlines(linewidth=0.5)
m.drawmapboundary()
# plot contour
level = 71
X = np.arange(-180.0, 180.0, .5)
Y = np.arange(-90.0, 90.1, .5) # 90 is the last element
cp = plt.contour(X, Y, T[0,level,:,:], 20, zorder=2)
plt.clabel(cp, inline=1, fontsize=9)
plt.title('Air temperature at the surface')
plt.show()
pygrads
R
This example requires the ncdf4 and rworldmap packages.
> library(ncdf4)
> library(rworldmap)
Loading required package: sp
### Welcome to rworldmap ###
For a short introduction type : vignette('rworldmap')
> im <- 720
> jm <- 361
> lm <- 72
> nc <- nc_open("http://opendap.nccs.nasa.gov:9090/dods/OSSE/GEOS-5.12/BETA9/0.5000_deg/inst/inst01hr_3d_T_Cv")
< t <- ncvar_get(nc,"t",start=c(1,1,1,37),count=c(im,jm,lm,1))
> str(t)
num [1:720, 1:361, 1:72] 218 218 218 218 218 ...
> summary(t)
Min. 1st Qu. Median Mean 3rd Qu. Max.
179.7 220.9 241.7 243.4 265.3 316.7
> mapGriddedData(t[1:im,1:jm,71])
NCL
IDV
IDV is an OPeNDAP tool that can access and display the nature run data. In our OPenDAP server, all files are time aggregated, so they appear as a single dataset for each location.
This is an example to open and display the field 'T' (air temperature) from the collection 'inst01hr_3d_T_Cv'. The OPenDAP URL for this dataset is http://opendap.nccs.nasa.gov:80/dods/OSSE/GEOS-5.12/BETA9/0.5000_deg/inst/inst01hr_3d_T_Cv. The following steps are valid for IDV version 5.0u1 running on a Linux desktop.
From the 'Dashboard' panel
- Select Data Choosers -> URLS.In the URL field, enter the above OPenDAP URL and click on 'Add Source'
- Select Field Selector and choose the 3D field'air_temperature'. The 'Times' tab lists all the available levels and times for this data. At this point, one can select specific times, level and regions (subsetting) from the 'Times' and 'Level' and 'Region' tabs. Click on 'Create Display'.