GEOS-5

Useful Tools: Difference between revisions

← Older editNewer edit →
mNo edit summary
Line 142: Line 142:


=== Extract fields(s) from a file ===
=== Extract fields(s) from a file ===
==== Extract variable(s) from a file ====


Often, our NetCDF4 files have many variables and we only care about one. CDO allows one to extract or ''select'' one or more variables. For example, if you only want CLCN, use '''selname''':
Often, our NetCDF4 files have many variables and we only care about one. CDO allows one to extract or ''select'' one or more variables. For example, if you only want CLCN, use '''selname''':
Line 168: Line 170:
  cdo sinfon: Processed 1 variable over 1 timestep ( 0.00s )
  cdo sinfon: Processed 1 variable over 1 timestep ( 0.00s )


CDO has many of these operators:
==== Extract time(s) from a file ====
 
  selparam Select parameters by identifier
  delparam Delete parameters by identifier
  selcode Select parameters by code number
  delcode Delete parameters by code number
  selname Select parameters by name
  delname Delete parameters by name
  selstdname Select parameters by standard name
  sellevel Select levels
  sellevidx Select levels by index
  selgrid Select grids
  selzaxis Select z-axes
  selltype Select GRIB level types
  seltabnum Select parameter table numbers
  seltimestep Select timesteps
  seltime Select times
  selhour Select hours
  selday Select days
  selmon Select months
  selyear Select years
  selseas Select seasons
  seldate Select dates
  selsmon Select single month
  sellonlatbox Select a longitude/latitude box
  selindexbox Select an index box
 
=== Extract time(s) from a file ===


To extract a single (or multiple) year from a multi-step file, you can use <tt>selyear</tt>,''year''. For multiple years, you can either do <tt>cdo '''selyear,1999,2000,2001'''</tt> or <tt>cdo '''selyear,1999/2001'''</tt>.
To extract a single (or multiple) year from a multi-step file, you can use <tt>selyear</tt>,''year''. For multiple years, you can either do <tt>cdo '''selyear,1999,2000,2001'''</tt> or <tt>cdo '''selyear,1999/2001'''</tt>.
Line 285: Line 260:
   2001-09-14 06:00:00  2001-10-14 16:00:00  2001-11-14 02:00:00  2001-12-14 12:00:00
   2001-09-14 06:00:00  2001-10-14 16:00:00  2001-11-14 02:00:00  2001-12-14 12:00:00
  cdo sinfon: Processed 7 variables over 36 timesteps ( 0.00s )
  cdo sinfon: Processed 7 variables over 36 timesteps ( 0.00s )
==== Other select operators ====
CDO has many of these operators:
  selparam Select parameters by identifier
  delparam Delete parameters by identifier
  selcode Select parameters by code number
  delcode Delete parameters by code number
  selname Select parameters by name
  delname Delete parameters by name
  selstdname Select parameters by standard name
  sellevel Select levels
  sellevidx Select levels by index
  selgrid Select grids
  selzaxis Select z-axes
  selltype Select GRIB level types
  seltabnum Select parameter table numbers
  seltimestep Select timesteps
  seltime Select times
  selhour Select hours
  selday Select days
  selmon Select months
  selyear Select years
  selseas Select seasons
  seldate Select dates
  selsmon Select single month
  sellonlatbox Select a longitude/latitude box
  selindexbox Select an index box
=== Combining Operators ===
Often, you want to do multiple operations on a file. You could, say, do a '''selname''' and output only one variable to a file, then a '''sellevel''' on that new file to select a single level, and then a '''selyear''' on ''that'', etc.:
$ cdo '''selname,OX''' pchem.species.CMIP-5.1870-2097.z_91x72.nc4 onlyOX.nc4
cdo selname: Processed 17926272 values from 7 variables over 2736 timesteps ( 9.60s )
$ cdo '''sellevel,1.5''' onlyOX.nc4 onlyOX.only1.5.nc4
cdo sellevel: Processed 248976 values from 1 variable over 2736 timesteps ( 0.76s )
$ cdo '''selyear,1999''' onlyOX.only1.5.nc4 onlyOX.only1.5.only1999.nc4
cdo selyear: Processed 1092 values from 1 variable over 2736 timesteps ( 0.05s )
$ cdo sinfon onlyOX.only1.5.only1999.nc4
  File format: netCDF4
    -1 : Institut Source  Ttype    Levels Num  Gridsize Num Dtype : Parameter name
    1 : unknown  http://geos5.org/wiki/index.php?title=GEOS-5_Configuration_for_AR5 instant      1  1        91  1  F32  : OX       
  Grid coordinates :
    1 : lonlat      > size      : dim = 91  nx = 0  ny = 91
                        lat      : first = -1.57079637  last = 1.57079625  inc = 0.0349065065  radians
  Vertical coordinates :
    1 : generic            layer : 1.5
  Time coordinate :  12 steps
    RefTime =  1870-01-15 12:00:00  Units = hours  Calendar = standard
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  1999-01-14 22:00:00  1999-02-14 08:00:00  1999-03-16 18:00:00  1999-04-16 04:00:00
  1999-05-16 14:00:00  1999-06-16 00:00:00  1999-07-16 10:00:00  1999-08-15 20:00:00
  1999-09-15 06:00:00  1999-10-15 16:00:00  1999-11-15 02:00:00  1999-12-15 12:00:00
cdo sinfon: Processed 1 variable over 12 timesteps ( 0.00s )
Of course, this is not only annoying, but wasteful as you are creating many temporary files. Instead, CDO allows one to "combine" or "chain" operators. This is done by using -operator:
  cdo -L operatorN -operatorN-1 ... -operator2 -operator1 input (output)
The -L is used because HDF5 isn't thread-safe as currently compiled. This "locks" I/O preventing an issue. We are working on trying to get CDO to work in parallel better.
So, doing the above operator in one step:
$ cdo -L selyear,1999 -sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 onlyOX-1.5-1999.nc4
cdo selyear: Started child process "sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 (pipe1.1)".
cdo(2) sellevel: Started child process "selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 (pipe2.1)".
cdo(3) selname: Processed 17926272 values from 7 variables over 2736 timesteps ( 3.18s )
cdo(2) sellevel: Processed 248976 values from 1 variable over 2736 timesteps ( 3.18s )
cdo selyear: Processed 1092 values from 1 variable over 2736 timesteps ( 3.18s )
$ cdo sinfon onlyOX-1.5-1999.nc4
  File format: netCDF4
    -1 : Institut Source  Ttype    Levels Num  Gridsize Num Dtype : Parameter name
    1 : unknown  http://geos5.org/wiki/index.php?title=GEOS-5_Configuration_for_AR5 instant      1  1        91  1  F32  : OX       
  Grid coordinates :
    1 : lonlat      > size      : dim = 91  nx = 0  ny = 91
                        lat      : first = -1.57079637  last = 1.57079625  inc = 0.0349065065  radians
  Vertical coordinates :
    1 : generic            layer : 1.5
  Time coordinate :  12 steps
    RefTime =  1870-01-15 12:00:00  Units = hours  Calendar = standard
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  1999-01-14 22:00:00  1999-02-14 08:00:00  1999-03-16 18:00:00  1999-04-16 04:00:00
  1999-05-16 14:00:00  1999-06-16 00:00:00  1999-07-16 10:00:00  1999-08-15 20:00:00
  1999-09-15 06:00:00  1999-10-15 16:00:00  1999-11-15 02:00:00  1999-12-15 12:00:00
cdo sinfon: Processed 1 variable over 12 timesteps ( 0.00s )
This file is the same as the one done in three steps:
$ cdo diffn onlyOX.only1.5.only1999.nc4 onlyOX-1.5-1999.nc4
  0 of 12 records differ
cdo diffn: Processed 2184 values from 2 variables over 24 timesteps ( 0.00s )
Of course, you could even do the '''diffn''' as well in the command:
$ cdo -L diffn -selyear,1999 -sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 onlyOX.only1.5.only1999.nc4
cdo diffn: Started child process "selyear,1999 -sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 (pipe1.1)".
cdo(2) selyear: Started child process "sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 (pipe2.1)".
cdo(3) sellevel: Started child process "selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 (pipe3.1)".
  0 of 12 records differ
cdo(4) selname: Processed 17926272 values from 7 variables over 2736 timesteps ( 3.13s )
cdo(3) sellevel: Processed 248976 values from 1 variable over 2736 timesteps ( 3.13s )
cdo(2) selyear: Processed 1092 values from 1 variable over 2736 timesteps ( 3.13s )
cdo diffn: Processed 2184 values from 2 variables over 24 timesteps ( 3.13s )
Note: if you don't want the extraneous information, use '''-s''' to enable '''silent''' mode:
cdo -s -L diffn -selyear,1999 -sellevel,1.5 -selname,OX pchem.species.CMIP-5.1870-2097.z_91x72.nc4 onlyOX.only1.5.only1999.nc4
  0 of 12 records differ


== tkcvs ==
== tkcvs ==
Retrieved from "https://geos5.org/wiki/index.php?title=Useful_Tools"