Useful Tools: Difference between revisions
| (6 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
CDO is a collection of command line Operators to manipulate and analyse Climate and NWP model Data. | CDO is a collection of command line Operators to manipulate and analyse Climate and NWP model Data. | ||
=== Where to find CDO === | |||
==== NCCS ==== | |||
Users can access CDO in a variety of ways. On discover, the easiest way is to load the cdo module: | |||
=== Display info about a file === | module load other/cdo | ||
It is also included in GMAO-Baselibs-4_0_0 and higher (corresponds to Ganymed-3_0 or newer). It will be located at | |||
$BASEDIR/Linux/bin/cdo | |||
once <tt>g5_modules</tt> is sourced. | |||
==== NAS ==== | |||
At NAS, there are a couple options. First, it is available through Baselibs as above. You can also access a portable version (compiled with system gcc) at | |||
/nobackup/gmao_SIteam/Utilities/bin/cdo | |||
As this is completely portable, you can also copy this to a local <tt>bin</tt> directory if desired. | |||
==== GSFC Desktops ==== | |||
If you are on a GMAO desktop and have access to <tt>/ford1</tt>, there is a version at | |||
/ford1/local/EL6-64/bin/cdo | |||
as well as at: | |||
/ford1/share/gmao_SIteam/Utilities/bin/cdo | |||
The <tt>/ford1/share</tt> version could be more bleeding-edge than the <tt>/ford1/local</tt> version as it will usually be whatever version is in the latest Baselibs tag. | |||
=== Example Uses of CDO === | |||
==== Display info about a file ==== | |||
CDO offers many operators that provide information about a file. The most important two are '''infon''' and '''sinfon'''. Note the '''n''' at the end: that means "display variable names". If you don't provide that, you'll still get information, but it'll be for variables -1, -2, -3, etc. | CDO offers many operators that provide information about a file. The most important two are '''infon''' and '''sinfon'''. Note the '''n''' at the end: that means "display variable names". If you don't provide that, you'll still get information, but it'll be for variables -1, -2, -3, etc. | ||
==== Short Information ==== | ===== Short Information ===== | ||
If you just want a summary of what's in the file, use '''sinfon''': | If you just want a summary of what's in the file, use '''sinfon''': | ||
| Line 64: | Line 96: | ||
cdo sinfon: Processed 26 variables over 1 timestep ( 0.02s ) | cdo sinfon: Processed 26 variables over 1 timestep ( 0.02s ) | ||
==== Information and Simple Statistics ==== | ===== Information and Simple Statistics ===== | ||
The other operator, '''infon''', provides more information and some useful statistics: | The other operator, '''infon''', provides more information and some useful statistics: | ||
| Line 126: | Line 158: | ||
As you can see, you not only get information, but statistics. It provides the number of values on the grid, <tt>Gridsize</tt>; the number of Missing Values, <tt>Miss</tt>; and the <tt>Minimum</tt>, <tt>Mean</tt>, and <tt>Maximum</tt> for each <tt>Level</tt> for each value. | As you can see, you not only get information, but statistics. It provides the number of values on the grid, <tt>Gridsize</tt>; the number of Missing Values, <tt>Miss</tt>; and the <tt>Minimum</tt>, <tt>Mean</tt>, and <tt>Maximum</tt> for each <tt>Level</tt> for each value. | ||
=== Diff two files === | ==== Diff two files ==== | ||
One of the main reasons CDO is attractive is for diffing two files. With binary files, such as the old restarts, one could use <tt>cmp</tt> or <tt>diff</tt> to check for differences. However, <tt>cmp</tt> will not work on NetCDF files because while the data might be the same, the metadata surely won't: | One of the main reasons CDO is attractive is for diffing two files. With binary files, such as the old restarts, one could use <tt>cmp</tt> or <tt>diff</tt> to check for differences. However, <tt>cmp</tt> will not work on NetCDF files because while the data might be the same, the metadata surely won't: | ||
| Line 212: | Line 244: | ||
endif | endif | ||
=== Extract fields(s) from a file === | ==== Extract fields(s) from a file ==== | ||
==== Extract variable(s) from a file ==== | ===== Extract variable(s) from a file ===== | ||
Often, our NetCDF4 files have many variables and we only care about one. CDO allows one to extract or ''select'' one or more variables. For example, if you only want CLCN, use '''selname''': | Often, our NetCDF4 files have many variables and we only care about one. CDO allows one to extract or ''select'' one or more variables. For example, if you only want CLCN, use '''selname''': | ||
| Line 241: | Line 273: | ||
cdo sinfon: Processed 1 variable over 1 timestep ( 0.00s ) | cdo sinfon: Processed 1 variable over 1 timestep ( 0.00s ) | ||
==== Extract time(s) from a file ==== | ===== Extract time(s) from a file ===== | ||
To extract a single (or multiple) year from a multi-step file, you can use <tt>selyear</tt>,''year''. For multiple years, you can either do <tt>cdo '''selyear,1999,2000,2001'''</tt> or <tt>cdo '''selyear,1999/2001'''</tt>. | To extract a single (or multiple) year from a multi-step file, you can use <tt>selyear</tt>,''year''. For multiple years, you can either do <tt>cdo '''selyear,1999,2000,2001'''</tt> or <tt>cdo '''selyear,1999/2001'''</tt>. | ||
| Line 332: | Line 364: | ||
cdo sinfon: Processed 7 variables over 36 timesteps ( 0.00s ) | cdo sinfon: Processed 7 variables over 36 timesteps ( 0.00s ) | ||
==== Other select operators ==== | ===== Other select operators ===== | ||
CDO has many of these operators: | CDO has many of these operators: | ||
| Line 360: | Line 392: | ||
selindexbox Select an index box | selindexbox Select an index box | ||
=== Combining Operators === | ==== Combining Operators ==== | ||
Often, you want to do multiple operations on a file. You could, say, do a '''selname''' and output only one variable to a file, then a '''sellevel''' on that new file to select a single level, and then a '''selyear''' on ''that'', etc.: | Often, you want to do multiple operations on a file. You could, say, do a '''selname''' and output only one variable to a file, then a '''sellevel''' on that new file to select a single level, and then a '''selyear''' on ''that'', etc.: | ||
| Line 367: | Line 399: | ||
cdo selname: Processed 17926272 values from 7 variables over 2736 timesteps ( 9.60s ) | cdo selname: Processed 17926272 values from 7 variables over 2736 timesteps ( 9.60s ) | ||
$ cdo '''sellevel,1.5''' onlyOX.nc4 onlyOX.only1.5.nc4 | $ cdo '''sellevel,1.5''' onlyOX.nc4 onlyOX.only1.5.nc4 | ||
cdo sellevel: Processed 248976 values from 1 variable over 2736 timesteps ( 0.76s ) | cdo sellevel: Processed 248976 values from 1 variable over 2736 timesteps ( 0.76s ) | ||
$ cdo '''selyear,1999''' onlyOX.only1.5.nc4 onlyOX.only1.5.only1999.nc4 | $ cdo '''selyear,1999''' onlyOX.only1.5.nc4 onlyOX.only1.5.only1999.nc4 | ||
cdo selyear: Processed 1092 values from 1 variable over 2736 timesteps ( 0.05s ) | cdo selyear: Processed 1092 values from 1 variable over 2736 timesteps ( 0.05s ) | ||
| Line 441: | Line 473: | ||
0 of 12 records differ | 0 of 12 records differ | ||
== | == TkCVS == | ||
The main website for TkCVS can be found [http://www.twobarleycorns.net/tkcvs.html here]. | |||
On discover, you can use TkCVS by loading the <tt>other/tkcvs-8.2.3</tt> module: | |||
module load other/tkcvs-8.2.3 | |||
== ack == | == ack == | ||
The main website for ack can be found [http://beyondgrep.com/ here]. | |||
If you'd like to try ack out, you can run: | |||
curl http://beyondgrep.com/ack-2.04-single-file > ~/bin/ack && chmod 0755 !#:3 | |||
and it will install ack for you in your local bin directory. | |||
[[Category:SI Team]] | [[Category:SI Team]] | ||
[[Category:Brown Bags]] | [[Category:Brown Bags]] | ||