GEOS-5 Software Engineering: Difference between revisions

(9 intermediate revisions by the same user not shown)

Line 5:

== Source Code Configuration Management ==

==== General Policies ====

=== General Policies ===

==== The ESMA Project ====

=== The ESMA Project ===

The ESMA project is concerned with the deployment of modeling and data

assimilation applications that are part of the ESMF testbed applications.

Line 19:

* Takacs, Larry

==== The ESMA CVS repository ====

=== The ESMA CVS repository ===

CVSACL is a version control server intended to specifically support the access control patches to CVS (https://progress.nccs.nasa.gov/trac/admin/wiki/CVSACL). Access to CVSACL requires an NCCS account (https://www.nccs.nasa.gov/). The web interface to CVS ESMA project is available at https://cvsacl.nccs.nasa.gov/cgi-bin/.

Line 25:

The ESMA CVS repository has a flat directory structure designed to accommodate a variety of modeling systems. The repository holds 'Applications', 'Components' and other software libraries needed to build earth modeling systems. The directories under esma/src/ are Applications/, Components/, Config/, Couplers/, Documentation/, Shared/ and of course CVS/.

~~PACKAGES~~: A collection of source files having one or more software deliverables

'''Packages:''' A collection of source files having one or more software deliverables

* Libraries

* Executables (binaries, scripts)

Line 32:

* Examples

~~MODULES~~: CVS modules are then used to compose individual modeling systems. A module is a collection of packages comprising some stand alone application, e.g. GEOSGCM_m0 (GMAO Unified Model, GEOS-5). A complete list of ESMA CVS modules is available at CVSROOT/modules.

'''Modules:''' CVS modules are then used to compose individual modeling systems. A module is a collection of packages comprising some stand alone application, e.g. GEOSGCM_m0 (GMAO Unified Model, GEOS-5). A complete list of ESMA CVS modules is available at CVSROOT/modules.

Some examples are:

# ESMF/MAPL Tutorial

G5tutorial -d G5tutorial/src esma/src/Applications/G5tutorial &Config \

G5tutorial -d G5tutorial/src esma/src/Applications/G5tutorial &Config &GEOSgcm_Shared_m2

&GEOSgcm_Shared_m2

# Ganymed

Ganymed -d GEOSagcm &GEOSGCM_m3

~~etc.~~

==== CVS/CVSACL ====

=== CVS/CVSACL ===

===== Approaches, loopholes etc. =====

==== Approaches, loopholes etc. ====

There are two basic approaches to perform configuration management under CVS:

Line 64:

Line 62:

* User can logon to machine where repository resides and tinker with files There really is no good way to enforce CVS policies

===== Solution: CVS/ACL =====

==== Solution: CVS/ACL ====

CVS/ACL is a patch to CVS for access control list management. It provides advanced ACL definitions per modules, directories and files on branch/tag for remote cvs repository connections. As a result the execution of all CVS subcommands can be controlled with eight different permissions.

Line 82:

Line 81:

* General users are not allowed to directly manipulate repository files.

===== GMAO development group =====

==== GMAO development group ====

===== CVSACL implementation =====

==== CVSACL implementation ====

# CVSROOT :ext:USERNAME@cvsacldirect:/cvsroot/esma

Line 94:

Line 93:

For uniformity, we suggest following the [[CVS Best Practices]].

===== Tag conventions =====

==== Tag conventions ====

===== CVS/ACL administration =====

==== CVS/ACL administration ====

== Release Engineering ==

Line 110:

Line 109:

== The ESMA Build Mechanism ==

==== Baselibs: Managing External Dependencies ====

=== Baselibs: Managing External Dependencies ===

==== Building GEOS-5 AGCM ====

=== Building GEOS-5 AGCM ===

This section lists the steps to checkout and build GEOS-5 mechanism using the latest stable tag: Ganymed-2_1_p5.

===== Set up CVSROOT =====

==== Set up CVSROOT ====

First, set up your CVSROOT environment variable using the scheme provided at the [https://progress.nccs.nasa.gov/trac/admin/wiki/CVSACL NCCS's CVSACL webpage] (requires NCCS login) where:

Line 125:

Start the tunnel (machines other than Discover and Pleiades)

===== Checking out the model =====

==== Checking out the model ====

Make the directory in which you wish to checkout the model:

Make the directory in which you wish to checkout the model and do the actual checkout:

$ mkdir G21p5

$ cd G21p5

~~And do the actual checkout using:~~

$ cvs co -r Ganymed-2_1_p5 Ganymed

In general:

In general, one uses <tt>$ cvs co -r <Tag Name> <Module Name></tt> where <Tag Name> is the tag for the model to check out (e.g., Ganymed-2_0_UNSTABLE, Fortuna-2_5_p6) and <Module Name> is the module (e.g., Ganymed, Fortuna).

$ cvs co -r <Tag Name> <Module Name>

where <Tag Name> is the tag for the model to check out (e.g., Ganymed-2_0_UNSTABLE, Fortuna-2_5_p6) and <Module Name> is the module (e.g., Ganymed, Fortuna).

===== Build and install the model =====

==== Build and install the model ====

Go into the src/ directory of your model. Following above:

$ cd ~~Ganymed-2_1_p5~~/GEOSagcm/src

$ cd G21p5/GEOSagcm/src

Setup the environment by sourcing the <code>g5_modules</code> file:

Line 178:

Line 171:

$ gmake ESMA_FC=pgfortran --jobs=N pinstall |& tee make.install.log (on tcsh)

===== Monitor build process =====

==== Monitor build process ====

The build can be monitored using the utility gmh.pl in the directory Config. From the src directory

Line 224:

Line 217:

===== Advanced features =====

==== Advanced features ====

======Check load balance of build======

=====Check load balance of build=====

The model includes useful tools like build timers, Config/esma_timer.sh, Config/esma_tgraph.pl. These are useful to time the build and check the load balance of the build, process. There is a way to hook these timers to the build process by setting

Line 234:

Line 227:

ESMA_base.mk:ESMA_TIMER_CO = # command to end timer (for user to backet code segments)

======Customize build======

=====Customize build=====

A build can be customized by using <tt>$HOME/.esma_xxxx.mk</tt>

Line 243:

Line 236:

These effectively let you change whatever you want - useful for debugging, etc. For example, you can set your timers in ~/.esma_base.mk.

<!--

=== Building ESMA Components ===

Line 249:

Line 241:

=== Building ESMA Systems ===

-->

== Testing and validation==

@@ Line 5: / Line 5: @@
 == Source Code Configuration Management ==
-==== General Policies ====
+=== General Policies ===
-==== The ESMA Project ====
+=== The ESMA Project ===
 The ESMA project is concerned with the deployment of modeling and data
 assimilation applications that are part of the ESMF testbed applications.
@@ Line 19: / Line 19: @@
 * Takacs, Larry
-==== The ESMA CVS repository ====
+=== The ESMA CVS repository ===
 CVSACL is a version control server intended to specifically support the access control patches to CVS (https://progress.nccs.nasa.gov/trac/admin/wiki/CVSACL). Access to CVSACL requires an NCCS account (https://www.nccs.nasa.gov/). The web interface to CVS ESMA project is available at https://cvsacl.nccs.nasa.gov/cgi-bin/.
@@ Line 25: / Line 25: @@
 The ESMA CVS repository has a flat directory structure designed to accommodate a variety of modeling systems. The repository holds 'Applications', 'Components' and other software libraries needed to build earth modeling systems. The directories under esma/src/ are Applications/, Components/, Config/, Couplers/, Documentation/, Shared/ and of course CVS/.
-PACKAGES: A collection of source files having one or more software deliverables
+'''Packages:''' A collection of source files having one or more software deliverables
 * Libraries
 * Executables (binaries, scripts)
@@ Line 32: / Line 32: @@
 * Examples
-MODULES: CVS modules are then used to compose individual modeling systems. A module is a collection of packages comprising some stand alone application, e.g. GEOSGCM_m0 (GMAO Unified Model, GEOS-5). A complete list of ESMA CVS modules is available at CVSROOT/modules.
+'''Modules:''' CVS modules are then used to compose individual modeling systems. A module is a collection of packages comprising some stand alone application, e.g. GEOSGCM_m0 (GMAO Unified Model, GEOS-5). A complete list of ESMA CVS modules is available at CVSROOT/modules.
 Some examples are:
   # ESMF/MAPL Tutorial
-  G5tutorial        -d G5tutorial/src  esma/src/Applications/G5tutorial &Config \
+  G5tutorial        -d G5tutorial/src  esma/src/Applications/G5tutorial &Config &GEOSgcm_Shared_m2
-                                      &GEOSgcm_Shared_m2
   # Ganymed
   Ganymed           -d GEOSagcm &GEOSGCM_m3
-etc.
-==== CVS/CVSACL ====
+=== CVS/CVSACL ===
-===== Approaches, loopholes etc. =====
+==== Approaches, loopholes etc. ====
 There are two basic approaches to perform configuration management under CVS:
@@ Line 64: / Line 62: @@
 * User can logon to machine where repository resides and tinker with files There really is no good way to enforce CVS policies
-===== Solution: CVS/ACL =====
+==== Solution: CVS/ACL ====
 CVS/ACL is a patch to CVS for access control list management. It provides advanced ACL definitions per modules, directories and files on branch/tag for remote cvs repository connections. As a result the execution of all CVS subcommands can be controlled with eight different permissions.
@@ Line 82: / Line 81: @@
 * General users are not allowed to directly manipulate repository files.
-===== GMAO development group =====
+==== GMAO development group ====
-===== CVSACL implementation =====
+==== CVSACL implementation ====
 # CVSROOT :ext:USERNAME@cvsacldirect:/cvsroot/esma
@@ Line 94: / Line 93: @@
 For uniformity, we suggest following the [[CVS Best Practices]].
-===== Tag conventions =====
+==== Tag conventions ====
-===== CVS/ACL administration =====
+==== CVS/ACL administration ====
 == Release Engineering ==
@@ Line 110: / Line 109: @@
 == The ESMA Build Mechanism ==
-==== Baselibs: Managing External Dependencies ====
+=== Baselibs: Managing External Dependencies ===
-==== Building GEOS-5 AGCM ====
+=== Building GEOS-5 AGCM ===
 This section lists the steps to checkout and build GEOS-5 mechanism using the latest stable tag: Ganymed-2_1_p5.
-===== Set up CVSROOT =====
+==== Set up CVSROOT ====
 First, set up your CVSROOT environment variable using the scheme provided at the [https://progress.nccs.nasa.gov/trac/admin/wiki/CVSACL NCCS's CVSACL webpage] (requires NCCS login) where:
@@ Line 125: / Line 125: @@
 Start the tunnel (machines other than Discover and Pleiades)
-===== Checking out the model =====
+==== Checking out the model ====
-Make the directory in which you wish to checkout the model:
+Make the directory in which you wish to checkout the model and do the actual checkout:
   $ mkdir G21p5
   $ cd G21p5
-And do the actual checkout using:
   $ cvs co -r Ganymed-2_1_p5 Ganymed
-In general:
+In general, one uses <tt>$ cvs co -r <Tag Name> <Module Name></tt> where <Tag Name> is the tag for the model to check out (e.g., Ganymed-2_0_UNSTABLE, Fortuna-2_5_p6) and <Module Name> is the module (e.g., Ganymed, Fortuna).
- $ cvs co -r <Tag Name> <Module Name>
-where <Tag Name> is the tag for the model to check out (e.g., Ganymed-2_0_UNSTABLE, Fortuna-2_5_p6) and <Module Name> is the module (e.g., Ganymed, Fortuna).
-===== Build and install the model =====
+==== Build and install the model ====
 Go into the src/ directory of your model. Following above:
-  $ cd Ganymed-2_1_p5/GEOSagcm/src
+  $ cd G21p5/GEOSagcm/src
 Setup the environment by sourcing the <code>g5_modules</code> file:
@@ Line 178: / Line 171: @@
   $ gmake ESMA_FC=pgfortran --jobs=N pinstall |& tee make.install.log (on tcsh)
-===== Monitor build process =====
+==== Monitor build process ====
 The build can be monitored using the utility gmh.pl in the directory Config. From the src directory
@@ Line 224: / Line 217: @@
-===== Advanced features =====
+==== Advanced features ====
-======Check load balance of build======
+=====Check load balance of build=====
 The model includes useful tools like build timers, Config/esma_timer.sh, Config/esma_tgraph.pl. These are useful to time the build and check the load balance of the build, process. There is a way to hook these timers to the build process by setting
@@ Line 234: / Line 227: @@
   ESMA_base.mk:ESMA_TIMER_CO  = # command to end   timer (for user to backet code segments)
-======Customize build======
+=====Customize build=====
 A build can be customized by using <tt>$HOME/.esma_xxxx.mk</tt>
@@ Line 243: / Line 236: @@
 These effectively let you change whatever you want - useful for debugging, etc. For example, you can set your timers in ~/.esma_base.mk.
 <!--
 === Building ESMA Components ===
@@ Line 249: / Line 241: @@
 === Building ESMA Systems ===
 -->
+== Testing and validation==