Updating a Baselibs Component: Difference between revisions

From GEOS-5
Jump to navigation Jump to search
Initial add. Not yet wikified.
 
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Process of Importing A New Component to Baselibs ==
== Process of Updating A New Component to Baselibs ==


1. In a temporary directory (ImportCheckout) checkout:
This example document will detail the process that was used updating netCDF 4.3.2 into GMAO Baselibs.


  $ bcvs co Baselibs
=== Set up the new tag ===


2. Move into checkout:
In this case, we were updating netCDF-C 4.3.2 to GMAO-Baselibs-4_0_5. So, the "internal" tag we went to was 4_0_6. In a temporary directory (say, NewBaselibs) you checkout Baselibs and tag relative so you know where you started from. You also make sure the tag you're making doesn't exist:


  $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs
  $ bcvs co -kk Baselibs
   $ cd Baselibs/src
   $ cd Baselibs/src
  $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6-before-netCDF-4_3_2-NCO-4_4_4-cURL-7_37_0
  $ bcvs tag -d Internal-4_0_6


3. Tag current state:
=== Unpack the new library ===


   $ bcvs tag -r GMAO-Baselibs-3_3_0 Internal-3_3_1-before-NCO-4_2_3
First, move to a directory that can contain your "new import",
 
 
3b. Untag Internal-3_3_1 (just in case)
   $ cd /discover/swdev/mathomp4/Baselibs/NewImports
 
untar the package,
 
  $ tar xzf netcdf-c-4.3.2.tar.gz
 
and move into the new directory directory:


   $ bcvs tag -d Internal-3_3_1
   $ cd netcdf-c-4.3.2


---------
=== Import the new package ===
NCO 4.2.3
---------


4. Move to /gpfsm/dswdev/mathomp4/Baselibs/NewImports
First, do a dry-run import of the package (note the <tt>-nq</tt>):


   $ cd /gpfsm/dswdev/mathomp4/Baselibs/NewImports
   $ bcvs -nq import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2


a. Untar nco-4.2.3.tar.gz (from http://nco.sourceforge.net/#Source)
If everything looks okay, then actually import the package:


   $ tar xzf nco-4.2.3.tar.gz
   $ bcvs -q import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2
 
  <snip>
 
  N Baselibs/src/netcdf/cmake/modules/FindXDR.cmake
  N Baselibs/src/netcdf/cmake/modules/windows/FindHDF5.cmake
    
    
b. Move into netcdf directory:
  205 conflicts created by this import.
  Use the following command to help the merge:
  cvs -d :ext:mathomp4@progressdirect:/cvsroot/baselibs checkout -j<prev_rel_tag> -jrelease-4_3_2 Baselibs/src/netcdf


  $ cd nco-4.2.3/
Note that this command provides helpful information on how to merge this package. You'll want to make a note of the previous release tag. The best way to find this out is via tkcvs, in my opinion.


c. Test import netcdf package
=== Merging the new package into the tree ===


   $ bcvs -nq import -kk -m 'MAT: import NCO 4.2.3 retrieved 2012-NOV-13 from http://nco.sourceforge.net/#Source' Baselibs/src/nco NCO release-4_2_3
Go back to your new Baselibs tree:
 
 
d. Really import the package 
   $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs


  $ bcvs import -kk -m 'MAT: import NCO 4.2.3 retrieved 2012-NOV-13 from http://nco.sourceforge.net/#Source' Baselibs/src/nco NCO release-4_2_3
Then, checkout/update to new release:


      39 conflicts created by this import.
  $ bcvs -nq checkout -jrelease-4_2_1_1 -jrelease-4_3_2 Baselibs/src/netcdf
      Use the following command to help the merge:


        cvs -d :ext:mathomp4@progressdirect:/cvsroot/baselibs checkout -j<prev_rel_tag> -jrelease-4_2_3 Baselibs/src/nco
and remove -n and run again once satisfied.


e. Check out model anew
Move into the new package directory:


   $ cd /discover/swdev/mathomp4/Baselibs
   $ cd Baselibs/src/netcdf
  $ mkdir NewBaselibs
  $ cd NewBaselibs
  $ bcvs co -kk Baselibs/src
      <snip checkout>
  $ bcvs -q checkout -kk Baselibs/src/nco


f. Update to new release
and check for conflicts:


  $ bcvs -nq update -kk -jrel-4_0_3-20100627 -jrelease-4_2_3 Baselibs/src/nco
<nowiki>$ bcvs -nq upd -kk | grep -r '^[C]'
C COPYRIGHT
C INSTALL
C Makefile.am
C Makefile.in
C aclocal.m4
C config.h.in
C configure
C configure.ac
C nc-config.in
C examples/Makefile.am
C examples/Makefile.in
C examples/C/Makefile.in
C examples/CDL/Makefile.in
C libsrc/Makefile.am
C libsrc/Makefile.in
C libsrc/ncx.c
C libsrc/ncx.m4
C libsrc/posixio.c
C libsrc/v1hpg.c
C libsrc4/Makefile.am
C libsrc4/Makefile.in
C libsrc4/nc4file.c
C libsrc4/nc4hdf.c
C libsrc4/nc4internal.c
C libsrc4/nc4var.c
C man4/Makefile.am
C man4/Makefile.in
C nc_test/Makefile.am
C nc_test/Makefile.in
C nc_test/large_files.c
C nc_test4/Makefile.am
C nc_test4/Makefile.in
C ncdap_test/Makefile.am
C ncdap_test/Makefile.in
C ncdap_test/test_cvt.c
C ncdap_test/test_varm3.c
C ncdap_test/tst_ncdap3.sh
C ncdap_test/tst_ncdap_shared.sh
C ncdap_test/tst_remote.sh
C ncdap_test/expected3/123.nc.dmp
C ncdap_test/expected3/123bears.nc.dmp
C ncdap_test/expected3/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected3/Makefile.in
C ncdap_test/expected3/bears.nc.dmp
C ncdap_test/expected3/data.nc.dmp
C ncdap_test/expected3/fnoc1.nc.dmp
C ncdap_test/expected3/in.nc.dmp
C ncdap_test/expected3/in1.nc.dmp
C ncdap_test/expected3/in_2.nc.dmp
C ncdap_test/expected3/in_no_three_double_dmn.nc.dmp
C ncdap_test/expected3/in_v.nc.dmp
C ncdap_test/expected3/synth1.dmp
C ncdap_test/expected3/test.PointFile.dmp
C ncdap_test/expected3/test.dfr1.dmp
C ncdap_test/expected3/test.dfr2.dmp
C ncdap_test/expected3/test.dfr3.dmp
C ncdap_test/expected3/test.nc.dmp
C ncdap_test/expected3/text.nc.dmp
C ncdap_test/expected4/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected4/Makefile.in
C ncdap_test/expected4/test.PointFile.dmp
C ncdap_test/expected4/test.dfr1.dmp
C ncdap_test/expected4/test.dfr2.dmp
C ncdap_test/expected4/test.dfr3.dmp
C ncdap_test/expected4/test.sds6.dmp
C ncdap_test/expected4/test.sds7.dmp
C ncdap_test/expectremote3/Makefile.am
C ncdap_test/expectremote3/Makefile.in
C ncdap_test/expectremote4/Makefile.in
C ncdap_test/testdata3/Makefile.in
C ncdump/Makefile.am
C ncdump/Makefile.in
C ncdump/dumplib.c
C ncdump/dumplib.h
C ncdump/nccopy.1
C ncdump/nccopy.c
C ncdump/ncdump.c
C ncdump/ncdump.h
C ncdump/nciter.c
C ncdump/nciter.h
C ncdump/tst_nccopy4.sh
C ncdump/tst_ncgen4_shared.sh
C ncdump/tst_netcdf4.sh
C ncdump/tst_netcdf4_4.sh
C ncdump/vardata.c
C ncdump/vardata.h
C ncdump/cdl4/Makefile.am
C ncdump/cdl4/Makefile.in
C ncdump/expected4/Makefile.am
C ncdump/expected4/Makefile.in
C ncgen/ConvertUTF.c
C ncgen/Makefile.am
C ncgen/Makefile.in
C ncgen/bytebuffer.c
C ncgen/cdata.c
C ncgen/data.h
C ncgen/debug.h
C ncgen/escapes.c
C ncgen/f77data.c
C ncgen/genbin.c
C ncgen/genc.c
C ncgen/genchar.c
C ncgen/genf77.c
C ncgen/genj.c
C ncgen/genlib.h
C ncgen/getfill.c
C ncgen/list.c
C ncgen/main.c
C ncgen/ncgen.l
C ncgen/ncgen.y
C ncgen/ncgentab.c
C ncgen/ncgenyy.c
C ncgen/odom.c
C ncgen/semantics.c
C ncgen/util.c
C ncgen3/Makefile.am
C ncgen3/Makefile.in
C nctest/Makefile.am
C nctest/Makefile.in
C nctest/driver.c</nowiki>


  (remove -nq and run again once satisfied)
 
g. Move into nco


  $ cd Baselibs/src/nco
Note in this case that many conflicts were found. If there were conflicts, you should resolve them. When in doubt, I suggest always using '''only''' what the new package contained. The only exceptions might be in Makefiles or other files that might have been edited by Denis or other maintainers to get it to build with the other Baselibs. But, if at all possible, go with the stock library.
 
h. Check for conflicts


  $ bcvs -nq upd -kk | grep -r '^[C|M]'TmpBaselibs
Once all is good, commit the changes:


   cvs update: New directory `autom4te.cache' -- ignored
   $ bcvs commit -m "MAT: merge new netCDF-C 4.3.2 import, many conflicts resolved"
  cvs update: New directory `debian' -- ignored
  cvs update: New directory `src/nco++/.deps' -- ignored
  M doc/ANNOUNCE
  M doc/ChangeLog
  M doc/MANIFEST
  M doc/Makefile.in
  M doc/README
  M doc/TODO
  M doc/VERSION
  M doc/debian.txt
  M doc/i18n.txt
  M doc/index.shtml
  M doc/man_end.txt
  M doc/nco.dvi
  M doc/nco.html
  M doc/nco.info
  M doc/nco.pdf
  M doc/nco.ps
  M doc/nco.texi
  M doc/nco.txt
  M doc/nco.xml
  M doc/nco_news.shtml
  M doc/nco_src_frg.txt
  M doc/opendap.sh
  M man/Makefile.in
  M man/ncap.1
  M man/ncap2.1
  M man/ncatted.1
  M man/ncbo.1
  M man/ncdiff.1
  M man/ncea.1
  M man/ncecat.1
  M man/ncflint.1
  M man/ncks.1
  M man/nco.1
  M man/ncpdq.1
  M man/ncra.1
  M man/ncrcat.1
  M man/ncrename.1
  M man/ncwa.1


i. Fix conflicts
=== Add additional packages ===


  No conflicts found.
Repeat the above steps for any other packages you want to update as well.
j. Commit changes


  $ bcvs commit -m "MAT: merge new NCO release 4.2.3, no conflicts found"
=== Tagging your new release ===


-----------
To make your new tag, first do a relative tag at the <tt>src/</tt> level:
 
5. Tag new release at src level


   $ cd ..
   $ cd ..
   $ bcvs tag -r GMAO-Baselibs-3_3_0 Internal-3_3_1
   $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6
 
6. UNTAG nco and retag the new vendor release


  $ cd nco/
Then, for safety's sake, '''untag''' the new library and retag the against the new vendor release tag:
  $ bcvs tag -d Internal-3_3_1
  $ bcvs tag -r release-4_2_3 Internal-3_3_1


7. Edit Changelog and VERSION and commit
  $ cd netcdf/
  $ bcvs tag -d Internal-4_0_6
  $ bcvs tag -r release-4_3_2 Internal-4_0_6


8. Push tag on new files
Next, edit the usual metafiles (<tt>Changelog</tt>, <tt>CONTENTS</tt>, and <tt>VERSION</tt>) as well as any needed make system changes and commit those on the HEAD. Then, push your new tag on those files:


   $ bcvs tag -F Internal-3_3_1 ChangeLog INSTALL VERSION
   $ bcvs tag -F Internal-4_0_6 ChangeLog CONTENTS VERSION


11. Checkout new tag and build
=== Verify new tag ===


  a. Try doing a cvscmp to the "before" tag. Make sure the difference
The final step is to checkout the new tag and build the Baselibs per usual. My first step is to usually do a <tt>cvscmp</tt> against the old tag to make sure everything looks sane. I then like to do a <tt>diff -r</tt> between the updated library and the stock package that was downloaded from the web. Hopefully, all is well.
      looks okay


12. If all works, delete the "before" tag
If everything works, you can then delete your "before" tag and if all '''really''' works, rename the <tt>Internal</tt> tag with an official <tt>GMAO-Baselibs</tt> tag.


13. If all *really* works, rename Internal-3_3_1 to GMAO-Baselibs-3_3_1
[[Category:Baselibs|Updating]]
[[Category:SI Team]]

Latest revision as of 04:45, 20 August 2014

Process of Updating A New Component to Baselibs

This example document will detail the process that was used updating netCDF 4.3.2 into GMAO Baselibs.

Set up the new tag

In this case, we were updating netCDF-C 4.3.2 to GMAO-Baselibs-4_0_5. So, the "internal" tag we went to was 4_0_6. In a temporary directory (say, NewBaselibs) you checkout Baselibs and tag relative so you know where you started from. You also make sure the tag you're making doesn't exist: 

  $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs
  $ bcvs co -kk Baselibs
  $ cd Baselibs/src
  $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6-before-netCDF-4_3_2-NCO-4_4_4-cURL-7_37_0
  $ bcvs tag -d Internal-4_0_6

Unpack the new library

First, move to a directory that can contain your "new import", 

  $ cd /discover/swdev/mathomp4/Baselibs/NewImports

untar the package, 

  $ tar xzf netcdf-c-4.3.2.tar.gz

and move into the new directory directory: 

  $ cd netcdf-c-4.3.2

Import the new package

First, do a dry-run import of the package (note the -nq): 

  $ bcvs -nq import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2

If everything looks okay, then actually import the package: 

  $ bcvs -q import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2
  
  <snip>
  
  N Baselibs/src/netcdf/cmake/modules/FindXDR.cmake
  N Baselibs/src/netcdf/cmake/modules/windows/FindHDF5.cmake
  
  205 conflicts created by this import.
  Use the following command to help the merge:

 	cvs -d :ext:mathomp4@progressdirect:/cvsroot/baselibs checkout -j<prev_rel_tag> -jrelease-4_3_2 Baselibs/src/netcdf

Note that this command provides helpful information on how to merge this package. You'll want to make a note of the previous release tag. The best way to find this out is via tkcvs, in my opinion.

Merging the new package into the tree

Go back to your new Baselibs tree: 

  $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs

Then, checkout/update to new release: 

  $ bcvs -nq checkout -jrelease-4_2_1_1 -jrelease-4_3_2 Baselibs/src/netcdf

and remove -n and run again once satisfied.

Move into the new package directory: 

  $ cd Baselibs/src/netcdf

and check for conflicts: 

$ bcvs -nq upd -kk | grep -r '^[C]'
C COPYRIGHT
C INSTALL
C Makefile.am
C Makefile.in
C aclocal.m4
C config.h.in
C configure
C configure.ac
C nc-config.in
C examples/Makefile.am
C examples/Makefile.in
C examples/C/Makefile.in
C examples/CDL/Makefile.in
C libsrc/Makefile.am
C libsrc/Makefile.in
C libsrc/ncx.c
C libsrc/ncx.m4
C libsrc/posixio.c
C libsrc/v1hpg.c
C libsrc4/Makefile.am
C libsrc4/Makefile.in
C libsrc4/nc4file.c
C libsrc4/nc4hdf.c
C libsrc4/nc4internal.c
C libsrc4/nc4var.c
C man4/Makefile.am
C man4/Makefile.in
C nc_test/Makefile.am
C nc_test/Makefile.in
C nc_test/large_files.c
C nc_test4/Makefile.am
C nc_test4/Makefile.in
C ncdap_test/Makefile.am
C ncdap_test/Makefile.in
C ncdap_test/test_cvt.c
C ncdap_test/test_varm3.c
C ncdap_test/tst_ncdap3.sh
C ncdap_test/tst_ncdap_shared.sh
C ncdap_test/tst_remote.sh
C ncdap_test/expected3/123.nc.dmp
C ncdap_test/expected3/123bears.nc.dmp
C ncdap_test/expected3/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected3/Makefile.in
C ncdap_test/expected3/bears.nc.dmp
C ncdap_test/expected3/data.nc.dmp
C ncdap_test/expected3/fnoc1.nc.dmp
C ncdap_test/expected3/in.nc.dmp
C ncdap_test/expected3/in1.nc.dmp
C ncdap_test/expected3/in_2.nc.dmp
C ncdap_test/expected3/in_no_three_double_dmn.nc.dmp
C ncdap_test/expected3/in_v.nc.dmp
C ncdap_test/expected3/synth1.dmp
C ncdap_test/expected3/test.PointFile.dmp
C ncdap_test/expected3/test.dfr1.dmp
C ncdap_test/expected3/test.dfr2.dmp
C ncdap_test/expected3/test.dfr3.dmp
C ncdap_test/expected3/test.nc.dmp
C ncdap_test/expected3/text.nc.dmp
C ncdap_test/expected4/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected4/Makefile.in
C ncdap_test/expected4/test.PointFile.dmp
C ncdap_test/expected4/test.dfr1.dmp
C ncdap_test/expected4/test.dfr2.dmp
C ncdap_test/expected4/test.dfr3.dmp
C ncdap_test/expected4/test.sds6.dmp
C ncdap_test/expected4/test.sds7.dmp
C ncdap_test/expectremote3/Makefile.am
C ncdap_test/expectremote3/Makefile.in
C ncdap_test/expectremote4/Makefile.in
C ncdap_test/testdata3/Makefile.in
C ncdump/Makefile.am
C ncdump/Makefile.in
C ncdump/dumplib.c
C ncdump/dumplib.h
C ncdump/nccopy.1
C ncdump/nccopy.c
C ncdump/ncdump.c
C ncdump/ncdump.h
C ncdump/nciter.c
C ncdump/nciter.h
C ncdump/tst_nccopy4.sh
C ncdump/tst_ncgen4_shared.sh
C ncdump/tst_netcdf4.sh
C ncdump/tst_netcdf4_4.sh
C ncdump/vardata.c
C ncdump/vardata.h
C ncdump/cdl4/Makefile.am
C ncdump/cdl4/Makefile.in
C ncdump/expected4/Makefile.am
C ncdump/expected4/Makefile.in
C ncgen/ConvertUTF.c
C ncgen/Makefile.am
C ncgen/Makefile.in
C ncgen/bytebuffer.c
C ncgen/cdata.c
C ncgen/data.h
C ncgen/debug.h
C ncgen/escapes.c
C ncgen/f77data.c
C ncgen/genbin.c
C ncgen/genc.c
C ncgen/genchar.c
C ncgen/genf77.c
C ncgen/genj.c
C ncgen/genlib.h
C ncgen/getfill.c
C ncgen/list.c
C ncgen/main.c
C ncgen/ncgen.l
C ncgen/ncgen.y
C ncgen/ncgentab.c
C ncgen/ncgenyy.c
C ncgen/odom.c
C ncgen/semantics.c
C ncgen/util.c
C ncgen3/Makefile.am
C ncgen3/Makefile.in
C nctest/Makefile.am
C nctest/Makefile.in
C nctest/driver.c


Note in this case that many conflicts were found. If there were conflicts, you should resolve them. When in doubt, I suggest always using only what the new package contained. The only exceptions might be in Makefiles or other files that might have been edited by Denis or other maintainers to get it to build with the other Baselibs. But, if at all possible, go with the stock library.

Once all is good, commit the changes: 

  $ bcvs commit -m "MAT: merge new netCDF-C 4.3.2 import, many conflicts resolved"

Add additional packages

Repeat the above steps for any other packages you want to update as well.

Tagging your new release

To make your new tag, first do a relative tag at the src/ level: 

  $ cd ..
  $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6

Then, for safety's sake, untag the new library and retag the against the new vendor release tag: 

  $ cd netcdf/
  $ bcvs tag -d Internal-4_0_6
  $ bcvs tag -r release-4_3_2 Internal-4_0_6

Next, edit the usual metafiles (Changelog, CONTENTS, and VERSION) as well as any needed make system changes and commit those on the HEAD. Then, push your new tag on those files: 

  $ bcvs tag -F Internal-4_0_6 ChangeLog CONTENTS VERSION

Verify new tag

The final step is to checkout the new tag and build the Baselibs per usual. My first step is to usually do a cvscmp against the old tag to make sure everything looks sane. I then like to do a diff -r between the updated library and the stock package that was downloaded from the web. Hopefully, all is well.

If everything works, you can then delete your "before" tag and if all really works, rename the Internal tag with an official GMAO-Baselibs tag.

Retrieved from "https://geos5.org/wiki/index.php?title=Updating_a_Baselibs_Component&oldid=2560"