Updating a Baselibs Component: Difference between revisions

Added Category
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Process of Importing A New Component to Baselibs ==
== Process of Updating A New Component to Baselibs ==


This example document will detail the process that was used importing NCO 4.2.3 into GMAO Baselibs.
This example document will detail the process that was used updating netCDF 4.3.2 into GMAO Baselibs.


=== Set up the new tag ===
=== Set up the new tag ===


In this case, we were adding NCO 4.2.3 to GMAO-Baselibs-3_3_0. So, the "internal" tag we went to was 3_3_1. In a temporary directory (say, ImportCheckout) you checkout Baselibs and tag relative so you know where you started from. You also make sure the tag you're making doesn't exist:
In this case, we were updating netCDF-C 4.3.2 to GMAO-Baselibs-4_0_5. So, the "internal" tag we went to was 4_0_6. In a temporary directory (say, NewBaselibs) you checkout Baselibs and tag relative so you know where you started from. You also make sure the tag you're making doesn't exist:


   $ cd /gpfsm/dswdev/mathomp4/Baselibs/ImportCheckout
   $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs
   $ bcvs co Baselibs
   $ bcvs co -kk Baselibs
   $ cd Baselibs/src
   $ cd Baselibs/src
   $ bcvs tag -r GMAO-Baselibs-3_3_0 Internal-3_3_1-before-NCO-4_2_3
   $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6-before-netCDF-4_3_2-NCO-4_4_4-cURL-7_37_0
   $ bcvs tag -d Internal-3_3_1
   $ bcvs tag -d Internal-4_0_6


=== Importing NCO 4.2.3 ===
=== Unpack the new library ===


4. Move to /gpfsm/dswdev/mathomp4/Baselibs/NewImports
First, move to a directory that can contain your "new import",


   $ cd /gpfsm/dswdev/mathomp4/Baselibs/NewImports
   $ cd /discover/swdev/mathomp4/Baselibs/NewImports


a. Untar nco-4.2.3.tar.gz (from http://nco.sourceforge.net/#Source)
untar the package,


   $ tar xzf nco-4.2.3.tar.gz
   $ tar xzf netcdf-c-4.3.2.tar.gz
 
 
b. Move into netcdf directory:
and move into the new directory directory:
 
  $ cd netcdf-c-4.3.2
 
=== Import the new package ===
 
First, do a dry-run import of the package (note the <tt>-nq</tt>):


   $ cd nco-4.2.3/
   $ bcvs -nq import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2


c. Test import netcdf package
If everything looks okay, then actually import the package:


   $ bcvs -nq import -kk -m 'MAT: import NCO 4.2.3 retrieved 2012-NOV-13 from http://nco.sourceforge.net/#Source' Baselibs/src/nco NCO release-4_2_3
   $ bcvs -q import -kk -m "MAT: Import netCDF-C 4.3.2 retrieved 2014-JUN-10 from https://github.com/Unidata/netcdf-c/releases/tag/v4.3.2" Baselibs/src/netcdf Unidata release-4_3_2
 
  <snip>
 
  N Baselibs/src/netcdf/cmake/modules/FindXDR.cmake
  N Baselibs/src/netcdf/cmake/modules/windows/FindHDF5.cmake
    
    
d. Really import the package
  205 conflicts created by this import.
  Use the following command to help the merge:
  cvs -d :ext:mathomp4@progressdirect:/cvsroot/baselibs checkout -j<prev_rel_tag> -jrelease-4_3_2 Baselibs/src/netcdf
 
Note that this command provides helpful information on how to merge this package. You'll want to make a note of the previous release tag. The best way to find this out is via tkcvs, in my opinion.
 
=== Merging the new package into the tree ===


  $ bcvs import -kk -m 'MAT: import NCO 4.2.3 retrieved 2012-NOV-13 from http://nco.sourceforge.net/#Source' Baselibs/src/nco NCO release-4_2_3
Go back to your new Baselibs tree:


      39 conflicts created by this import.
  $ cd /discover/swdev/mathomp4/Baselibs/NewBaselibs
      Use the following command to help the merge:


        cvs -d :ext:mathomp4@progressdirect:/cvsroot/baselibs checkout -j<prev_rel_tag> -jrelease-4_2_3 Baselibs/src/nco
Then, checkout/update to new release:


e. Check out model anew
  $ bcvs -nq checkout -jrelease-4_2_1_1 -jrelease-4_3_2 Baselibs/src/netcdf


  $ cd /discover/swdev/mathomp4/Baselibs
and remove -n and run again once satisfied.
  $ mkdir NewBaselibs
  $ cd NewBaselibs
  $ bcvs co -kk Baselibs/src
      <snip checkout>
  $ bcvs -q checkout -kk Baselibs/src/nco


f. Update to new release
Move into the new package directory:


   $ bcvs -nq update -kk -jrel-4_0_3-20100627 -jrelease-4_2_3 Baselibs/src/nco
   $ cd Baselibs/src/netcdf


  (remove -nq and run again once satisfied)
and check for conflicts:
 
g. Move into nco


  $ cd Baselibs/src/nco
<nowiki>$ bcvs -nq upd -kk | grep -r '^[C]'
 
C COPYRIGHT
h. Check for conflicts
C INSTALL
C Makefile.am
C Makefile.in
C aclocal.m4
C config.h.in
C configure
C configure.ac
C nc-config.in
C examples/Makefile.am
C examples/Makefile.in
C examples/C/Makefile.in
C examples/CDL/Makefile.in
C libsrc/Makefile.am
C libsrc/Makefile.in
C libsrc/ncx.c
C libsrc/ncx.m4
C libsrc/posixio.c
C libsrc/v1hpg.c
C libsrc4/Makefile.am
C libsrc4/Makefile.in
C libsrc4/nc4file.c
C libsrc4/nc4hdf.c
C libsrc4/nc4internal.c
C libsrc4/nc4var.c
C man4/Makefile.am
C man4/Makefile.in
C nc_test/Makefile.am
C nc_test/Makefile.in
C nc_test/large_files.c
C nc_test4/Makefile.am
C nc_test4/Makefile.in
C ncdap_test/Makefile.am
C ncdap_test/Makefile.in
C ncdap_test/test_cvt.c
C ncdap_test/test_varm3.c
C ncdap_test/tst_ncdap3.sh
C ncdap_test/tst_ncdap_shared.sh
C ncdap_test/tst_remote.sh
C ncdap_test/expected3/123.nc.dmp
C ncdap_test/expected3/123bears.nc.dmp
C ncdap_test/expected3/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected3/Makefile.in
C ncdap_test/expected3/bears.nc.dmp
C ncdap_test/expected3/data.nc.dmp
C ncdap_test/expected3/fnoc1.nc.dmp
C ncdap_test/expected3/in.nc.dmp
C ncdap_test/expected3/in1.nc.dmp
C ncdap_test/expected3/in_2.nc.dmp
C ncdap_test/expected3/in_no_three_double_dmn.nc.dmp
C ncdap_test/expected3/in_v.nc.dmp
C ncdap_test/expected3/synth1.dmp
C ncdap_test/expected3/test.PointFile.dmp
C ncdap_test/expected3/test.dfr1.dmp
C ncdap_test/expected3/test.dfr2.dmp
C ncdap_test/expected3/test.dfr3.dmp
C ncdap_test/expected3/test.nc.dmp
C ncdap_test/expected3/text.nc.dmp
C ncdap_test/expected4/1990-S1700101.HDF.WVC_Lat.dmp
C ncdap_test/expected4/Makefile.in
C ncdap_test/expected4/test.PointFile.dmp
C ncdap_test/expected4/test.dfr1.dmp
C ncdap_test/expected4/test.dfr2.dmp
C ncdap_test/expected4/test.dfr3.dmp
C ncdap_test/expected4/test.sds6.dmp
C ncdap_test/expected4/test.sds7.dmp
C ncdap_test/expectremote3/Makefile.am
C ncdap_test/expectremote3/Makefile.in
C ncdap_test/expectremote4/Makefile.in
C ncdap_test/testdata3/Makefile.in
C ncdump/Makefile.am
C ncdump/Makefile.in
C ncdump/dumplib.c
C ncdump/dumplib.h
C ncdump/nccopy.1
C ncdump/nccopy.c
C ncdump/ncdump.c
C ncdump/ncdump.h
C ncdump/nciter.c
C ncdump/nciter.h
C ncdump/tst_nccopy4.sh
C ncdump/tst_ncgen4_shared.sh
C ncdump/tst_netcdf4.sh
C ncdump/tst_netcdf4_4.sh
C ncdump/vardata.c
C ncdump/vardata.h
C ncdump/cdl4/Makefile.am
C ncdump/cdl4/Makefile.in
C ncdump/expected4/Makefile.am
C ncdump/expected4/Makefile.in
C ncgen/ConvertUTF.c
C ncgen/Makefile.am
C ncgen/Makefile.in
C ncgen/bytebuffer.c
C ncgen/cdata.c
C ncgen/data.h
C ncgen/debug.h
C ncgen/escapes.c
C ncgen/f77data.c
C ncgen/genbin.c
C ncgen/genc.c
C ncgen/genchar.c
C ncgen/genf77.c
C ncgen/genj.c
C ncgen/genlib.h
C ncgen/getfill.c
C ncgen/list.c
C ncgen/main.c
C ncgen/ncgen.l
C ncgen/ncgen.y
C ncgen/ncgentab.c
C ncgen/ncgenyy.c
C ncgen/odom.c
C ncgen/semantics.c
C ncgen/util.c
C ncgen3/Makefile.am
C ncgen3/Makefile.in
C nctest/Makefile.am
C nctest/Makefile.in
C nctest/driver.c</nowiki>


  $ bcvs -nq upd -kk | grep -r '^[C|M]'TmpBaselibs


  cvs update: New directory `autom4te.cache' -- ignored
Note in this case that many conflicts were found. If there were conflicts, you should resolve them. When in doubt, I suggest always using '''only''' what the new package contained. The only exceptions might be in Makefiles or other files that might have been edited by Denis or other maintainers to get it to build with the other Baselibs. But, if at all possible, go with the stock library.
  cvs update: New directory `debian' -- ignored
  cvs update: New directory `src/nco++/.deps' -- ignored
  M doc/ANNOUNCE
  M doc/ChangeLog
  M doc/MANIFEST
  M doc/Makefile.in
  M doc/README
  M doc/TODO
  M doc/VERSION
  M doc/debian.txt
  M doc/i18n.txt
  M doc/index.shtml
  M doc/man_end.txt
  M doc/nco.dvi
  M doc/nco.html
  M doc/nco.info
  M doc/nco.pdf
  M doc/nco.ps
  M doc/nco.texi
  M doc/nco.txt
  M doc/nco.xml
  M doc/nco_news.shtml
  M doc/nco_src_frg.txt
  M doc/opendap.sh
  M man/Makefile.in
  M man/ncap.1
  M man/ncap2.1
  M man/ncatted.1
  M man/ncbo.1
  M man/ncdiff.1
  M man/ncea.1
  M man/ncecat.1
  M man/ncflint.1
  M man/ncks.1
  M man/nco.1
  M man/ncpdq.1
  M man/ncra.1
  M man/ncrcat.1
  M man/ncrename.1
  M man/ncwa.1


i. Fix conflicts
Once all is good, commit the changes:


   No conflicts found.
   $ bcvs commit -m "MAT: merge new netCDF-C 4.3.2 import, many conflicts resolved"
j. Commit changes


  $ bcvs commit -m "MAT: merge new NCO release 4.2.3, no conflicts found"
=== Add additional packages ===


-----------
Repeat the above steps for any other packages you want to update as well.
 
5. Tag new release at src level


  $ cd ..
=== Tagging your new release ===
  $ bcvs tag -r GMAO-Baselibs-3_3_0 Internal-3_3_1


6. UNTAG nco and retag the new vendor release
To make your new tag, first do a relative tag at the <tt>src/</tt> level:


   $ cd nco/
   $ cd ..
   $ bcvs tag -d Internal-3_3_1
   $ bcvs tag -r GMAO-Baselibs-4_0_5 Internal-4_0_6
  $ bcvs tag -r release-4_2_3 Internal-3_3_1


7. Edit Changelog and VERSION and commit
Then, for safety's sake, '''untag''' the new library and retag the against the new vendor release tag:


8. Push tag on new files
  $ cd netcdf/
  $ bcvs tag -d Internal-4_0_6
  $ bcvs tag -r release-4_3_2 Internal-4_0_6


  $ bcvs tag -F Internal-3_3_1 ChangeLog INSTALL VERSION
Next, edit the usual metafiles (<tt>Changelog</tt>, <tt>CONTENTS</tt>, and <tt>VERSION</tt>) as well as any needed make system changes and commit those on the HEAD. Then, push your new tag on those files:


11. Checkout new tag and build
  $ bcvs tag -F Internal-4_0_6 ChangeLog CONTENTS VERSION


  a. Try doing a cvscmp to the "before" tag. Make sure the difference
=== Verify new tag ===
      looks okay


12. If all works, delete the "before" tag
The final step is to checkout the new tag and build the Baselibs per usual. My first step is to usually do a <tt>cvscmp</tt> against the old tag to make sure everything looks sane. I then like to do a <tt>diff -r</tt> between the updated library and the stock package that was downloaded from the web. Hopefully, all is well.


13. If all *really* works, rename Internal-3_3_1 to GMAO-Baselibs-3_3_1
If everything works, you can then delete your "before" tag and if all '''really''' works, rename the <tt>Internal</tt> tag with an official <tt>GMAO-Baselibs</tt> tag.


[[Category:Baselibs|Updating]]
[[Category:Baselibs|Updating]]
[[Category:SI Team]]