Merging and Updating the SVN Repository For Icarus
This page will detail information about the updating SVN repository for Heracles
Creating new branch
Copy trunk to new branch
First, svn copy the trunk (or another branch) into a new branch:
svn copy https://geos5.org/svn/trunk https://geos5.org/svn/branches/Heracles-4_3 -m "geos: created Heracles-4_3 branch"
Note that this is just a copy, so that means the Heracles-4_3 branch isn't actually Heracles-4_3 yet.
Checkout SVN version of new branch
cd /path/to/svn/directory ls branches/ tags/ trunk/ cd branches/ svn checkout https://geos5.org/svn/branches/Heracles-4_3
Copy new branch to a SAVE directory (optional)
This is a bit "belt-and-suspenders", but to be sure you don't muck up the "merge" from CVS to SVN, I made a copy of the new branch:
cp -a Heracles-4_3 Heracles-4_3-SAVE
Checkout a CVS copy of the new branch
The next step is to check out the tag that will become the new branch:
cd /path/to/cvs/directory mkdir Heracles-4_3 cd Heracles-4_3 cvs -Q co -r Heracles-4_3 GEOSagcm
Compare CVS to previous revision
This isn't really necessary, but it's a good step. Run a cvscmp on this checkout:
$ cvscmp Heracles-4_2
This will give you a good idea of what changed and how many differences the SVN commit should be committing.
Apply copyright to CVS code
Next, in the CVS branch, run the HandleCopyright.py script:
cd /path/to/cvs/directory/Heracles-4_3/GEOSagcm/src HandleCopyright.py -p .
At the moment, we *think* the copyright script correctly handles all files, but the proof will be when you try and build the "copyrighted" code. The files that have the greatest chance of getting screwed up are .h files since some in the GEOS-5 GCM are .h files that contain C code (mainly in GFDL_fms) and most are .h files that contain Fortran code.
Strip copyright from Copyrighted Files
I've created a script in:
~mathomp4/bin/stripCopyright.bash
that automatically removes the copyright notice from directories that are external (GFDL_fms, RRTMG, NCEP_Shared, LANL_Shared)
Copy the CVS Code into the SVN directory
Using plain ol' cp, copy the copyrighted CVS into the SVN directory:
cd /path/to/svn/directory/branches/Heracles-4_3/src cp -rv /path/to/cvs/directory/Heracles-4_3/GEOSagcm/src/* .
Stub out nh_core.F90 (optional)
NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE
If needed, we must "stub out" nh_core.F90 in the FV3 code on the SVN if it is an unreleased version. If this is true, make sure there are only the interfaces and nothing else.
$ vi GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp/FVdycoreCubed_GridComp/fvdycore/model/nh_core.F90
Now check to make sure no code is there!
Remove the CVS directories
Now remove the CVS directories using find:
find . -type d -name 'CVS' -exec rm -rf {} +
Add the files that are new to the branch
You can see all the files that must be added by doing a:
cd /path/to/svn/directory/branches/Heracles-4_3/src svn status | grep "^?"
What we need to do is add those.
Remove Giant Data Files
Note: Before blindly adding all new files, some big tarballs from the Surface Shared directory might be there as well as some large NC4 files from LANL_cice. For sanity's sake, you should 'rm' or 'svn delete' these from the branch:
$ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/120x65/grid_spec.nc.gz $ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/360x200/grid_spec.nc.gz $ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/720x410/grid_spec.nc.gz $ rm -rf GMAO_Shared/LANL_Shared/LANL_cice/input_templates
Or perhaps:
$ find . -iname '*.nc' -exec rm {} +
$ find . -iname '*.nc.gz' -exec rm {} +
Automated add
Running:
svn add `svn status | grep "^?" | awk '{print $2}'`
will add all the un-added files. It's good to make sure they are expected.
Do an xxdiff to make sure
At this point, you can do an xxdiff between the CVS and the SVN checkout:
$ xxdiff /path/to/svn/directory/branches/Heracles-4_3/src /path/to/cvs/directory/Heracles-4_3/GEOSagcm/src
If all is well there should be either 0 differences or 1 difference (the grid_spec files above. If there are more differences, that usually means you need to svn delete files that were removed between versions of the model. Go through these one-by-one until the only difference is grid_spec or none at all.
Once more: CHECK nh_core.F90
CHECK IT!
Do a diff between CVS and svn (optional)
If you like, you can do the above copyright work on the CVS checkout and then do a diff. If you did things right, you should only see diffs with the MOM tarballs above:
$ diff -rq --exclude='.svn' --exclude='CVS' /path/to/svn/directory/branches/Heracles-4_3/ /path/to/cvs/directory/Heracles-4_3/GEOSagcm/
Commit the new branch
$ svn commit -m "geos: Update to code from Heracles-4_3 CVS checkout."
Test compile the new branch
Now make sure the branch can actually be built on discover:
$ cd /path/to/model/directory $ mkdir Heracles-4_3-SVN $ cd Heracles-4_3-SVN $ svn checkout https://geos5.org/svn/branches/Heracles-4_3 GEOSagcm $ cd GEOSagcm/src $ make install |& tee make.log
If there are issues, fix them. Most likely ones are g5_modules isn't quite right, or a copyright was put in in the wrong format. Make note of the latter issues since the HandleCopyright.py script needs to handle them correctly.
Make sure to commit all changes to the SVN repo.
Merge latest code to trunk without reintegrate (if not a feature branch)
Once the code builds, if it's not a feature branch, but the latest code, this should be merged to trunk. In this case, we're going to use old-style merges. In order to do so, we need to know the revision we copied from the trunk, and the revision that is latest on our branch:
$ cd /path/to/svn/directory/branches/Heracles-4_3 $ svn update Updating '.': At revision 369. $ svn log --stop-on-copy ------------------------------------------------------------------------ r369 | geos | 2016-01-19 12:14:54 -0500 (Tue, 19 Jan 2016) | 1 line geos: Remove large binary files from LANL_cice. ------------------------------------------------------------------------ r368 | geos | 2016-01-19 12:13:59 -0500 (Tue, 19 Jan 2016) | 1 line geos: Update to code from Heracles-4_3 CVS checkout. ------------------------------------------------------------------------ r367 | geos | 2016-01-19 11:49:45 -0500 (Tue, 19 Jan 2016) | 1 line geos: created Heracles-4_3 branch ------------------------------------------------------------------------
Note the value of the revision that was on trunk when we branched. Now we need to do the merge on the trunk and take all updates since that on our branch:
$ cd /path/to/svn/directory/trunk $ svn update At revision 369.
So, we have to merge from revision 249 to revision 293 (note, some commits were not on this branch):
$ svn merge -r367:369 https://geos5.org/svn/branches/Heracles-4_3 --- Merging r368 through r369 into '.': U src/g5_modules U src/Config/protex U src/Config/GMAO_base.mk ...snip... U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GCM_Preamble.tex U src/GEOSgcs_GridComp/GEOS_GcsGridComp.F90 U src/GEOSgcs_GridComp/GNUmakefile --- Recording mergeinfo for merge of r368 through r369 into '.': U .
Now run a diff:
$ diff -r --exclude='.svn' /path/to/svn/directory/trunk/src /path/to/svn/directory/branches/Heracles-4_3/src
That diff should so no difference.
Now we need to commit with a VERY SPECIFIC MESSAGE so that if there are any updates on the branch, we can correctly merge them to trunk.
$ svn commit -m "geos: Merged branches/Heracles-4_3 changes r367:369 into the trunk."
Now the trunk and branches/Heracles-4_3 should be identical.
$ svn diff https://geos5.org/svn/trunk https://geos5.org/svn/branches/Heracles-4_3
Create tag
svn copy https://geos5.org/svn/trunk https://geos5.org/svn/tags/Heracles-4_3 -m "geos: Create Heracles-4_3 tag"
Miscellaneous SVN Activities
Diffing Two Directories Excluding the .svn Directory
diff -r --exclude=".svn" sub/dir/one sub/dir/two