Merging and Updating the SVN Repository For Icarus
This page will detail information about the updating SVN repository for Icarus
Creating new branch
Copy trunk to new branch
First, svn copy the trunk (or another branch) into a new branch:
svn copy https://geos5.org/svn/trunk https://geos5.org/svn/branches/Icarus-2_0 -m "geos: created Icarus-2_0 branch"
Note that this is just a copy, so that means the Icarus-2_0 branch isn't actually Icarus-2_0 yet.
Checkout SVN version of new branch
cd /path/to/svn/directory ls branches/ tags/ trunk/ cd branches/ svn checkout https://geos5.org/svn/branches/Icarus-2_0
Copy new branch to a SAVE directory (optional)
This is a bit "belt-and-suspenders", but to be sure you don't muck up the "merge" from CVS to SVN, I made a copy of the new branch:
cp -a Icarus-2_0 Icarus-2_0-SAVE
Checkout a CVS copy of the new branch
The next step is to check out the tag that will become the new branch:
cd /path/to/cvs/directory mkdir Icarus-2_0 cd Icarus-2_0 cvs -Q co -r Icarus-2_0 GEOSagcm
Compare CVS to previous revision
This isn't really necessary, but it's a good step. Run a cvscmp on this checkout:
$ cvscmp Icarus-1_0_p1
This will give you a good idea of what changed and how many differences the SVN commit should be committing.
Apply copyright to CVS code
Next, in the CVS branch, run the HandleCopyright.py script:
cd /path/to/cvs/directory/Icarus-2_0/GEOSagcm/src HandleCopyright.py -p .
At the moment, we *think* the copyright script correctly handles all files, but the proof will be when you try and build the "copyrighted" code. The files that have the greatest chance of getting screwed up are .h files since some in the GEOS-5 GCM are .h files that contain C code (mainly in GFDL_fms) and most are .h files that contain Fortran code.
Strip copyright from Copyrighted Files
I've created a script in:
~mathomp4/bin/stripCopyright.bash
that automatically removes the copyright notice from directories that are external (GFDL_fms, RRTMG, NCEP_Shared, LANL_Shared)
Copy the CVS Code into the SVN directory
Using plain ol' cp, copy the copyrighted CVS into the SVN directory:
cd /path/to/svn/directory/branches/Icarus-2_0/src cp -rv /path/to/cvs/directory/Icarus-2_0/GEOSagcm/src/* .
Stub out nh_core.F90 (optional)
NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE
If needed, we must "stub out" nh_core.F90 in the FV3 code on the SVN if it is an unreleased version. If this is true, make sure there are only the interfaces and nothing else.
$ vi GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp/FVdycoreCubed_GridComp/fvdycore/model/nh_core.F90
Now check to make sure no code is there!
Remove the CVS directories
Now remove the CVS directories using find:
find . -type d -name 'CVS' -exec rm -rf {} +
Add the files that are new to the branch
You can see all the files that must be added by doing a:
cd /path/to/svn/directory/branches/Icarus-2_0/src svn status | grep "^?"
What we need to do is add those.
Remove Giant Data Files
Note: Before blindly adding all new files, some big tarballs from the Surface Shared directory might be there as well as some large NC4 files from LANL_cice. For sanity's sake, you should 'rm' or 'svn delete' these from the branch:
$ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/120x65/grid_spec.nc.gz $ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/360x200/grid_spec.nc.gz $ rm GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/MOM/720x410/grid_spec.nc.gz $ rm -rf GMAO_Shared/LANL_Shared/LANL_cice/input_templates
Or perhaps:
find . -iname '*.nc' -exec rm {} +
find . -iname '*.nc.gz' -exec rm {} +
rm -rf GMAO_Shared/LANL_Shared/LANL_cice/input_templates
Automated add
Running:
svn add `svn status | grep "^?" | awk '{print $2}'`
will add all the un-added files. It's good to make sure they are expected.
Do an xxdiff to make sure
At this point, you can do an xxdiff between the CVS and the SVN checkout:
$ xxdiff /path/to/svn/directory/branches/Icarus-2_0/src /path/to/cvs/directory/Icarus-2_0/GEOSagcm/src
If all is well there should be either 0 differences or 1 difference (the grid_spec files above. If there are more differences, that usually means you need to svn delete files that were removed between versions of the model. Go through these one-by-one until the only difference is grid_spec or none at all.
Once more: CHECK nh_core.F90
CHECK IT!
Do a diff between CVS and svn (optional)
If you like, you can do the above copyright work on the CVS checkout and then do a diff. If you did things right, you should only see diffs with the MOM tarballs above:
$ diff -rq --exclude='.svn' --exclude='CVS' --exclude='*.nc' /path/to/svn/directory/branches/Icarus-2_0/ /path/to/cvs/directory/Icarus-2_0/GEOSagcm/
Commit the new branch
$ svn commit -m "geos: Update to code from Icarus-2_0 CVS checkout."
Test compile the new branch
Now make sure the branch can actually be built on discover:
$ cd /path/to/model/directory $ mkdir Icarus-2_0-SVN $ cd Icarus-2_0-SVN $ svn checkout https://geos5.org/svn/branches/Icarus-2_0 GEOSagcm $ cd GEOSagcm/src $ make install |& tee make.log
If there are issues, fix them. Most likely ones are g5_modules isn't quite right, or a copyright was put in in the wrong format. Make note of the latter issues since the HandleCopyright.py script needs to handle them correctly.
Make sure to commit all changes to the SVN repo.
Merge latest code to trunk without reintegrate (if not a feature branch)
Once the code builds, if it's not a feature branch, but the latest code, this should be merged to trunk. In this case, we're going to use old-style merges. In order to do so, we need to know the revision we copied from the trunk, and the revision that is latest on our branch:
$ cd /path/to/svn/directory/branches/Icarus-2_0 $ svn update Updating '.': At revision 415. $ svn log --stop-on-copy ------------------------------------------------------------------------ r415 | geos | 2017-07-26 08:06:39 -0400 (Wed, 26 Jul 2017) | 1 line geos: Update to code from Icarus-2_0 CVS checkout. ------------------------------------------------------------------------ r414 | geos | 2017-07-26 07:57:26 -0400 (Wed, 26 Jul 2017) | 1 line geos: created Icarus-2_0 branch ------------------------------------------------------------------------
Note the value of the revision that was on trunk when we branched. Now we need to do the merge on the trunk and take all updates since that on our branch:
$ cd /path/to/svn/directory/trunk $ svn update At revision 415.
So, we have to merge from revision 249 to revision 293 (note, some commits were not on this branch):
$ svn merge -r414:415 https://geos5.org/svn/branches/Icarus-2_0 --- Merging r415 into '.': U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSmkiau_GridComp/GEOS_mkiauGridComp.F90 U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOS_PhysicsGridComp.F90 U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSmoist_GridComp/GEOS_MoistGridComp.F90 U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSmoist_GridComp/cloudnew.F90 U src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GEOS_ChemEnvGridComp.F90 ... U src/GMAO_Shared/GEOS_Util/plots/grads_util/stats_montage U src/GMAO_Shared/GEOS_Util/post/gcmpost.script U src/GMAO_Shared/GEOS_Util/post/GNUmakefile U src/GMAO_Shared/GEOS_Util/post/gcmclim.script --- Recording mergeinfo for merge of r415 into '.': U .
Now run a diff:
$ diff -r --exclude='.svn' /path/to/svn/directory/trunk/src /path/to/svn/directory/branches/Icarus-2_0/src
That diff should so no difference.
Now we need to commit with a VERY SPECIFIC MESSAGE so that if there are any updates on the branch, we can correctly merge them to trunk.
$ svn commit -m "geos: Merged branches/Icarus-2_0 changes r414:415 into the trunk."
Now the trunk and branches/Icarus-2_0 should be identical.
$ svn diff https://geos5.org/svn/trunk https://geos5.org/svn/branches/Icarus-2_0
Create tag
svn copy https://geos5.org/svn/trunk https://geos5.org/svn/tags/Icarus-2_0 -m "geos: Create Icarus-2_0 tag"
Miscellaneous SVN Activities
Diffing Two Directories Excluding the .svn Directory
diff -r --exclude=".svn" sub/dir/one sub/dir/two