Merging and Updating the SVN Repository From Fortuna to Ganymed
This page will detail information about the SVN repository
Creating new branch
Copy trunk to new branch
First, svn copy the trunk (or another branch) into a new branch:
svn copy http://geos5.org/svn/trunk http://geos5.org/svn/branches/Ganymed-1_0_p4 -m "geos: created Ganymed-1_0_p4 branch"
Note that this is just a copy, so that means the Ganymed-1_0_p4 branch isn't actually Ganymed-1_0_p4 yet.
Checkout SVN version of new branch
cd /path/to/svn/directory ls branches/ tags/ trunk/ cd branches/ svn checkout http://geos5.org/svn/branches/Ganymed-1_0_p4
Copy new branch to a SAVE directory (optional)
This is a bit "belt-and-suspenders", but to be sure you don't muck up the "merge" from CVS to SVN, I made a copy of the new branch:
cp -a Ganymed-1_0_p4 Ganymed-1_0_p4-SAVE
Checkout a CVS copy of the new branch
The next step is to check out the tag that will become the new branch:
cd /path/to/cvs/directory mkdir Ganymed-1_0_p4 cd Ganymed-1_0_p4 cvs co -r Ganymed-1_0_p4 Ganymed
Apply copyright to CVS code
Next, in the CVS branch, run the HandleCopyright.py script:
cd /path/to/cvs/directory/Ganymed-1_0_p4/GEOSagcm/src HandleCopyright.py -p .
At the moment, we *think* the copyright script correctly handles all files, but the proof will be when you try and build the "copyrighted" code. The files that have the greatest chance of getting screwed up are .h files since some in the GEOS-5 GCM are .h files that contain C code (mainly in GFDL_fms) and most are .h files that contain Fortran code.
Remove files that were in Fortuna but not in Ganymed
A list of these files retrieved by cvscmp Ganymed-1_0_p4:
Applications/GEOSgcm_App/fvcore_layout.rc.tmpl Applications/GEOSgcm_App/gcm_archive.j.tmpl Applications/GEOSgcm_App/gcm_plot.tmpl.tmpl Applications/GEOSgcm_App/gcm_post.j.tmpl Applications/GEOSgcm_App/gcm_regress.j.tmpl Applications/GEOSgcm_App/gcm_run.j.tmpl Applications/GEOSgcm_App/scm_util/MERRAobsys.rc Applications/GEOSgcm_App/scm_util/README_scm_MERRA Applications/GEOSgcm_App/scm_util/getter.j Applications/GEOSgcm_App/scm_util/watcher.j GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/CARMAchem_GridComp/CARMA/source/base/sulfate.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/CARMAchem_GridComp/CARMA/tests/carma_sulfategrowtest.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/CARMAchem_GridComp/CARMA/tests/read_cldicetest.pro GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/CARMAchem_GridComp/CARMA/tests/read_sulfategrowtest.pro GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/chem_update.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/grav_set_update.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/qj_update.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/qk_update.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/sad_update.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiChemistry/strat_loss.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiEmission/harvard/emiss_soil.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiEmission/ioEmission/ReadVegLaiData_mod.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/GMIchem_GridComp/GMI_GridComp/GmiEmission/lightning/emiss_lightning.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/GNUmakefile GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/SC_DataMod.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/doy.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/fixSSG.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/interp3OP.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/listClim.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/listInput.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/listRates.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/listSpecies.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/ratecal.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/rdChemFiles.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/setmcor.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSchem_GridComp/StratChem_GridComp/SC_GridComp/tables_interp.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSgwd_GridComp/geopotential.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSsatsim_GridComp/tauir.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSsatsim_GridComp/tausw.F GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSsolar_GridComp/gettau.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts/MAPL_Hash.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts/hash.c GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts/mk_lai_green_clim GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts/mk_lai_green_clim.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/data/CatchData.tar.gz GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/src/MAPL_Hash.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/src/MAPL_Sort.F90 GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/src/hash.c GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Shared/Raster/src/sort.c GMAO_Shared/GEOS_Util/plots/nettoa/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/nettoac/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/olr/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/olrc/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/olrcf/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/osr/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/osrc/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/plots/osrcf/VERIFICATION.CERES.rc GMAO_Shared/GEOS_Util/post/rs_vinterp.F GMAO_Shared/MAPL_Base/MAPL_ArthParser.F90 GMAO_Shared/MAPL_Base/MAPL_stubs2.F90
Put these in a file and run:
$ cd /path/to/svn/branches/Ganymed-1_0_p4/src
$ foreach line ( `cat ~/filesinFortunanotinGanymed.txt` )
svn delete $line
end
However, there are more that will be around as well. One way to find these is:
diff -qr --exclude='.svn' --exclude='CVS' /path/to/svn/branches/Ganymed-1_0_p4/src/ /path/to/cvs/Ganymed-1_0_p4/GEOSagcm/src. | grep Only
and then manually remove them. Tedious.
Copy the CVS Code into the SVN directory
Using plain ol' cp, copy the copyrighted CVS into the SVN directory:
cd /path/to/svn/directory/branches/Ganymed-1_0_p4/src cp -rv /path/to/cvs/directory/Ganymed-1_0_p4/GEOSagcm/src/* .
Remove the CVS directories
Now remove the CVS directories using find:
find . -type d -name 'CVS' -exec rm -rf {} +
Strip copyright from Copyrighted Files
Some files in GEOS-5 already have copyrights, so we shouldn't copyright those.
FMS
FMS has a copyright, so we can strip ours from that:
cd /path/to/svn/directory/branches/Ganymed-1_0_p4/src HandleCopyright.py -s -p GMAO_Shared/GFDL_fms/shared/
Satsim
Using grep, the files that don't have a copyright seem to be (files in bold):
$ cd GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSsatsim_GridComp/ $ HandleCopyright.py -s -p . $ grep -iL copyright * | grep -v svn actsim cmor geos5_cosp_const.F90 geos5_read.f90 GEOS_SatsimGridComp.F90 GNUmakefile llnl mcai.data quickbeam
So, we strip the copyright from all the files and add a copyright to ours:
HandleCopyright.py -s -p . HandleCopyright.py -f geos5_cosp_const.F90 HandleCopyright.py -f geos5_read.f90 HandleCopyright.py -f GEOS_SatsimGridComp.F90 HandleCopyright.py -f GNUmakefile HandleCopyright.py -f mcai.data
RRTMG
RRTMG also is external code. So remove the copyright from all the files in the directory, but then re-copyright the GNUmakefiles:
cd GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSradiation_GridComp/GEOSirrad_GridComp HandleCopyright.py -s -p RRTMG/ HandleCopyright.py -f RRTMG/GNUmakefile HandleCopyright.py -f RRTMG/rrtmg_lw/GNUmakefile HandleCopyright.py -f RRTMG/rrtmg_lw/gcm_model/GNUmakefile HandleCopyright.py -f RRTMG/rrtmg_lw/gcm_model/modules/GNUmakefile HandleCopyright.py -f RRTMG/rrtmg_lw/gcm_model/src/GNUmakefile
Stub out nh_core.F90
NOTE NOTE NOTE NOTE NOTE NOTE
We must "stub out" nh_core.F90 in the FV3 code on the SVN. To do so, we leave only the interfaces and nothing else.
$ cd GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSsuperdyn_GridComp/FVdycoreCubed_GridComp/fvdycore/model $ cp ~/nh_core.F90.stubs.forG10p4 nh_core.F90
Now check to make sure no code is there!
Add the files that are new to the branch
You can see all the files that must be added by doing a:
cd /path/to/svn/directory/branches/Ganymed-1_0_p4/src svn status | grep "^?"
What we need to do is add those:
svn add `svn status | grep "^?" | awk '{print $2}'`
Once more: CHECK nh_core.F90
CHECK IT!
Commit the new branch
svn commit -m "geos: Update to code from Ganymed-1_0_p4 CVS checkout"
Test compile the new branch
Now make sure the branch can actually be built on discover:
cd /path/to/model/directory mkdir Ganymed-1_0_p4-SVN cd Ganymed-1_0_p4-SVN svn checkout http://geos5.org/svn/branches/Ganymed-1_0_p4 GEOSagcm cd GEOSagcm/src make install |& tee make.log
If there are issues, fix them. Most likely ones are g5_modules isn't quite right, or a copyright was put in in the wrong format. Make note of the latter issues since the HandleCopyright.py script needs to handle them correctly.
Make sure to commit all changes to the SVN repo.
Ignoring unversioned files (if needed)
When you do a test build, you might find there are lots of files that appear with an svn status. To ignore them, create a file called ignore-from-svn.txt which has:
*.d *.mod *.x *.xx *___.h *___.F90 *___.rc
Then, on the src/ directory do:
cd /path/to/checkout/directory/Fortuna-2_5_p6/GEOSagcm svn propset -R svn:ignore -F ~/ignore-from-svn.txt src svn commit
This might not be needed if svn is keeping properties correctly.
Merge latest code to trunk without reintegrate (if not a feature branch)
Once the code builds, if it's not a feature branch, but the latest code, this should be merged to trunk. In this case, we're going to use old-style merges. In order to do so, we need to know the revision we copied from the trunk, and the revision that is latest on our branch:
$ cd /path/to/svn/directory/branches/Ganymed-1_0_p4
$ svn update
At revision 77.
$ svn log --stop-on-copy
------------------------------------------------------------------------
r77 | geos | 2013-01-29 13:24:01 -0500 (Tue, 29 Jan 2013) | 4 lines
geos: Strip copyright from .AGCM_VERSION
Fix HISTORY.AGCM.rc.tmpl's handling of GOCART
------------------------------------------------------------------------
r76 | geos | 2013-01-29 11:40:35 -0500 (Tue, 29 Jan 2013) | 2 lines
geos: Strip copyright from ChangeLog
------------------------------------------------------------------------
r75 | geos | 2013-01-29 11:37:59 -0500 (Tue, 29 Jan 2013) | 2 lines
geos: path_names files in GFDL_fms and FV3 can't have copyright notices
------------------------------------------------------------------------
r74 | geos | 2013-01-29 11:07:29 -0500 (Tue, 29 Jan 2013) | 1 line
geos: Update to code from Ganymed-1_0_p4 CVS checkout
------------------------------------------------------------------------
r73 | geos | 2013-01-29 10:32:05 -0500 (Tue, 29 Jan 2013) | 2 lines
geos: Remove files that were in Fortuna-2_5_p6 but are not in Ganymed-1_0_p4
------------------------------------------------------------------------
r72 | geos | 2013-01-29 09:36:57 -0500 (Tue, 29 Jan 2013) | 1 line
geos: created Ganymed-1_0_p4 branch from trunk at Fortuna-2_5_p6 (revision 71)
------------------------------------------------------------------------
Note the value of the revision that was on trunk when we branched. Now we need to do the merge on the trunk and take all updates since that on our branch:
$ cd /path/to/svn/directory/trunk $ svn update At revision 77.
So, we have to merge from revision 72 to revision 77:
$ svn merge -r 72:77 http://geos5.org/svn/branches/Ganymed-1_0_p4 --- Merging r73 through r77 into '.': U src/g5_modules U src/Config/protex U src/Config/GMAO_base.mk ...
Now we need to commit with a VERY SPECIFIC MESSAGE so that if there are any updates on the branch, we can correctly merge them to trunk.
$ svn status $ diff /path/to/svn/directory/trunk/src /path/to/svn/directory/branches/Ganymed-1_0_p4/src
$ svn commit -m "geos: Merged branches/Ganymed-1_0_p4 changes r72:77 into the trunk."
Merge branch to trunk using reintegrate
$ cd /path/to/svn/directory/trunk $ svn update At revision 78. $ svn merge --reintegrate http://geos5.org/svn/branches/Ganymed-1_0_p4@HEAD $ svn commit -m "geos: Merge branches/Ganymed-1_0_p4 into trunk using --reintegrate at revision 78"
Now restore the branch as said in the svn redbook:
$ svn delete http://geos5.org/svn/branches/Ganymed-1_0_p4 -m "Remove reintegrated branches/Ganymed-1_0_p4" Committed revision 80. $ svn copy http://geos5.org/svn/trunk http://geos5.org/svn/branches/Ganymed-1_0_p4 -m "geos: Recreate branches/Ganymed-1_0_p4 with history from trunk@HEAD" Committed revision 82.
Create tag
svn copy http://geos5.org/svn/trunk http://geos5.org/svn/tags/Ganymed-1_0_p4 -m "geos: Create Ganymed-1_0_p4 tag"
Miscellaneous SVN Activities
Diffing Two Directories Excluding the .svn Directory
diff -r --exclude=".svn" sub/dir/one sub/dir/two