Setting up to create an experiment page on GMAO Intranet
On this page, the steps necessary to create an experiment on the GMAO Intranet will be detailed.
If you have any issues or questions, please email the GMAO SI Team at siteam_AT_gmao.gsfc.nasa.gov
Note: On this page I will refer only to polar for clarity, but nearly all steps also apply to train. If there are any differences, I will be sure to highlight them.
Get an account on polar
The first step in this process is to get an account on polar. To do so, contact GMAO Support for an account and tell them it's to put experiments on the GMAO Intranet Model Development Page. The main thing you need is to be part of the modeling group.
Once you get an account, make sure you can ssh from discover to polar using your password.
Set up passwordless SSH from discover
In order to easily move plots from discover to polar, we need to set up passwordless SSH access to polar from discover. It's assumed that you are able to run the model on discover, thus, you have already created an RSA or DSA key on discover. We need to transfer this key to polar; to do this, we will use the ssh-copy-id utility.
On discover, do an ls of your $HOME/.ssh directory to see what you have:
$ ls ~/.ssh/*pub /home/mathomp4/.ssh/id_rsa.pub
We know you can ssh from discover to polar as tested above, now let's transfer the key to polar and see if you can do so passwordless. Run:
$ ssh-copy-id -i ~/.ssh/id_rsa.pub polar
\033[33m
\033[31m
##############################################################################
WARNING! This is a U.S. Government Computer
This U.S. Government computer is for authorized use only.
This U.S. Government computer is for authorized users only.
By accessing this system, you are consenting to complete
monitoring with no expectation of privacy.
Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.
##############################################################################
\033[32m
****************************************************************************
* Welcome to Polar *
****************************************************************************
\033[00m
mathomp4@polar's password:
Now try logging into the machine, with "ssh 'polar'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
You should now be able to ssh to polar without using a password:
$ ssh polar
\033[33m
\033[31m
##############################################################################
WARNING! This is a U.S. Government Computer
This U.S. Government computer is for authorized use only.
This U.S. Government computer is for authorized users only.
By accessing this system, you are consenting to complete
monitoring with no expectation of privacy.
Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.
##############################################################################
\033[32m
****************************************************************************
* Welcome to Polar *
****************************************************************************
\033[00m
Last login: Mon Mar 10 09:14:35 2014 from anvil.gsfc.nasa.gov
[mathomp4@polar ~]$
Set up polar for CVSACL access
Now that you are on polar, you must now set up access to CVSACL which is done differently on polar than on discover or pleiades.
Upload SSH Key to NCCS
If you haven't done so, you'll need to generate an RSA key on polar and upload them (as mentioned at the NCCS QuickStart page) to https://progress.nccs.nasa.gov/keyupload/.
Edit .ssh/config
You'll need to edit ~/.ssh/config to get access to CVSACL, but we need something before then.
On the command line get your userid:
$ id -u 3456
It (probably) won't be 3456, but that's representative. Now we edit ~/.ssh/config in your favorite editor and add:
Host cvsacl
HostName cvsacl.nccs.nasa.gov
LocalForward 53456 localhost:22223
Host ctunnel
HostName localhost
Port 53456
where you substitute your id for 3456.
Longer uids
Note, in recent times, longer ids have been being issued:
$ id -u 31128555
If you see this, just use the last four digits. Ports should be in the 5xxxx range.
Add CVSROOT Environment variable
Next we need to add some environment variables. If your shell is bash, add to .bashrc:
export CVS_RSH=ssh export CVSROOT=:ext:username@ctunnel:/cvsroot/esma
or to your .cshrc or .tcshrc:
setenv CVS_RSH ssh setenv CVSROOT :ext:username@ctunnel:/cvsroot/esma
where username is your NASA AUID.
Create a tunnel to CVSACL
The next step is to create a tunnel to CVSACL. Unlike discover or pleiades which have an "always-on" port to CVSACL, polar does not. We must create this tunnel instead. To do so, we'll use screen. Start a screen with:
$ screen
You've now created a "terminal" session inside your main one. Now connect to CVSACL with:
$ ssh cvsacl This U.S. Government resource is for authorized users only. By accessing this system you are consenting to complete monitoring with no expectation of privacy. Unauthorized access or use may subject you to disciplinary action and criminal prosecution. PASSCODE: Password: Authentication successful, but shell access here is restricted. Please see the documentation for how to use this service. If you do not know where that is, but are authorized to use this service, please contact support. This connection will terminate after 7 days if not restarted. (Press Control-C if you wish to terminate authentication early.)
You'll notice that the screen session seems to "hang". This is expected as this is now holding a port open. You now want to "detach the screen" this is done by typing Ctrl-a d (i.e., Ctrl-a and then the d key). When you do this, you'll return to your original terminal session.
Problems with CVSACL Tunnel Creation
It's possible you will get an error when you try to ssh into CVSACL. If so, the most likely cause is that you do not have an account on the server. To check this, email NCCS Support and ask them if you have an account on CVSACL and, if not, can they create one for you.
Make an experiment
Next, you'll actually make the experiment. Navigate to the experiment directory. On polar this is:
$ cd /san_agcm/geos5
and on train you go to:
$ cd /gmao/intranet/research/modeling/agcm/geos5
In this directory you'll see many, many experiments, now it's time to make your own with the makexp script. This script requires access to CVSACL--which is why we set up the screen--because you actually "check out" an experiment. In the example session below, the experiment will b
$ ./makexp
First, you'll enter the experiment ID. This is normally the name of the experiment you made on discover.
Enter the Experiment ID: EXPID
mat-G40B11-c48
-------------------------------------------------------------------------------
NASA/GMAO GEOS GCM
MAKEXP Utility
-------------------------------------------------------------------------------
Enter the Resolution Code (xnn) for the New Experiment:
where x = 'b' for ~ 2.00-degree ( 144 x 91)
'c' for ~ 1.00-degree ( 288 x 181)
'd' for ~ 0.50-degree ( 540 x 361 or 576 x 361 )
'e' for ~ 0.25-degree (1080 x 721 or 1152 x 721 )
and nn = the number of levels
b72
Enter the CONTROL expid:
mat-G40B11-c48
The CONTROL expid is used if you have done a series of experiments and you have one you wish to use as a baseline. If you have such an experiment, and you added it as a CMPEXP in plot.rc, an entry for this experiment will be created in the comparison dropdown menu on the GMAO Model Development page for your run. If you don't have a control, then you can enter anything, like the name of the experiment again.
Enter a Brief Description: Ganymed-4_0_BETA11 run at C48 Enter any KEYWORD(s) associated with Experiment: Do you wish this experiment to be 'public' or 'private'? (Default: public)
The choice of 'public' versus 'private' is just that. If you set it to 'private', the experiment will not be visible on the main GMAO Model Development page, but will be accessible under your name.
Enter Experiment Type: cli for Climatology Run (Default)
rep for DAS Replay
ana for Analysis/Verification
Enter Experiment Year: (eg: Y2001 (Default is CLIM))
Enter Month or Season for Experiment (3-character, Default = DJF):
(Options: JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC DJF JJA MAM SON ANN)
If you run a "stock" experiment with gcm_setup you'll usually use the defaults for these three items. However, if you are running an analysis or replay, the other options are there for you. Contact Larry Takacs for more information. Now the experiment will be checked out from CVS:
This U.S. Government resource is for authorized users only. By accessing this system you are consenting to complete monitoring with no expectation of privacy. Unauthorized access or use may subject you to disciplinary action and criminal prosecution. cvs checkout: Updating GEOS_Web U GEOS_Web/archive U GEOS_Web/archive_das_list U GEOS_Web/archive_list U GEOS_Web/chacc U GEOS_Web/check.jpg U GEOS_Web/current_das_list U GEOS_Web/exp_item.template ...snip... ? expdesc ? RC ? CLIM /cvsroot/esma/esma/src/Shared/GEOS_util/web/buttons/set_exps.php,v <-- buttons/set_exps.php new revision: 1.12.74.1; previous revision: 1.12 /cvsroot/esma/esma/src/Shared/GEOS_util/web/buttons/stats_exps.php,v <-- buttons/stats_exps.php new revision: 1.4.74.1; previous revision: 1.4 $
If you do an ls, you'll see your new experiment, though there are no images in it:
$ ls -ltr | tail drwxr-xr-x. 19 dao_ops users 4096 Apr 13 15:35 b5122_m2swp_jan00/ drwxr-sr-x. 12 dao_ops modeling 4096 Apr 13 18:44 e5123_tst_01/ -rw-rw-rw-. 1 takacs modeling 202652 Apr 14 13:21 intranet_ana_list drwxr-sr-x. 12 takacs modeling 4096 Apr 14 13:22 G40B10_DAS0_JAN09/ drwxr-sr-x. 12 takacs modeling 4096 Apr 14 13:22 G40B10_DAS2_JAN09/ -rw-r--r--. 1 mathomp4 modeling 96028 Apr 15 08:19 AGCM_ChangeLog drwxr-sr-x. 11 mathomp4 modeling 4096 Apr 15 08:19 mat-G40B11-c48/ -rw-rw-rw-. 1 takacs modeling 1185868 Apr 15 08:19 intranet_list -rw-rw-rw-. 1 takacs modeling 889559 Apr 15 08:19 intranet_cli_list
Note, if you see this when you run makexp:
Enter Month or Season for Experiment (3-character, Default = DJF): (Options: JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC DJF JJA MAM SON ANN) ssh: connect to host localhost port 53456: Connection refused cvs [checkout aborted]: end of file from server (consult above messages if any) /bin/mv: cannot stat `GEOS_Web': No such file or directory mat-G40B11-c48: No such file or directory.
that means you forgot to connect to CVSACL as shown above.
If you see:
cvs [checkout aborted]: cannot make directory GEOS_Web: Permission denied /bin/mv: cannot stat ‘GEOS_Web’: No such file or directory
this means that you are not a member of the modeling group. Please see the first section of this page.
Move the plots to polar
To move the plots from discover to polar, we'll use the moveplot script. Move to the directory containing the plots. In this example, the plots created were of type "CLIM" and so are in the plots_CLIM directory:
$ cd /discover/nobackup/mathomp4/mat-G40B11-c48/plots_CLIM
Now, the first time you run moveplot it will ask for information about where your experiment plots live (polar or train), and your ID on that server (usually your NASA AUID). This is only done once. After this information is set up, the plots will move.
$ $ESMADIR/src/GMAO_Shared/GEOS_Util/plots/moveplot
Enter WEB Server to use: train
polar
polar
Enter WEB ID to use on polar.gsfc.nasa.gov:
mathomp4
Hit -ENTER- to confirm: mathomp4
The authenticity of host 'polar.gsfc.nasa.gov (xxx.xxx.xxx.xxx)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'polar.gsfc.nasa.gov' (RSA) to the list of known hosts.
\033[33m
\033[31m
##############################################################################
WARNING! This is a U.S. Government Computer
This U.S. Government computer is for authorized use only.
This U.S. Government computer is for authorized users only.
By accessing this system, you are consenting to complete
monitoring with no expectation of privacy.
Unauthorized access or use may subject you to disciplinary
action and criminal prosecution.
##############################################################################
\033[32m
****************************************************************************
* Welcome to Polar *
****************************************************************************
\033[00m
Warning: No xauth data; using fake authentication data for X11 forwarding.
sending incremental file list
AGCM.rc
CAP.rc
ExtData.rc
HISTORY.rc
fvcore_layout.rc
sent 60733 bytes received 107 bytes 24336.00 bytes/sec
total size is 60385 speedup is 0.99
\033[33m
...snip...
hdiag_WESNSC.LANDICE_0.JJA.gif
hdiag_WET2.SURFACE_0.DJF.gif
hdiag_WET2.SURFACE_0.JJA.gif
hdiag_WET3.SURFACE_0.DJF.gif
hdiag_WET3.SURFACE_0.JJA.gif
sent 44871836 bytes received 5180 bytes 5983602.13 bytes/sec
total size is 44848632 speedup is 1.00
If you have any issues or questions, please email the GMAO SI Team at siteam_AT_gmao.gsfc.nasa.gov