Deploying a SmartSite Prod Release

Owned by Thomas Amsler
Last updated: Oct 05, 2009 by AnonymousVersion comment
2 min read

Information

Once a prod tag has been made here, it should be deployed.

As of this writing, smartsite-prod has six nodes:

  • sakaiprod1 (headnode)
  • sakaiprod2
  • sakaiprod3
  • sakaiprod4
  • sakaiprod5
  • sakaiprod6
  • sakaiprodmail1

Preparing Maintenance Documentation

  • The production maintenance information is documented here:
    • /wiki/spaces/UCDSAKAI/pages/120456679
    • This confluence page is reference in an information email, which is described in the next section.
    • Please follow the existing documentation outline/pattern.

Communicating Maintenance Information

  • Preferably on Friday before the Tuesday maintenance but no later than 8:00 a.m. the Monday before activate (or have someone activate) the regular maintenance schedule MOTD and set at the bottom the date for the next maintenance time. Patch weekend MOTD should be posted 3-4 days before the weekend.
  • On Monday send an email to: sakai-releasemgt@ucdavis.edu 
    • Email Subject:
    SmartSite Maintenance : May 06, 2008 : 05:30 to 07:00

Email Body:
    Hello,
    We are performing scheduled maintenance on SmartSite on May 06, 2008 from 05:30 to 07:00. For more information, please see:
    https://confluence.ucdavis.edu:8443/confluence/x/URM
    Best,
    <Your Name>

  • On Tuesday, after you have finished maintenance, reply to the above email with the following message: 
    • Email Body:
    Hello,
    We have finished maintenance on SmartSite. Users will be able to access SmartSite after the maintenance window ends at 07:00.
    Best,
    <Your Name>

Setup of deployment

  • If deploying a staging release, follow the instructions here instead of the ones in this section. Resume work in the Deployment section after the special instructions are complete.
  • Prior to deployment, run as yourself (not sakai):
sudo /ucd/local/sbin/set_status.sh maintenance


  • In ~sakai/src, make sure to clear the oldest tag before checkout/building. We only have space to support two tags(current and next most recent)
  • Start the deployment by building the prod tag as follows on the headnode:
  1. ~/config-staging/tomcatclean.sh


  2. cd ~/src


  3. svn co https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-XYZ


  4. cd sakai_2-5-x-prod-XYZ


  5. patches/scripts/apply-patches.pl


  6. mvn clean install sakai:deploy -P with-jsmath

  • Next step is to deploy the sakai.properties tag in the staging area
  1. cd $HOME/config-staging

  2. Delete any old sakai.properties
  3. svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-XYZ/sakai.properties
svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-XYZ/jameskiller_whitelist.dat

Deployment

  • Stop all nodes in the cluster by ~/bin/stop.sh and make sure all tomcat processes are dead (eg ps -ef | grep java| grep sakai)
  • On each node do(starting with the head node):
  1. cd $HOME


  2. ./config-staging/setup-sakai-node.sh


  3. cd $CATALINA_HOME/logs


  4. ~/bin/start.sh ; tail -f catalina.out

  5. Once sakai is fully up, test login and continue with the next node

After Deployment

  • Make sure all JIRAs associated with the tag have a fix rev of the tag, and that they are all in proper state.
  • Place each server back into normal mode:
sudo /ucd/local/sbin/set_status.sh normal