Implementation Plan November 03, 2009

Owned by James Renfro
Last updated: Nov 02, 2009Version comment

Personnel

  • Programmer is James Renfro
  • QC is Sandra

Preparation for production release

  • (Programmer) (tick) On local system: copy over sakai, gb2, and properties tags to non-release-candidate version 
    svn copy https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024-RC1 https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024 -m "NOJIRA : Cutting sakai_2-5-x-prod-024 from release candidate 1"
svn copy https://source.sakaiproject.org/contrib/gradebook2/tags/1.1.3-rc1 https://source.sakaiproject.org/contrib/gradebook2/tags/1.1.3 -m "NOJIRA : Cutting 1.1.3 from release candidate 1"
svn copy https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024-RC1 https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024 -m "NOJIRA : Cutting sakai-properties-2-5-x-prod-024 from release candidate 1"
  • (Programmer) (tick) On sakaiprod1: backup sakai.properties and toolOrder.xml, export new files, checkout tag, build 
    cd ~/config-staging
./tomcatclean.sh &
mv sakai.properties sakai.properties-prod-023
mv toolOrder.xml toolOrder.xml-prod-023
svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024/sakai.properties
svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024/toolOrder.xml
cd ~/src
rm -rf sakai_2-5-x-prod-022 &
svn co https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024 > ~/co.log
^Z
bg
tail -f ~/co.log
[wait until tinyurl is checked out]
^C
cd sakai_2-5-x-prod-024
patches/scripts/apply_patches.pl all
mvn install sakai:deploy -Pwith-jsmath > ~/build.log &
tail -f ~/build.log
[wait until build successful]
^C

During maintenance window

S = maintenance window + 30 mins

  • (S - 10 min Programmer) Put all application servers into maintenance mode and shutdown Sakai on each one
  • (S + 0m Programmer) Deploy sakai release tag via setup-sakai-node.sh to sakaiprod1 and restart sakai instance 
    ~/config-staging/stop.sh
jps
[verify that sakai instance is down]
~/config-staging/setup-sakai-node.sh
cd $CATALINA_HOME/logs
~/config-staging/start.sh ; tail -f catalina.out
[wait until server has started up successfully]
  • (S + 20m QC) Verification
    • Use ANT 001, Section A01 for Open Campus students.  Check to verify student Webb Derek and Wong Rainbow are no longer identified as Open Campus on Site Info but are flagged with OC on the roster.
    • Use site UWP 101 011 SQ 2009 to observe the affect of removing removing the option for manually managed sections has on a site that currently is making use of this option.
    • Observe other sites to verify that the option to manually manage sites has been removed.
  • (S + 30m Programmer) Deploy release tag via setup-sakai-node.sh to sakaiprod2 . . . sakaiprod6, and sakaiprodmail1.
  • (S + 40m Programmer) Start Sakaiprod2 . . . sakaiprod6, and sakaiprodmail1.
  • (S + 45m Programmer) Take smartsite-prod out of maintenance mode 
  • (S + 50m Sandra) Send Release Management email

Fall-back Procedure (If needed) Time D is the decision time made to fallback

  • (D + 0m Programmer) Shutdown Sakai on smartsite-prod and clean out catalina staging 
    ~/config-staging/stop.sh
jps
[verify that sakai instance is down]
~/config-staging/tomcatclean.sh
  • (D + 2m Programmer) On sakaiprod1 remove sakai.properties from config-staging and rename sakai.properties.023 to sakai.properties 
    cd ~/config-staging
rm sakai.properties
rm toolOrder.xml
cp sakai.properties-prod-023 sakai.properties
cp toolOrder.xml-prod-023 toolOrder.xml
  • (D + 4m Programmer) Run mvn sakai:deploy from previous tag directory 
    cd ~/src/sakai_2-5-x-prod-023
mvn sakai:deploy
  • (D + 15m Programmer) Deploy previous tag on head node and restart sakai 
    ~/config-staging/setup-sakai-node.sh
cd $CATALINA_HOME/logs
~/config-staging/start.sh ; tail -f catalina.out
  • (D + 25m QC) Verify correct version of sakai is running on head node
  • (D + 40m Programmer) Run setup-sakai-node.sh and restart sakai on remaining nodes
  • (D + 50m Programmer) Take application servers out of maintenance mode