Implementation Plan November 03, 2009

Personnel

  • Programmer is James Renfro
  • QC is Sandra

Preparation for production release

  • (Programmer) (tick) On local system: copy over sakai, gb2, and properties tags to non-release-candidate version
    svn copy https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024-RC1 https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024 -m "NOJIRA : Cutting sakai_2-5-x-prod-024 from release candidate 1"
    svn copy https://source.sakaiproject.org/contrib/gradebook2/tags/1.1.3-rc1 https://source.sakaiproject.org/contrib/gradebook2/tags/1.1.3 -m "NOJIRA : Cutting 1.1.3 from release candidate 1"
    svn copy https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024-RC1 https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024 -m "NOJIRA : Cutting sakai-properties-2-5-x-prod-024 from release candidate 1"
    
  • (Programmer) (tick) On sakaiprod1: backup sakai.properties and toolOrder.xml, export new files, checkout tag, build
    cd ~/config-staging
    ./tomcatclean.sh &
    mv sakai.properties sakai.properties-prod-023
    mv toolOrder.xml toolOrder.xml-prod-023
    svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024/sakai.properties
    svn export https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/tags/sakai-properties-2-5-x-prod-024/toolOrder.xml
    cd ~/src
    rm -rf sakai_2-5-x-prod-022 &
    svn co https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-5-x-prod-024 > ~/co.log
    ^Z
    bg
    tail -f ~/co.log
    [wait until tinyurl is checked out]
    ^C
    cd sakai_2-5-x-prod-024
    patches/scripts/apply_patches.pl all
    mvn install sakai:deploy -Pwith-jsmath > ~/build.log &
    tail -f ~/build.log
    [wait until build successful]
    ^C
    

During maintenance window

S = maintenance window + 30 mins

  • (S - 10 min Programmer) Put all application servers into maintenance mode and shutdown Sakai on each one
  • (S + 0m Programmer) Deploy sakai release tag via setup-sakai-node.sh to sakaiprod1 and restart sakai instance
    ~/config-staging/stop.sh
    jps
    [verify that sakai instance is down]
    ~/config-staging/setup-sakai-node.sh
    cd $CATALINA_HOME/logs
    ~/config-staging/start.sh ; tail -f catalina.out
    [wait until server has started up successfully]
    
  • (S + 20m QC) Verification
    • Use ANT 001, Section A01 for Open Campus students.  Check to verify student Webb Derek and Wong Rainbow are no longer identified as Open Campus on Site Info but are flagged with OC on the roster.
    • Use site UWP 101 011 SQ 2009 to observe the affect of removing removing the option for manually managed sections has on a site that currently is making use of this option.
    • Observe other sites to verify that the option to manually manage sites has been removed.
  • (S + 30m Programmer) Deploy release tag via setup-sakai-node.sh to sakaiprod2 . . . sakaiprod6, and sakaiprodmail1.
  • (S + 40m Programmer) Start Sakaiprod2 . . . sakaiprod6, and sakaiprodmail1.
  • (S + 45m Programmer) Take smartsite-prod out of maintenance mode 
  • (S + 50m Sandra) Send Release Management email

Fall-back Procedure (If needed) Time D is the decision time made to fallback

  • (D + 0m Programmer) Shutdown Sakai on smartsite-prod and clean out catalina staging
    ~/config-staging/stop.sh
    jps
    [verify that sakai instance is down]
    ~/config-staging/tomcatclean.sh
    
  • (D + 2m Programmer) On sakaiprod1 remove sakai.properties from config-staging and rename sakai.properties.023 to sakai.properties
    cd ~/config-staging
    rm sakai.properties
    rm toolOrder.xml
    cp sakai.properties-prod-023 sakai.properties
    cp toolOrder.xml-prod-023 toolOrder.xml
    
  • (D + 4m Programmer) Run mvn sakai:deploy from previous tag directory
    cd ~/src/sakai_2-5-x-prod-023
    mvn sakai:deploy
    
  • (D + 15m Programmer) Deploy previous tag on head node and restart sakai
    ~/config-staging/setup-sakai-node.sh
    cd $CATALINA_HOME/logs
    ~/config-staging/start.sh ; tail -f catalina.out
    
  • (D + 25m QC) Verify correct version of sakai is running on head node
  • (D + 40m Programmer) Run setup-sakai-node.sh and restart sakai on remaining nodes
  • (D + 50m Programmer) Take application servers out of maintenance mode