Implementation Details, January 13, 2009

Implementation Details, January 13, 2009

January 13, 2009 sakai deployment

Implementation Steps:

  • (05:30) Put sakaiprod1, ..., sakaiprod6, sakaiprodmail1 into maintenance mode

  • (05:35) SAK-2467 installation

    • - On sakaiprod1, ..., sakaiprod6, sakaiprodmail1 -- Remove cron job for james monitoring script "10,25,40,55 * * * * /ucd/opt/sakai/config-staging/jameswatcher.pl" crontab -e -- Verify crontab via crontab -l - On sakaiprod1 cd $HOME/config-staging rm setup-sakai-node.sh svn -r 6558 export https://mware.ucdavis.edu/svn/ucd-sakai/deployment-scripts/branches/manual-prod-deploy/setup-sakai-node.sh svn -r 6562 export https://mware.ucdavis.edu/svn/ucd-sakai/maintenance/scripts/branches/smartsite-prod/jameskiller.pl svn export -r 6577 https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/branches/sakai_2-5-x/smartsite-prod/jameskiller_whitelist.dat - On sakaiprod1, ...,sakaiprod6, sakaiprodmail1 $HOME/config-staging/setup-sakai-node.sh nohup /ucd/opt/sakai/bin/jameskiller.pl >> /ucd/opt/sakai/logs/jameskiller-20090113.log &





 
Database Backups, other Database preparation:

  • None

Fall-back Procedure (If needed)

  • Kill jameskiller.pl process on sakaiprod1, ..., sakaiprod6, sakaiprodmail1

QA Test Plan

  • Tested script. For detailed test result output, please see the attached SAK-2467-TestResults.txt file

Test1:

  •  

    • on sakaitestmail1 (isaac)

      • echo "WARN connections: Maximum number of open connections exceeded" >> /ucd/opt/sakai/sakai-home/logs/james-2008-12-12.log
        Result: PASS

Test2:

  •  

    • Repeat Test 1 but modify /ucd/opt/sakai/config-staging/jameskiller_whitelist.dat setting isaac=monitor

      • Result: PASS if bellow assumption is correct

      • Observation: script assumes that "WARN connections: Maximum number of open connections exceeded" is the last line in the
        james log file. If for some reason james logs something other than "WARN connections: Maximum number of open connections exceeded" the scripts thinks that james is back to normal.

Test3:

  •  

    • on sakaitest1 (littlejohn) with host configured to "monitor"

      • echo "WARN connections: Maximum number of open connections exceeded" >> /ucd/opt/sakai/sakai-home/logs/james-2008-12-12.log

      • Result: PASS if bellow assumption is correct

      • Observation: same observation as under Test2

OVERALL RESULT: QA Pass.

This application was load tested with acceptable results.

Background on this implementation: