Implementation Details, January 13, 2009

January 13, 2009 sakai deployment

Implementation Steps:

• (05:30) Put sakaiprod1, ..., sakaiprod6, sakaiprodmail1 into maintenance mode
• (05:35) SAK-2467 installation

  • - On sakaiprod1, ..., sakaiprod6, sakaiprodmail1
    -- Remove cron job for james monitoring script "10,25,40,55 * * * * /ucd/opt/sakai/config-staging/jameswatcher.pl"
    crontab -e
    -- Verify crontab via
    crontab -l
    
    - On sakaiprod1
    cd $HOME/config-staging
    rm setup-sakai-node.sh
    svn -r 6558 export https://mware.ucdavis.edu/svn/ucd-sakai/deployment-scripts/branches/manual-prod-deploy/setup-sakai-node.sh
    svn -r 6562 export https://mware.ucdavis.edu/svn/ucd-sakai/maintenance/scripts/branches/smartsite-prod/jameskiller.pl
    svn export -r 6577 https://mware.ucdavis.edu/svn/ucd-sakai/sakai-properties/branches/sakai_2-5-x/smartsite-prod/jameskiller_whitelist.dat
    
    - On sakaiprod1, ...,sakaiprod6, sakaiprodmail1
    $HOME/config-staging/setup-sakai-node.sh
    nohup /ucd/opt/sakai/bin/jameskiller.pl >> /ucd/opt/sakai/logs/jameskiller-20090113.log &
    

    
    
    
    

 
Database Backups, other Database preparation:

• None

Fall-back Procedure (If needed)

• Kill jameskiller.pl process on sakaiprod1, ..., sakaiprod6, sakaiprodmail1

QA Test Plan

• Tested script. For detailed test result output, please see the attached SAK-2467-TestResults.txt file

Test1:

• • on sakaitestmail1 (isaac)
    • echo "WARN connections: Maximum number of open connections exceeded" >> /ucd/opt/sakai/sakai-home/logs/james-2008-12-12.log
      Result: PASS

Test2:

• • Repeat Test 1 but modify /ucd/opt/sakai/config-staging/jameskiller_whitelist.dat setting isaac=monitor
    • Result: PASS if bellow assumption is correct
    • Observation: script assumes that "WARN connections: Maximum number of open connections exceeded" is the last line in the
      james log file. If for some reason james logs something other than "WARN connections: Maximum number of open connections exceeded" the scripts thinks that james is back to normal.

Test3:

• • on sakaitest1 (littlejohn) with host configured to "monitor"
    • echo "WARN connections: Maximum number of open connections exceeded" >> /ucd/opt/sakai/sakai-home/logs/james-2008-12-12.log
    • Result: PASS if bellow assumption is correct
    • Observation: same observation as under Test2

OVERALL RESULT: QA Pass.

This application was load tested with acceptable results.

Background on this implementation: