Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 50 Current »

IRC Channel for Communication

  • server: irc.freenode.net
  • channel: #ucdsakai
  • Windows IRC client:
  • If you have any questions or would like help getting configured, talk to Emily (emily@ucdavis.edu)

Phase 1 : External Database Preparation

  • Date: June 15, 2007
  • Time : 06:00 to 08:00
  • (06:00 Prabhu ?) Run mviews creation script <add subversion repository link>
    • This is the most up-to-date script that has been applied to smartsite-test
  • (07:00 Brian ? ) Run mviews test/verification scripts <add subversion resource link>
  • (07:15 Brian ? ) Run DDL comparison of external DB (mviews) between smartsite-test and smartsite production
    • TODO: Describe this process here
  • (07:30 Prabhu ? ) Replace smartsite-test external DB mviews with the ones from smartsite production
    • Link to mviews creation script <add subversion repository link>

Phase 2 : Application and Internal Database Upgrade

Maintenance Window

  • Date: June 23, 2007
  • Time: 00:00 to 07:00

Upgrade Process

  • (00:00 Joncarlo) Prevent any client access to SmartSite via iptables and redirect clients to a maintenance/outage page
    • We need to allow access from dedicated IPs for possible pre-upgrade work, such as exporting Melete modules, etc.
    • Adjust iptables on head node (stubing) to allow JMX access to port 9000 for specified IPs
      • PORT 8443:
        • 169.237.11.246 (thomas)
        • 69.12.224.247 (thomas)
        • 169.237.97.91 (zoidberg: vpn, terminal server)
        • 169.237.97.133 (kirk)
        • 169.237.11.220 (james)
        • 169.237.11.23 (sandra)
        • 169.237.11.211 (emily)
        • 169.237.11.6 (jon)
        • 169.237.11.242 (mike)
        • 169.237.11.30 (lisa)
        • 169.237.97.223 (pat)
        • ...
      • PORT 9000:
        • IPs : 169.237.11.246
  • (00:10 Prabhu) Backing up (CLONE) production database so that it can be fully restored.
  • (01:00 Thomas) Run sakai_2_3_1-2_4_0_oracle_conversion.sql
  • (01:15 Thomas) Run SmartSite Extra Updates
  • (01:30 Thomas) Run SAKAI_USER_ID_MAP conversion script
  • (01:40 Thomas) Run SAKAI_REALM and SAKAI_REALM_PROVIDER conversion script
  • (01:50 Thomas) Run SmartSite Extra Updates Part II
  • (01:55 Thomas/Emily) Backup all configuration files in production staging area
    • /afs/.ucdavis.edu/app/sakai/staging/config/* 
      • mkdir /afs/.ucdavis.edu/app/sakai/staging/config/config-2-3-x.bak
cp /afs/.ucdavis.edu/app/sakai/staging/config/* /afs/.ucdavis.edu/app/sakai/staging/config/config-2-3-x.bak/

  • (02:10 Thomas/Emily) Start process of preparing Sakai v2.4.x source on the head node (stubing).
    • Getting the source code: 
      • cd ~/src
svn co https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-4-x-prod-001

    • Setting up the build environment:
    • ~/build.properties 
      • maven.repo.remote=http://source.sakaiproject.org/maven/,https://mware.ucdavis.edu/maven/,http://www.ibiblio.org/maven2/
maven.tomcat.home=/ucd/opt/sakai/tomcat-staging/
maven.compile.source = 1.5
maven.compile.target = 1.5

    • ~/.bash_profile 
      • PATH=/ucd/opt/java5/bin:/ucd/opt/maven/bin:$PATH
MAVEN_HOME=/ucd/opt/maven
CATALINA_HOME=/ucd/pkg/apache-tomcat
JAVA_HOME=/ucd/opt/java5
JAVA_OPTS=" -d64 -server -Xmx6144m -Xms6144m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=256m -XX:MaxPermSize=512m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dsakai.home=/ucd/opt/sakai/sakai-home/"
CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
MAVEN_OPTS="-Xmx1024m -Xms1024m"
export PATH MAVEN_HOME CATALINA_HOME JAVA_HOME JAVA_OPTS CATALINA_OPTS MAVEN_OPTS

    • Delete old tomcat content 
      • rm -fr /ucd/pkg/apache-tomcat/*

    • Delete old tomcat staging build 
      • cd ~/bin
./tomcatclean.sh

    • Build the source code 
      • cd ~/src/sakai_2-4-x-prod-001
maven sakai

    • Copy staged tomcat 
      • rm -fr /ucd/pkg/apache-tomcat/*
cp -R /afs/.ucdavis.edu/app/sakai-test/staging/tomcat/apache-tomcat-5.5.23/* /ucd/pkg/apache-tomcat/

  • (02:50 Thomas/Emily) Making sure all the configuration files are in place for v2.4.x such as sakai.properties etc.
    • Configure JAVA_OPTS to allow JMX access via JConsole
    • Synchronize all configuration files on the head node (stubing) with the AFS configuration staging area
      • /afs/.ucdavis.edu/app/sakai/staging/config
    • Edit sakai.properties on stubing 
      • performChatMigration@org.sakaiproject.chat2.model.impl.ChatDataMigration=true
chatMigrationExecuteImmediate@org.sakaiproject.chat2.model.impl.ChatDataMigration=true
auto.ddl=true

    • Start-up head node (stubing) and check catalina.out for any errors
      • This startup will take longer than the usual because of the chat conversion
      • Check catalina.out for DB errors
      • Once the head node started up without any errors and you see the message that the chat conversion has completed, shut it down again
    • Edit sakai.properties on stubing 
      • #performChatMigration@org.sakaiproject.chat2.model.impl.ChatDataMigration=false
#chatMigrationExecuteImmediate@org.sakaiproject.chat2.model.impl.ChatDataMigration=false
auto.ddl=false
    • Configure site creation to send email to smartsite-request@ucdavis.edu when unusual roster requests are made
    • Start-up head node (stubing) again
      • Make sure that there are no errors in catalina.out
    • Access head node and perform any appropriate testing
    • If there are no errors/problems, perform installation on second node (walton)
    • Start-up walton and perform any appropriate testing
  • (03:30 Kirk, Sandra, ...) Release SmartSite for initial testing to determine "successful upgrade"
  • (06:55 Joncarlo) Adjust iptables on all nodes (stubing, walton) to allow public access
  • (07:00) System ready for public access

(minus) Fallback Procedure (If needed)

  • In case anything goes wrong that cannot be fixed within the allocated maintenance window we perform the following tasks
  • (08:00 Thomas/Emily) Shutdown all application servers
  • (08:10 Prabhu) Restore Sakai internal production database on caje
  • (08:40 Thomas/Emily) Install latest sakai-core-2-3-x-prod snapshot on all application nodes (stubing, walton)
    • Restore v2.3.x configuration files
  • (09:30 Thomas/Emily) Start nodes and test
  • (09:40 Kirk, Sandra, ...) Start testing
  • (10:00 Joncarlo) Adjust iptables
  • (10:05) System ready for public access

Post Upgrade TODOs:

  • SAK-664 : Verify footer display, content and links are correct for the SmartSite 2.4 upgrade
  • (tick) SAK-746 : Post-upgrade task for 2.4: Add site.add function to registered user role so that new users can create course and project sites
  • (tick) SAK-751 : create CM role mapping for waitlisted students
  • No labels