DRAFT
Maintenance Window
- Date: ?
- Time: ?
Upgrade Process
- Prevent any client access to SmartSite via iptables and redirect clients to a maintenance/outage page
- We may need to allow access from dedicated IPs for possible pre-upgrade work, such us exporting Melete modules, etc.
- Shutting down production instances on all nodes
- Application server production nodes are stubing and walton
- Backing up production database so that it can be fully restored.
- This backup should maintain the names for all constraints, indexes, etc.
- Production SmartSite internal DB server is caje
- Perform maintenance on the SmartSite internal DB on caje
- Remove all existing melete tables (We only do this if we choose to export/import all Melete modules)
- Apply Sakai update scripts:
- https://mware.ucdavis.edu/svn/ucd-sakai/update/sakai_2_1_2-2_2_0_oracle_conversion.sql
- https://mware.ucdavis.edu/svn/ucd-sakai/update/sakai_2_2_0-2_2_1_oracle_conversion.sql
- https://mware.ucdavis.edu/svn/ucd-sakai/update/sakai_2_2_1-2_3_0_oracle_conversion.sql
- https://mware.ucdavis.edu/svn/ucd-sakai/update/sakai_2_3_0-2_3_1_oracle_conversion.sql
- Apply additional table creation scripts if we choose to create our tables via DDL scripts instead of using Hibernate's audo.ddl
- The tables in question are:
- UCDSAKAI_COURSE_REQUEST
- UCDSAKAI_META_DATA
- UCDSAKAI_PROJECT_REQUEST
- UCDSAKAI_SITE_REQUEST
- SAKAI_DYNAMIC_CONFIGURATION
- The tables in question are:
- Perform maintenance on the SmartSite AFS volumes
- Backup and then remove all existing melete entries (We only do this if we choose to export/import all Melete modules)
- Melete volumes are:
- /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/melete
- /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/meleteDocs
- /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/uploads
- Melete volumes are:
- Backup and then remove all existing melete entries (We only do this if we choose to export/import all Melete modules)
- Backup all configuration files on production stating area
- /afs/.ucdavis.edu/app/sakai/staging/config/*
- Start process of preparing Sakai v2.3.x source on the head node (stubing). Follow normal weekend maintenance and source code update instructions
- Environments need to be adjusted to only depend on Java 1.5. This includes:
- build.properties
- .bash_profile
- Environments need to be adjusted to only depend on Java 1.5. This includes:
- Follow specific Melete installation instructions for their v2.3 release
- TODO: Melete installation instructions
- Making sure all the configuration files are in place for v2.3.x such as sakai.properties etc.
- Configure JAVA_OPTS to allow JMX access via JConsole
- Synchronize all configuration files on the head node (stubing) with the AFS configuration stating area
- /afs/.ucdavis.edu/app/sakai/staging/config
- Start-up head node (stubing) and check catalina.out for any errors
- Access head node and perform any appropriate testing
- If there are no errors/problems, perform installation on second node (walton)
- Start-up walton and perform any appropriate testing
- Adjust iptables on head node (stubing) to allow JMX access to port 9000 for specified IPs
Fallback Procedure
- In case anything goes wrong that cannot be fixed within the allocated maintenance window we perform the following tasks
- Shutdown all application servers
- Restore deleted Melete AFS volumes
- Restore full database backup
- Install latest sakai-core-2-1-x-prod snapshot on all application nodes
- Restore v2.1.x configuration files
- Start nodes and test