SmartSite Database Upgrade Process from Caje to Hammer

IRC Channel for Communication

server: irc.freenode.net
channel: #ucdsakai
Windows IRC client:
- http://www.mirc.com/ (Information on the program)
- Direct link to MIRC Download

Steps for DBA's

Maintenance Window

Date: May 13, 2008
Time: 05:00 to 07:00

Upgrade Process

(05:00 Joncarlo) Prevent any client access to SmartSite via iptables and redirect clients to a maintenance/outage page
- We need to allow access from dedicated IPs:
  - PORT 8443:
    - 169.237.11.246 (thomas)
    - 69.12.224.247 (thomas)
    - 169.237.97.91 (zoidberg: vpn, terminal server)
    - 169.237.97.133 (kirk)
    - 169.237.11.220 (james)
    - 169.237.11.23 (sandra)
    - 169.237.11.6 (jon)
    - 169.237.11.242 (mike)
    - 169.237.11.211 (DC)
    - 169.237.11.30 (DC)
    - 169.237.205.246 (DC)
  - PORT 9000:
    - IPs : 169.237.11.246
(05:01 Thomas) Stopping sakai on smartsite production nodes (sakaiprod1, ..., sakaiprod6)
(05:05 Prabhu)
Prabhu's DB Migration Tasks
- Backup Caje Sakai internal DB
- Restore Caje Sakai internal DB on Hammer Sakai internal DB
- Change LDAP entry for ldap://oraldap1.ucdavis.edu:389/sakai and ldap://oraldap2.ucdavis.edu:389/sakai to return the connect string for hammer instead of caje
- local.properties JDBC connect string:
  - jdbc:oracle:thin:@ldap://oraldap1.ucdavis.edu:389/sakai,cn=OracleContext,dc=ucdavis,dc=edu ldap://oraldap2.ucdavis.edu:389/sakai,cn=OracleContext,dc=ucdavis,dc=edu
(05:45 Thomas) Startup sakai on smartsite production nodes (sakaiprod1, ..., sakaiprod6)
- ldap service name change in local.properties: '/sakai'
(06:00 Kirk, Sandra, ...) Start testing/verifying SmartSite
- Monitor catalina.out for errors on all application nodes during this step
(06:30 Kirk, Sandra, ...) Decide if we go into production using Hammer
(07:00 DC Operations) Adjust iptables on all nodes (sakaiprod1, ..., sakaiprod6) to allow public access

Fallback Procedure (If needed)

(06:30 Thomas)
- If we decide to go back to caje, stop sakai on smartsite production nodes (sakaiprod1, ..., sakaiprod6)
- Revert LDAP connection string so that it points at caje
- Startup sakai on smartsite production nodes (sakaiprod1, ..., sakaiprod6)
(07:00 DC Operations) Adjust iptables on all nodes (sakaiprod1, ..., sakaiprod6) to allow public access