Page Comparison

...

server: irc.freenode.net
channel: #ucdsakai
Windows IRC client:
- http://www.mirc.com/ (Information on the program)
- Direct link to MIRC Download
If you have any questions or would like help getting configured, talk to Emily (emily@ucdavis.edu)

Maintenance Window

Date: August 2318, 2007
Time: 08:00 to 12:00 (noon)

...

(08:00 Joncarlo) Prevent any client access to SmartSite via iptables and redirect clients to a maintenance/outage page
- We need to allow access from dedicated IPs for possible pre-upgrade work, such as exporting Melete modules, etc.
- Adjust iptables on head node (stubing) to allow JMX access to port 9000 for specified IPs
  - PORT 8443:
    - 169.237.11.246 (thomas)
    - 69.12.224.247 (thomas)
    - 169.237.97.91 (zoidberg: vpn, terminal server)
    - 169.237.97.133 (kirk)
    - 169.237.11.220 (james)
    - 169.237.11.23 (sandra)169.237.11.211 (emily)
    - 169.237.11.6 (jon)
    - 169.237.11.242 (mike)
    - 169.237.11.30 (lisa)
    - 169.237.97.223 (pat)
    - ...
  - PORT 9000:
    - IPs : 169.237.11.246
(08:10 Prabhu) Backing up (CLONE) production database so that it can be fully restored.
(09:00 Prabhu) Run sakai_2_3_1-2_4_0_oracle_conversion.sql sakai_2_3_1-2_4_0_oracle_conversiondatabase upgrade scripts in the following order
1. SAK-869.sql
2. SAK-878.sql
(0109:15 Thomas) Run SmartSite Extra Updates
- smartsite_extra_update_2-4-x.sql
(01:30 Thomas) Run SAKAI_USER_ID_MAP conversion script
- eidChange.groovy
(01:40 Thomas) Run SAKAI_REALM and SAKAI_REALM_PROVIDER conversion script
- ProviderIdConversion.groovy
(01:50 Thomas) Run SmartSite Extra Updates Part II
- smartsite_extra_update_2-4-x.sql

(01:55 Thomas/Emily) Backup all configuration files in production staging area

/afs/.ucdavis.edu/app/sakai/staging/config/*

Code Block
mkdir /afs/.ucdavis.edu/app/sakai/staging/config/config-2-34-x0.bak cp /afs/.ucdavis.edu/app/sakai/staging/config/* /afs/.ucdavis.edu/app/sakai/staging/config/config-2-34-x0.bak/

(0209:10 20 Thomas/Emily) Start process of preparing Sakai v2.4.x 1 source on the head node (stubing).

Getting the source code:

Code Block
cd ~/src svn co https://mware.ucdavis.edu/svn/ucd-sakai/sakai/tags/sakai_2-4-x-prod-001

Setting up the build environment:

~/build.properties

Code Block


maven.repo.remote=http://source.sakaiproject.org/maven/,https://mware.ucdavis.edu/maven/,http://www.ibiblio.org/maven2/
maven.tomcat.home=/ucd/opt/sakai/tomcat-staging/
maven.compile.source = 1.5
maven.compile.target = 1.5

~/.bash_profile

Code Block


PATH=/ucd/opt/java5/bin:/ucd/opt/maven/bin:$PATH
MAVEN_HOME=/ucd/opt/maven
CATALINA_HOME=/ucd/pkg/apache-tomcat
JAVA_HOME=/ucd/opt/java5
JAVA_OPTS=" -d64 -server -Xmx6144m -Xms6144m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=256m -XX:MaxPermSize=512m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dsakai.home=/ucd/opt/sakai/sakai-home/"
CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
MAVEN_OPTS="-Xmx1024m -Xms1024m"
export PATH MAVEN_HOME CATALINA_HOME JAVA_HOME JAVA_OPTS CATALINA_OPTS MAVEN_OPTS

Delete old tomcat content

Code Block
rm -fr /ucd/pkg/apache-tomcat/*

Delete old tomcat staging build

Code Block
cd ~/bin ./tomcatclean.sh

Build the source code

Code Block
cd ~/src/sakai_2-4-x-prod-001 maven sakai

Copy staged tomcat

Code Block
rm -fr /ucd/pkg/apache-tomcat/* cp -R /afs/.ucdavis.edu/app/sakai-test/staging/tomcat/apache-tomcat-5.5.23/* /ucd/pkg/apache-tomcat/

(02:50 Thomas/Emily

- 003
Cleaning, building, deploying source on head node

(09:35 Thomas) Making sure all the configuration files are in place for v2.4.x 1 such as sakai.properties etc.

Configure JAVA_OPTS to allow JMX access via JConsole
Synchronize all configuration files on the head node (stubing) with the AFS configuration staging area
- /afs/.ucdavis.edu/app/sakai/staging/config

Edit sakai.properties on stubing

Code Block
performChatMigration@org.sakaiproject.chat2.model.impl.ChatDataMigration=true chatMigrationExecuteImmediate@org.sakaiproject.chat2.model.impl.ChatDataMigration=true auto.ddl=true

Start-up head node (stubing) and check catalina.out for any errors

This startup will take longer than the usual because of the chat conversion

Check catalina.out for DB errors.

There is one known issue where hibernate auto.ddl creates the following warning:

No Format


WARN: Unsuccessful schema statement: alter table GB_LETTERGRADE_MAPPING add LG_MAPPING_ID  number(19,0) (2007-08-07 16:07:57,962
main_org.sakaiproject.springframework.orm.hibernate.AddableSessionFactoryBean) java.sql.SQLException: ORA-01430: column being added already exists in table
...

Once the head node started up without any errors and you see the message that the chat conversion has completed, shut it down again

Edit sakai.properties on stubing

Code Block
#performChatMigration@org.sakaiproject.chat2.model.impl.ChatDataMigration=false #chatMigrationExecuteImmediate@org.sakaiproject.chat2.model.impl.ChatDataMigration=false auto.ddl=false

Configure site creation to send email to smartsite-request@ucdavis.edu when unusual roster requests are made

Start-up head node (stubing) again
- Make sure that there are no errors in catalina.out
Access head node and perform any appropriate testing
If there are no errors/problems, perform installation on second node (walton)
Start-up walton and perform any appropriate testing

(0310:30 00 Kirk, Sandra, Lisa, ...) Release SmartSite for initial testing to determine "successful upgrade"
- Test Phase Issue Handling Process
- Access it via:
  - https://smartsite.ucdavis.edu:8443/portal
    - https://stubing.ucdavis.edu:8443/portal
    - https://walton.ucdavis.edu:8443/portal
(0612:55 00 Joncarlo) Adjust iptables on all nodes (stubing, walton) to allow public access
- (12:01) System ready for public access
...
- In case anything goes wrong that cannot be fixed within the allocated maintenance window we perform the following tasks
- (0812:00 Thomas/EmilyJoncarlo) Prevent any client access to SmartSite via iptables and redirect clients to a maintenance/outage page
- (12:01 Thomas) Shutdown all application servers
- (0812:10 Prabhu) Restore Sakai internal production database on caje
- (0812:40 Thomas/Emily) Install latest sakai-core-2-34-x-prod-002 snapshot on all application nodes (stubing, walton)
  - Restore v2.34.x 0 configuration files
- (0913:30 Thomas/Emily) Start nodes and test
- (0913:40 Kirk, Sandra, ...) Start testing
- (1014:00 Joncarlo) Adjust iptables
- (1014:0501) System ready for public access
Post Upgrade TODOs:
- SAK-664 : Verify footer display, content and links are correct for the SmartSite 2.4 upgrade SAK-746 : Post-upgrade task for 2.4: Add site.add function to registered user role so that new users can create course and project sites SAK-751 : create CM role mapping for waitlisted students

Versions Compared

Old Version 2

New Version Current

Key

Maintenance Window

Post Upgrade TODOs: