Sakai Cluster (PROD)

PROD: Setup Sakai Cluster

Table of Contents

JIRA: SAK-130

Load Balancer

  • IP: 169.237.104.23
  • DNS: smartsite.ucdavis.edu
  • TTL: 60 min.

Nodes

  • stubing
  • walton
  • braddock (during live updates)

System Administrator Tasks

Sakai system user on each node

  • User name: sakai
  • Home: /ucd/opt/sakai

Applicatoins

  • /ucd/pkg/jdk1.5.0_11 ( or newer)
    • Create soft-link: /ucd/opt/java5 --> /ucd/pkg/jdk1.5.0_11
    • (warning) NOTE: Whenever there is a newer java version available, we need to make sure that this new version is installed on all new and existing nodes
  • /ucd/pkg/maven-1.0.2
    • Create soft-link: /ucd/opt/maven --> /ucd/pkg/maven-1.0.2
  • Subversion client

Tomcat deployment directory

  • Create a directory "/ucd/pkg/apache-tomcat/" ... so that the sakai user has "RW" access to it
  • The following soft links need to be created on each node
    /var/sakai/content --> /afs/.ucdavis.edu/app/sakai/misc/content/smartsite
    /var/sakai/archive --> /afs/.ucdavis.edu/app/sakai/misc/archive/smartsite
    /var/sakai/ucd-data --> /afs/.ucdavis.edu/app/sakai/misc/ucd-data/smartsite
    /var/melete --> /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/melete
    /var/meleteDocs --> /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/meleteDocs
    /var/uploads --> /afs/.ucdavis.edu/app/sakai/misc/melete/smartsite/uploads
    /var/samigo --> /afs/.ucdavis.edu/app/sakai/misc/samigo/smartsite 
    

Programmer Tasks

Node Setup

#!/bin/sh

# Normal Node
ln -s /afs/.ucdavis.edu/app/sakai/staging/config $HOME/config-staging
cp $HOME/config-staging/bash_profile $HOME/.bash_profile
source $HOME/.bash_profile
mkdir $HOME/sakai-home
mkdir $HOME/bin

cp $HOME/config-staging/copy-tomcat.sh $HOME/bin
cp $HOME/config-staging/SmartSiteCheck.sh $HOME/bin
cp $HOME/config-staging/start.sh $HOME/bin
cp $HOME/config-staging/stop.sh $HOME/bin


cp $HOME/config-staging/sakai.properties $HOME/sakai-home
cp $HOME/config-staging/toolOrder.xml $HOME/sakai-home
touch $HOME/sakai-home/local.properties
echo "# Local server name" > $HOME/sakai-home/local.properties
echo "serverId=`hostname`" >> $HOME/sakai-home/local.properties

$HOME/bin/copy-tomcat.sh

# Head Node
# cp $HOME/config-staging/tomcatclean.sh $HOME/bin
# mkdir $HOME/src

AFS staging volume

  • /afs/.ucdavis.edu/app/sakai/staging
    • /afs/.ucdavis.edu/app/sakai/staging/tomcat
    • /afs/.ucdavis.edu/app/sakai/staging/config
    • /afs/.ucdavis.edu/app/sakai/staging/src

Common files in staging area

  • /afs/.ucdavis.edu/app/sakai/staging/tomcat/..
    • apache-tomcat-5.5.23
  • /afs/.ucdavis.edu/app/sakai/staging/config/..
    • bash_profile
    • sakai.properties
    • start.sh
    • stop.sh
    • tomcatclean.sh
    • sakai.properties
    • local.properties
    • build.properties
  • /afs/.ucdavis.edu/app/sakai/staging/src/..
    • sakai_2-4-x-prod-X

Directory setup on each node

  • /ucd/opt/sakai
    • build.properties NOTE: In case we need to build the source
  • /ucd/opt/sakai/bin
    • start.sh
    • stop.sh
    • tomcatclean.sh
  • /ucd/opt/sakai/sakai-home
    • sakai.properties
    • local.properties
  • /ucd/pkg/apache-tomcat
    • Ready to run tomcat instance

Detailed file descriptions and content

  • .bash_profile
    PATH=/ucd/opt/java5/bin:/ucd/opt/maven/bin:$PATH
    MAVEN_HOME=/ucd/opt/maven
    CATALINA_HOME=/ucd/pkg/apache-tomcat
    CATALINA_STAGING=/afs/.ucdavis.edu/app/sakai/staging/tomcat/apache-tomcat-5.5.23
    JAVA_HOME=/ucd/opt/java5
    JAVA_OPTS=" -d64 -server -Xmx6144m -Xms6144m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=256m -XX:MaxPermSize=512m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dsakai.home=/ucd/opt/sakai/sakai-home/"
    CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
    MAVEN_OPTS="-Xmx1024m -Xms1024m"
    export PATH MAVEN_HOME CATALINA_HOME JAVA_HOME JAVA_OPTS CATALINA_OPTS MAVEN_OPTS CATALINA_STAGING
    
  • start.sh
    #!/bin/sh
    
    JAVA_HOME=/ucd/opt/java5
    export JAVA_HOME
    $CATALINA_HOME/bin/startup.sh
    
  • stop.sh
    #!/bin/sh
    
    JAVA_HOME=/ucd/opt/java5
    export JAVA_HOME
    $CATALINA_HOME/bin/shutdown.sh
    
  • tomcatclean.sh
    #!/bin/sh
    
    cd $CATALINA_STAGING
    rm -fr components
    rm common/lib/sakai-*
    rm shared/lib/*
    rm -fr work/Catalina/localhost/*
    mv webapps/ROOT .
    rm -fr webapps/*
    mv ROOT webapps/
    
  • local.properties NOTE: The serverId field is different for each node
    # Local server name
    serverId=stubing
    
  • copy-tomcat.sh
    #!/bin/sh
    
    rm -fr $CATALINA_HOME/*
    cp -R $CATALINA_STAGING/* $CATALINA_HOME/
    

FAQ

  • Q: How do I know which node I am connected to?
  • A: Look at the bottom of the page, where you will see a filed called Server
    • e.g. Server littlejohn