Version 2.3 Code Upgrade Test Plan

Overall Test Goals

The goals for testing the SmartSite v2.3 upgrade are:

  • to ensure that current UCDavis modifications to the "plain vanilla" Sakai version 2.3 are merged properly and produce the desired results/functionality,
  • to ensure that new features and functionality introduced in version 2.3 are functional and produce the desired results,
  • to identify new issues or changes to functionality that may influence UCD's decision to upgrade and/or require notification of faculty and other users.

Testing is organized into the following areas:

#Upgrade Tests - UCD-specific Visuals
#Upgrade Tests - UCD-specific Functional Modifications
#Upgrade Tests - Sakai v2.3 Enhancements

Testing Schedule

Testing will begin after a V2.3 SmartSite is available on smartsite-test.ucdavis.edu (or another server) as specified by the team.
A testing schedule will be posted once the details are worked out.

Upgrade Tests - UCD-specific Visuals

This series of tests require visual review for validation. Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column.
Under Test Plan, a star indicates that the test is ready for use. A blue star (blue star) indicates that any user can perform this test; a yellow star (star) means the test is limited to tester's with administrative user access only.

Comparisons are specified for the following OS/browser combinations*:

  • Windows XP SP2, Windows 2000 SP4 - Internet Explorer 6, Firefox 1.5.0.8
  • Mac OS X 10.4.8, Mac OS X 10.3.9 - Safari 2.0.4, Safari 1.2, Firefox 1.5.0.8
  • Additional test environment: VISTA - IE7

This represents a total of 10 OS/browser combinations to test with the addition of VISTA/IE7 to be determined.
* from What IT Express Supports as of 1/23/07

Jira

Description

Test Plan

Released to QA

QA Results/Date

SAK-331

Allow to replace the help tool's main page with custom content. Sub-story of SAK-321

(blue star) To test, view the screen shot graphic at SAK-173 and verify that the Help tool defaults to this page when you select Help from any site menu. Select at least 2 sites at random to verify.

1/30/07

Passed 2/5/07

SAK-336

Remove Powered by Sakai image from the portal footer. Sub-story of SAK-321

(blue star) To test, view the screen graphic of the default footer and verify that the text "Powered By Sakai" does not appear in/is removed from the SmartSite footer.

1/30/07

Passed 2/5/07

SAK-338

The Guest Email Address function is misleading in that it impies that access to Sakai/SmartSite can be provided to someone without a UC Davis kerberos login ID. Sub-story of SAK-321

(blue star) Go to Site Info for a course which you manage and select Add Participant. Verify that the Guest Email Address form field and text in the graphic do NOT appear in Smartsite.

1/30/07

Passed 2/5/07

SAK-305

Reskin the Sakai 2.2/2.3 version with our SmartSite skin

(blue star) Follow the test steps as specified in Re-Skin Validation Test.

1/30/07

Failed 2/5/07 Going to prod w/known issues

SAK-330

Link to different gateway site. Sub-story of SAK-321

(blue star) Visually compare the test login screen against the production SmartSite login screen. Report differences in navigation links, content, fonts, background, graphics, or any other discernable difference.

1/30/07

Passed 2/12/07

SAK-377SAK-396

Set up Smartsite footer links and text content for v2.3 upgrade

(blue star) Follow the test steps as specified in UCD v2.3 Upgrade Tests#Set up Smartsite footer links and text content for v2.3 upgrade.

1/30/07 Re-released 2/7/06

Failed 2/2/07
Passed 2/15/07

For test results, (tick) indicates Pass, (error) indicates Fail, and (warning) indicates Alert. The date which indicates when the round of testing was completed and may be updated until the test passes. Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.
Test

Upgrade Tests - UCD-specific Functional Modifications

These tests which specifically address UCD modifications listed under SAK-214: Upgrade our SmartSite version to Sakai 2.3
Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column.
Under Test Plan, a star indicates that the test is ready for use. A blue star (blue star) indicates that any user can perform this test; a yellow star (star) means the test is limited to tester's with administrative user access only.

Jira

Description

Test plan

Released to QA

QA Results/Date

SAK-329

Add a link to the SmartSite logo image. Sub-story of SAK-321

(blue star) Follow test steps located at UCD v2.3 Upgrade Tests#Add a link to the logo image

1/30/01

Passed 2/12/07

SAK-325

Porting to v2.3.x : SiteInfo mailid modifications. Ensuring that an instructor can add valid (with Kerberos account) participants using their Kerberos id or email address. Sub-story of SAK-321

(blue star) Follow test steps located at UCD v2.3 Upgrade Tests#SiteInfo mailid modifications

1/30/01

Passed 2/2/07

SAK-370

Feedback for multiple choice questions patch for 2.3.

(blue star) Follow test steps located at UCD v2.3 Upgrade Tests#Feedback for multiple choice questions. Pulled on 2/5/07 due to issues with the Default template preventing the test from being attempted.

1/31/01
Pulled 2/5/07 Re-released 2/6/07

Passed 2/12/07

SAK-316

Import Question Pool into Samigo - implement this patch for v2.3 upgrade

(blue star) Follow test steps located at UCD v2.3 Upgrade Tests#Import Question Pool into Samigo

1/30/01

Passed 2/16/07

SAK-322

Logout Verification - loggedOutUrl modifications. Sub-story of SAK-321

(blue star) Follow test steps located at UCD v2.3 Upgrade Tests#Logout Verification

1/30/01

Passed 2/16/07

SAK-389

Bug fix for Tool list in Edit Tools is incorrect on smartsite-test

(star) Test steps listed in the Jira SAK-389. Admin access only - requires ability to create new site.

2/2/07

Passed 2/16/07

SAK-317

Refactor Gradebook 2.1.x mod's for 2.3

(star) Follow test steps located at Upgrade Tests - Admin Access Required#Refactor Gradebook 2.1.x mod's for 2.3. Limited to admin access testers only.

pulled for release - test data needs to be verified. LJW 2/2/07
Re-released 2/6/07

Passed 2/15/07

SAK-302

Resources - General testing of Resources functions and WebDAV to confirm no issues with usage and with 2.3 upgrade configuration.

(blue star) Test Resources and WebDAV using the test matrix TC-Resources.xls in the OS/browser combinations listed. May need to add test for new provider userID's for Pat to test.

2/2/07

2/21/07 Conditional Pass w/issues noted

SAK-333

Define contributor role and define message center roles. Sub-story of SAK-321

(star) Limited to admin access testers only - Kirk will run this test.

2/15/07

Pass per Kirk 2/22/07

SAK-303

Configure melete in Sakai 2.3 to latest melete release. Is related to SAK-340 and SAK-309

Thomas and team closed this out on 2/20/07.

NA

NA

SAK-383

Migrate faculty Melete content from 2.1.x production into SmartSite 2.3 production. Is dependant on success of SAK-303, SAK-340, SAK-309, and SAK-395

James and Kirk are testing this process since it is a one-time use admin function for migrating Melete materials.

NA

NA

SAK-304

Configure the FCKEditor to spellcheck and to include Greek fonts

Access a tool with the FCK editor, such as Message Center, Module, or Tests & Quizzes, and test out the spellcheck function. Spellcheck is the icon in the center of the top row with the letters 'ABC' and the checkmark. Report the result and your OS/browser.
Warped to migration 2/21/07

NA

NA

SAK-387

Porting Training Sites tool for site creation to 2.3

(star) Access Test Plan - Site Creation Tool for Trainers. Limited to trainer access only.

2/2/07

?

SAK-219

Need to implement preferredRole() in groupprovider

Per Jon - not testable, but system wouldn't work if not implemented.

NA

NA

SAK-354

Add Role based SU security. Sub-story of SAK-131

(star) Follow the steps detailed in SU Tool - Test Plan. Limited to admin access testers only. Will be released to after migration.

2/12/07

Passed QA 3/2/07

Upgrade Tests - UCD Data Providers Testing

These tests specifically address UCD Data Providers Testing listed under SAK-313: Port course batching functionality (Course Batching, White List, Course Set Provider Id) from v2.1.x to v2.3.x or any other functionality that is dependent on the data provider process to be run before the tests can be performed.

Testing includes verification of the following implementation points:

  • Course that are white listed can be created anytime regardless of enrollment
  • Courses that are black listed are not automatically provisioned
  • Batch provisioning generated a UCD-defined default toolset
  • Return of up-to-date provider information during provisioning

Per Thomas, setup required to run and test functionality includes:

  • Update of the white-list.txt file and valid Quartz tool run
  • External-Banner test data defined and shared with QA tester prior to the test run
  • A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id

Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column. All data provider tests require admin access and are designated with a yellow star (star).
These tests require coordination of new course generation from white list with Thomas. First whitelist run was completed on February 7, 2007.

Jira

Description

Test plan

Released to QA

QA Results/Date

SAK-313

Port course batching functionality (Course Batching, White List, Course Set Provider Id) from v2.1.x to v2.3.x.
Also includes SAK-308Upgrade providers and distauth code from 2.2 to work in 2.3

(star) Follow test steps located at UCD Data Providers Tests#Whitelist Course Creation. Admin access only.

2/7/07

Passed 2/9/07

SAK-328

1) Enable Database Ordering of Site Tools and 2) allow for required tools which cannot be removed by Site Info and are automatically added to a new site. Sub-story of SAK-321

(star) Follow test steps located at UCD Data Providers Tests#Enable Database Ordering of Site Tools. Admin access only.

2/7/07

Passed 2/9/07

SAK-332

Support and Training link should be automatically inserted on all newly created project and course sites.

(star) Follow test steps located at UCD Data Providers Tests#Support and Training link should be automatically inserted. Admin access only.

2/7/07

Passed 2/9/07

UCD Data Providers Banner Test Series

Data provider tests which require validation against changes in Banner are grouped here. Banner-test is being manipulated by Libby Bullock of the Banner team as the initiator of Banner-driven changes.
*These tests require coordination with the Banner-test team member.

Course changes scheduled for February 13th test date: dataProviders_021307.txt
Rerun of test scheduled for February 15 with Libby and Sandra - data provided via email.

Jira

Description

Test plan

Released to QA

QA Results/Date

SAK-337

API calls for latest authZGroup info ought to return up-to-date provider information. Banner test series. Also includes SAK-394.

(star) Coordinate with the Banner team member and follow the test steps located at Upgrade Tests - Admin Access Required#Validate Successful Banner Updates. Limited to admin access testers only.

NA for first run. Released 2/8/07

Failed* 2/2/07 Passed 2/15/07

SAK-220

Checkbox in site info tool that allows admin to remove non-provided users shows for provided users and also if the admin choosed to delete the user, she doesn't ultimately succeed.

(star) Coordinate with the Banner team member and follow the test steps located at Upgrade Tests - Admin Access Required#Validate Successful Banner Student Drops. Limited to admin access testers only.

Released 2/8/07

Passed 2/15/07

* On February 2, Thomas and Lisa coordinated Banner testing of SAK-337 with Libby Bullock of the Banner team.
Initial tests involved adding and removing students and instructors in Banner and then validating that the changes are seen in a timely manner in Smartsite. Due to the time coordination required, the tests did not involve QA. The results of the initial run indicated that students were added succeesfully, but student drops showed no change in smartsite-test. Instructor changes were recorded accurately. Per Thomas, the issues with students duplicates the current performance on production 2-1-x, so the test is technically not a failure but the desired behaviour is still not achieved. Thomas generated in Jira at SAK-394 and documented the issue in Confluence at SAK-394.

For test results, (tick) indicates Pass, (error) indicates Fail, and (warning) indicates Alert. The date which indicates when the round of testing was completed and may be updated until the test passes. Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.

Upgrade Tests - Sakai v2.3 Enhancements

Note: Testing for new feature functionality was partially completed during pre-2.3.x migration testing.
Results for these tests and for functionality testing post-upgrade may be found at: Smartsite 2.3.x Tests