Version 2.3 Code Upgrade Test Plan

Overall Test Goals

The goals for testing the SmartSite v2.3 upgrade are:

to ensure that current UCDavis modifications to the "plain vanilla" Sakai version 2.3 are merged properly and produce the desired results/functionality,
to ensure that new features and functionality introduced in version 2.3 are functional and produce the desired results,
to identify new issues or changes to functionality that may influence UCD's decision to upgrade and/or require notification of faculty and other users.

Testing is organized into the following areas:

#Upgrade Tests - UCD-specific Functional Modifications
#Upgrade Tests - Sakai v2.3 Enhancements
#Upgrade Tests - UCD-specific Look 'n' Feel
#Post-upgrade Implementations

Testing Schedule

Testing will begin after a V2.3 SmartSite is available on sakai-test.ucdavis.edu or another server specified by the team.
A testing schedule will be posted once the details are worked out.

Upgrade Tests - UCD-specific Functional Modifications

These tests which specifically address UCD modifications listed under SAK-214: Upgrade our SmartSite version to Sakai 2.3
Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column.
Under Test Plan, a star indicates that the test is ready for use. A blue star indicates that any user can perform this test; a yellow star means the test is limited to tester's with administrative user access only.

Jira	Description	Test plan	Released to QA	QA Results/Date
SAK-329	Add a link to the SmartSite logo image. Sub-story of SAK-321	Follow test steps located at UCD v2.3 Upgrade Tests#Add a link to the logo image	1/30/01	Passed 2/12/07
SAK-325	Porting to v2.3.x : SiteInfo mailid modifications. Ensuring that an instructor can add valid (with Kerberos account) participants using their Kerberos id or email address. Sub-story of SAK-321	Follow test steps located at UCD v2.3 Upgrade Tests#SiteInfo mailid modifications	1/30/01	Passed 2/2/07
SAK-370	Feedback for multiple choice questions patch for 2.3.	Follow test steps located at UCD v2.3 Upgrade Tests#Feedback for multiple choice questions. Pulled on 2/5/07 due to issues with the Default template preventing the test from being attempted.	1/31/01 Pulled 2/5/07 Re-released 2/6/07	Passed 2/12/07
SAK-316	Import Question Pool into Samigo - implement this patch for v2.3 upgrade	Follow test steps located at UCD v2.3 Upgrade Tests#Import Question Pool into Samigo	1/30/01	Passed 2/16/07
SAK-322	Logout Verification - loggedOutUrl modifications. Sub-story of SAK-321	Follow test steps located at UCD v2.3 Upgrade Tests#Logout Verification	1/30/01	Passed 2/16/07
SAK-389	Bug fix for Tool list in Edit Tools is incorrect on smartsite-test	Test steps listed in the Jira SAK-389. Admin access only - requires ability to create new site.	2/2/07	Passed 2/16/07
SAK-317	Refactor Gradebook 2.1.x mod's for 2.3	Follow test steps located at Upgrade Tests - Admin Access Required#Refactor Gradebook 2.1.x mod's for 2.3. Limited to admin access testers only.	pulled for release - test data needs to be verified. LJW 2/2/07 Re-released 2/6/07	Passed 2/15/07
SAK-302	Resources - General testing of Resources functions and WebDAV to confirm no issues with usage and with 2.3 upgrade configuration.	Test Resources and WebDAV using the test matrix TC-Resources.xls in the OS/browser combinations listed. May need to add test for new provider userID's for Pat to test.	2/2/07	2/21/07 Conditional Pass w/issues noted
SAK-333	Define contributor role and define message center roles. Sub-story of SAK-321	Limited to admin access testers only - Kirk will run this test.	2/15/07	Pass per Kirk 2/22/07
SAK-303	Configure melete in Sakai 2.3 to latest melete release. Is related to SAK-340 and SAK-309	Thomas and team closed this out on 2/20/07.	NA	NA
SAK-383	Migrate faculty Melete content from 2.1.x production into SmartSite 2.3 production. Is dependant on success of SAK-303, SAK-340, SAK-309, and SAK-395	James and Kirk are testing this process since it is a one-time use admin function for migrating Melete materials.	NA	NA
SAK-304	Configure the FCKEditor to spellcheck and to include Greek fonts	Access a tool with the FCK editor, such as Message Center, Module, or Tests & Quizzes, and test out the spellcheck function. Spellcheck is the icon in the center of the top row with the letters 'ABC' and the checkmark. Report the result and your OS/browser. Warped to migration 2/21/07	NA	NA
SAK-387	Porting Training Sites tool for site creation to 2.3	Access Test Plan - Site Creation Tool for Trainers. Limited to trainer access only.	2/2/07	?
SAK-219	Need to implement preferredRole() in groupprovider	Per Jon - not testable, but system wouldn't work if not implemented.	NA	NA
SAK-354	Add Role based SU security. Sub-story of SAK-131	Follow the steps detailed in SU Tool - Test Plan. Limited to admin access testers only. Will be released to after migration.	2/12/07	Passed QA 3/2/07

Upgrade Tests - UCD Data Providers Testing

These tests specifically address UCD Data Providers Testing listed under SAK-313: Port course batching functionality (Course Batching, White List, Course Set Provider Id) from v2.1.x to v2.3.x or any other functionality that is dependent on the data provider process to be run before the tests can be performed.

Testing includes verification of the following implementation points:

Course that are white listed can be created anytime regardless of enrollment
Courses that are black listed are not automatically provisioned
Batch provisioning generated a UCD-defined default toolset
Return of up-to-date provider information during provisioning

Per Thomas, setup required to run and test functionality includes:

Update of the white-list.txt file and valid Quartz tool run
External-Banner test data defined and shared with QA tester prior to the test run
A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id

Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column. All data provider tests require admin access and are designated with a yellow star .
These tests require coordination of new course generation from white list with Thomas. First whitelist run was completed on February 7, 2007.

Jira	Description	Test plan	Released to QA	QA Results/Date
SAK-313	Port course batching functionality (Course Batching, White List, Course Set Provider Id) from v2.1.x to v2.3.x. Also includes SAK-308Upgrade providers and distauth code from 2.2 to work in 2.3	Follow test steps located at UCD Data Providers Tests#Whitelist Course Creation. Admin access only.	2/7/07	Passed 2/9/07
SAK-328	1) Enable Database Ordering of Site Tools and 2) allow for required tools which cannot be removed by Site Info and are automatically added to a new site. Sub-story of SAK-321	Follow test steps located at UCD Data Providers Tests#Enable Database Ordering of Site Tools. Admin access only.	2/7/07	Passed 2/9/07
SAK-332	Support and Training link should be automatically inserted on all newly created project and course sites.	Follow test steps located at UCD Data Providers Tests#Support and Training link should be automatically inserted. Admin access only.	2/7/07	Passed 2/9/07

UCD Data Providers Banner Test Series

Data provider tests which require validation against changes in Banner are grouped here. Banner-test is being manipulated by Libby Bullock of the Banner team as the initiator of Banner-driven changes.
*These tests require coordination with the Banner-test team member.

Course changes scheduled for February 13th test date: dataProviders_021307.txt
Rerun of test scheduled for February 15 with Libby and Sandra - data provided via email.

Jira	Description	Test plan	Released to QA	QA Results/Date
SAK-337	API calls for latest authZGroup info ought to return up-to-date provider information. Banner test series. Also includes SAK-394.	Coordinate with the Banner team member and follow the test steps located at Upgrade Tests - Admin Access Required#Validate Successful Banner Updates. Limited to admin access testers only.	NA for first run. Released 2/8/07	Failed* 2/2/07 Passed 2/15/07
SAK-220	Checkbox in site info tool that allows admin to remove non-provided users shows for provided users and also if the admin choosed to delete the user, she doesn't ultimately succeed.	Coordinate with the Banner team member and follow the test steps located at Upgrade Tests - Admin Access Required#Validate Successful Banner Student Drops. Limited to admin access testers only.	Released 2/8/07	Passed 2/15/07

* On February 2, Thomas and Lisa coordinated Banner testing of SAK-337 with Libby Bullock of the Banner team.
Initial tests involved adding and removing students and instructors in Banner and then validating that the changes are seen in a timely manner in Smartsite. Due to the time coordination required, the tests did not involve QA. The results of the initial run indicated that students were added succeesfully, but student drops showed no change in smartsite-test. Instructor changes were recorded accurately. Per Thomas, the issues with students duplicates the current performance on production 2-1-x, so the test is technically not a failure but the desired behaviour is still not achieved. Thomas generated in Jira at SAK-394 and documented the issue in Confluence at SAK-394.

For test results, indicates Pass, indicates Fail, and indicates Alert. The date which indicates when the round of testing was completed and may be updated until the test passes. Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.

Upgrade Tests - Sakai v2.3 Enhancements

Currently, this is an enhancement list from Kirk's 1/19/07 email.

Global Tests

Description
FCK editor and use in various tools
file picker (new) - not sure of all the places this may be used now.
quick check on help pages since they have changed and we changed the splash screen
Roster uses a new API to manage privacy. We are not using the Roster tool by default and will not test as part of the 2.3 upgrade.
RWiki now includes RSS notifications http://bugs.sakaiproject.org/jira/browse/SAK-4295 and commenting

Gradebook

Jira	Description	Test plan	Released to QA	QA Results/Date
SAK-430	Gradebook - Selective release as specified in http://bugs.sakaiproject.org/confluence/download/attachments/4676/GB_Selective_Release_Requirements.doc	The general Gradebook test including the timed released feature is available here and in the Jira.	2/20/07	Closed by QA 3/2/07
	Gradebook - Import and gradebook spreadsheet upload functionality http://bugs.sakaiproject.org/confluence/download/attachments/4676/GB_Upload_Import_Requirements.doc
	Assignments - ability to link to previously created gradebook items

Resources

Jira	Description	Test plan	Released to QA	QA Results/Date
SAK-391	Resources - Timed release http://bugs.sakaiproject.org/confluence/display/RES/Timed+Release+of+Resources	Requires tester to log in as an Instructor for a course site. Follow the test located at Sakai v2.3 Enhancements Tests#Resources - Timed release test	2/2/07	Passed 2/5/07
SAK-402	Resources - Ability to customize order of Resources http://bugs.sakaiproject.org/confluence/display/RES/Ability+to+customize+order+of+Resources Note: existing bugs Collab SAK-7585 When using the reorder widget within Resources, reordering items changes the modified date for all items within the folder in which the reorder is taking place and Collab SAK-8310 Resources reorder function only works on intial selection and does not work on reverse order.	Requires tester to log in as an Instructor for a course site. Follow the test located at Sakai v2.3 Enhancements Tests#Resources - Ability to customize order of Resources	2/6/07	Passed 2/21/07

Tests & Quizzes

Description
Tests&Quizzes (Samigo) - built-in template choices for samigo (seems incompatible on my account)
Tests&Quizzes (Samigo) - new question types in Samigo (audio, numeric response)
Tests&Quizzes (Samigo) - allowing instructors to use the attachment helper to attach files and/or URLs to an assessment, part or question SAK-4396
Tests&Quizzes (Samigo) - Bulk download of student files (those uploaded with file-upload question type)SAK-4397
Tests&Quizzes (Samigo) - Filtered view in grading to allow full or abbreviated view of short answers and/or rationales SAK-4573
Tests&Quizzes (Samigo) - Page navigator and search capability for the Grading pages SAK-4402 and SAK-4403
Tests&Quizzes (Samigo) -a new question type: numeric response SAK-6325
Tests&Quizzes (Samigo) -java.policy requirement for audio recording questions removed

For test results, indicates Pass, indicates Fail, and indicates Alert. The date which indicates when the round of testing was completed and may be updated until the test passes. Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.

Upgrade Tests - UCD-specific Look 'n' Feel

This series of tests require visual review for validation. Conduct these tests in multiple OS/browser combinations and note any issues in the Jira specified in the first column.

Jira	Description	Test plan
SAK-331	Allow to replace the help tool's main page with custom content.	To test, view the screen shot graphic at SAK-173 and verify that the Help tool defaults to this page when you select Help from the site menu.
SAK-336	Remove Powered by Sakai image from the portal footer
SAK-338	The Guest Email Address function is misleading in that it impies that access to Sakai/SmartSite can be provided to someone without a UC Davis kerberos login ID.
SAK-305	Reskin the Sakai 2.2/2.3 version with our SmartSite skin

For test results, indicate Pass , Fail , or Alert . Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.

Post-upgrade Implementations

These tests are for code modifications introduced after the v2.3 upgrade is completed on production.

Jira	Description	Test plan	Test Results/Date
SAK-220	Checkbox in site info tool that allows admin to remove non-provided users shows for provided users and also if the admin choosed to delete the user, she doesn't ultimately succeed.
SAK-292	Site Application Registration Form
SAK-219	Need to implement preferredRole() in groupprovider
TBD	Implementing the PageOrder tool to allow users to reorder their menus?

For test results, indicate Pass , Fail , or Alert . Alert is used if new issues are raised, if the test is faulty or for any other issue which requires review.