TEST PLAN DATE
Testing date:
TEST PLAN OVERVIEW
Scope of testing
This document is designed to guide Quality Assurance in the functionality testing of the provider mechanisms within SmartSite. The functionality of the provider mechanisms, as detailed by the given test cases in this plan, are driven by the functional requirements at :http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/Sakai+Providers.
Additionally, the level of involvement by QA, system administrators, and programmers will be detailed for each test case.
Tip | ||
---|---|---|
| ||
The purpose of this test plan is to regression test the provider functionality, which in SmartSite includes: UserDirectoryProvider
GroupProvider
CourseManagementProvider
|
KNOWN ISSUES / PREVIOUS BUGS FIXED
Previously determined bugs have been fixed for this testing, they were identified as the following. They are also summarized in the table of Jiras referenced in this test plan.
- Realms not refreshing frequent enough
- Rosters not accurate with Banner
- Drops not showing in SmartSite UI
- Known (not fixed) issue #1 for Resources tool (e.g. using WebDav), is that when a file is uploaded to the system, metadata is written to the database before the file is stored on the file system. This is a feature of Sakai, which was detected during some resource discussion re:AFS file storage. For this reason, a resource may "seem" to be updated correctly, and listed in the resouce tool however the content cannot be seen. An http 404 error is thrown (No page can be found). This error is NOT fixed for this testing, just a known issue for Case 1a.
- Known (not fixed) issue #2 is that whatever user that creates the sites when logged in is placed as a member in the site. Therefore, if ssbatchadmin created sites then the roster for such sites may appear off by 1.
- Known (not fixed) issue #3 is that the instructor presented in the member list is not presented first. This is a default sorting behavior within SmartSite and will not be fixed.
- Known (not fixed) issue #4 http://jira.ucdavis.edu:8080/jira/browse/SAK-202 is that the instructor presented in the member list may be still listed even if they have been "replaced"
Info | |||||
---|---|---|---|---|---|
| |||||
Summary of the Jiras related to this test plan, which have been fixed:
|
TEST ENVIRONMENT
- Hardware Setup
INTEGRATION TESTING | SETUP |
Server Type | Location |
---|---|
database | bricker.ucdavis.edu with banner-test (external db), and sakai dev (internal db) |
application | hanley.ucdavis.edu, sakai-dev |
web server | url, https://hanley.ucdavis.edu:8443/portal |
population of data referenced | 2006 Fall Quarter Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created) |
QA TESTING | SETUP |
Server Type | Location |
---|---|
database | bricker.ucdavis.edu with banner-test (external db), and sakai test (internal db) |
application | isaac.ucdavis.edu, sakai-test |
web server | url, http://sakai-test.ucdavis.edu |
population of data referenced | 2006 Fall Quarter Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created) |
- Note: The application does not need to be stopped for these tests to occur
- Software Setup
run test under https://mware.ucdavis.edu/svn/ucdsakai/branches/sakai-core-2.1.x branch, with mods from https://mware.ucdavis.edu/svn/ucdakai/sakai-core-2.1.x-scottsdev branch incorporated
- Additional Setup Needed
**QA admin account (e.g. QAadmin)
**All login accounts that use kerberos names can be "provided" by the system (e.g. UserDirectoryProvider implemented).
**SU tool and Quartz tools are available in admin workspace/realm
**Testers have access to testable sites (via SU tool, admin workspace, as user QAAdmin, etc.)
**Run CourseManagement tests on existing batched sites perhaps? or run CourseManagementTests on new Fall Quarter Courses batched? - mView refresh times must be recorded
- Additional External-Banner Test Data Used for Provider Testing
A copy of test data that may be used for these test cases is attached: http://confluence.ucdavis.edu:8080/confluence/download/attachments/17096/200603TestData.xls
An example of a white list file to be used for course creation is: http://confluence.ucdavis.edu:8080/confluence/download/attachments/17226/white-list.txt
DEPENDENCIES
- All UCDavis Providers (UserDirectoryProvider, GroupProvider, and CourseManagementProvider) will be enabled to allow user accounts to be looked up automatically via kerberos name
- Distributed authentication using AFS should be enabled (SAK-143)
- Sakai-test (Sakai-dev for integration testing) database will be a either a clean copy or a copy of existing production data, so sakai-test will have to be backed up prior to testing,
Pradhu or Geeta will be contacted to schedule this. - We will be referencing sakai-test (banner test external, mViews..) for populating course data
ROLES AND RESPONSIBILITIES
- Testers (QA,students etc)*
- Admins and SmartSite Programmers
- Any of the following persons that may be available:
***Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
***Database/system admins (Geeta Rayala, Chris Callahan)
***Banner admins (e.g. Libby Bullock)
- Any of the following persons that may be available:
*All of the above users will have access to the admin realm in Sakai, in addition to:
***QAadmin
PROVIDER DOCUMENTATION
- Requirements
Provider requirements at: http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/UC+Davis+Data+Provisioning
- General Provider and Batching interaction documented at https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Requirements+for+Data+Provisioning+using+Providers+and+Batch+Processes
UNIT AND INTEGRATION TEST RESULTS
Unit Test Results
The JUnit tests results that have been previously run on sakai-dev (isaac) are at:
http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/SmartSite+Provider+Unit+Test+Results%2C+run+Sep+11%2C+2006
In summary, all unit tests passed!!
Integration Test Results
TEST PLAN PROCESS
This SmartSite Provider QA Test Plan will be coordinated by Pat Kava, with assistance from the programming group (e.g. Scott Amerson), and will document the results of QA testing. It is assumed that all bugs have been documented by the programming group as issues before handing off to QA.
Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).
Info |
---|
*NOTE: The order of the test cases will be as follows. Preliminary Case (0a): Accessing server Case 1: User Directory Provider Case 2: Group Provider Case 3: Course Management Provider |
TEST CASES
Info | ||
---|---|---|
| ||
Description = description of the test performed |
PRELIMINARY TESTS
Case 0 #Access,Login Test Cases
Case 0a
Note | ||
---|---|---|
| ||
Description = Test the capability of the user to access the test server Category of Testing = functionality,regression Prerequisite Steps Test Scenario 0a
Test Cleanup |
Test Results 0a
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Access to SmartSite, | Pat Kava | mpkava | access | access | P |
Case 0b
Note | ||
---|---|---|
| ||
Description = Test the capability of the user to log into the test server using their valid kerberos name Category of Testing = functionality,regression Prerequisite Steps Test Scenario 0b
Test Cleanup |
Test Results 0b
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Login to SmartSite, (DistAuth) | Pat Kava | mpkava | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Case 0c
Note | ||
---|---|---|
| ||
Description = Test the capability of the user to log into the test server using a valid SmartSite account Category of Testing = functionality,regression Prerequisite Steps Test Scenario 0c
Test Cleanup |
Test Results 0c
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Login to SmartSite, SmartSite account | Pat Kava | QAAdmin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Case 0d
Note | ||
---|---|---|
| ||
Description = Test the capability of the user to log into the test server using a valid SmartSite admin account and see SU Tool, and other admin functionality Category of Testing = functionality,regression Prerequisite Steps Test Scenario 0d
Test Cleanup |
Test Results 0d
For scenario 0d, the following actions:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Login to SmartSite, SmartSite admin account | Pat Kava | admin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Login to SmartSite, SmartSite admin account | Pat Kava | admin | see admin workspace | see admin workspace | P |
Login to SmartSite, SmartSite admin account | Pat Kava | admin | see Sites link | Sites link accessable | P |
Logout of SmartSite, using logout link | Pat Kava | admin | logout happens | logout happens | P |
Login to SmartSite, using QAadmin account | Pat Kava | QAadmin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
See SU page link | Pat Kava | QAadmin | SU page/tool available | SU page/tool accessable | P |
SU as some instructor using SU Tool | Pat Kava | QAadmin | SU by "View User Info" first, then "Become User" | SU happens correctly, become user | P |
Logout of SmartSite, using logout link | Pat Kava | SU'd person | logout happens | logout happens | P |
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
SmartSite admin account tools available | Pat Kava | admin,QAAdmin | administrative tools available | administrative tools work | P |
All Results for Case 0
Tip | ||
---|---|---|
| ||
results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-XXX |
USERDIRECTORYPROVIDER TESTS
Case 1 #UserDirectorProvider Test Cases
Note | ||
---|---|---|
| ||
Description = Test the capability of the UserDirectoryProvider to provide Kerberos authentication for WebDav Category of Testing = functionality,regression Prerequisite Steps
Test Scenario 1a, WebDav
Test Cleanup |
Test Results, Case 1a WebDav
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
WebDav authentication | Pat Kava | mpkava | authentication happens, file uploaded available for viewing, etc. | authentication occurs, file access available for upload/download | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-154
Note | ||
---|---|---|
| ||
Description = Test the capability of the UserDirectoryProvider to provide information about an externally provided user via the SU Tool Category of Testing = functionality,regression Prerequisite Steps
Test Scenario 1b, User lookup functionality
Test Cleanup |
Test Results Case 1b, User Lookup functionality
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
User information verification | Pat Kava | QAAdmin | look up user with valid kerberos name | User's name, email address and "registered" type are valid | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-155
GROUPPROVIDER TESTS
Case 2 #GroupProvider Test Cases
Note | ||
---|---|---|
| ||
Description = Test the capability of the GroupProvider to provide tabs for a user based on their enrollment/membership in sites Category of Testing = functionality,regression Prerequisite Steps
Test Scenario 2a, Tabs
-Additional step for programmers (Case 2ai), verify that data presented in tabs matches server log data-
Test Cleanup |
Test Results Case 2a, Tab functionality
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Site tab check | Pat Kava | fzolupon? | site tabs xxxx,xxxxx,xxxxx seen | all sites seen that have been created prior to logging in | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-156
Note | ||
---|---|---|
| ||
Description = Test the capability of the AuthxGroups to be updated more frequently than default OOTB Category of Testing = functionality,regression Prerequisite Steps
Test Scenario 2b, Realms refreshing
-Additional step for programmers (Case 2bi), verify that authz refresh is happening via the log output, and at what intervals-
Test Cleanup |
Test Results Case 2b, Realm refresh functionality
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Authz check | Pat Kava | fzolupon? | user xxxxx seen in roster? | user xxxx seen in SmartSite, site yyyyy roster after 5 minutes | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-132
COURSEMANAGEMENTPROVIDER TESTS
Case 3 #CourseManagementProvider Test Cases
Possible Test Data
Note | ||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||
|
Note | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Case 3a, Roster View Tests
Note | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, the roster from a site administrator perspective Category of Testing = functionality,regression Prerequisite Steps
|
Case 3b, Roster Update Tests
Note | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, students added to a Banner roster within an alotted time period Category of Testing = functionality,regression Prerequisite Steps
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, students dropped from a Banner roster within an alotted time period Category of Testing = functionality,regression Prerequisite Steps
Test Cleanup |
Test Results Case 3b(1),3b(2), Roster update functionality
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Roster add check | Pat Kava | x | roster added with Student xxxxx | Roster added with Student kerb name | P |
Roster drop check | Pat Kava | x | roster changed with Student xxxxx removed | Roster changed accordingly with kerb name xxxx removed | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-124
Case 3c, Roster Accuracy Tests
Note | |||||
---|---|---|---|---|---|
| |||||
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, members in a Roster (site member) list Category of Testing = functionality,regression Prerequisite Steps
Test Cleanup |
Test Results Case 3c Roster check
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Expected Result | Expected Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|---|---|---|
Roster check | Pat Kava | instructor x | site has accurate number of members | site has accurate instructor kerb name, user name | site has accurate students kerb name, user name | Format correct for member list | Member list formatted correctly | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-124
Case 3d, Instructor Change Tests
Note | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, instructors added to a Banner course within an alotted time period Category of Testing = functionality,regression Prerequisite Steps
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, instructors removed from a Banner course within an alotted time period Category of Testing = functionality,regression Prerequisite Steps
Description = Test the capability of the CourseMangementProvider to accurately reflect within the SmartSite UI, staff removed from a Banner course, and replaced with another instructor within an alotted time period Category of Testing = functionality,regression Prerequisite Steps
Test Cleanup |
Test Results Case 3d(1),3d(2),3d(3) Instructor update functionality
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
---|---|---|---|---|---|
Instructor add check | Pat Kava | x | CRN xxxx added with instructor xxxxx | Instructor added to list | P |
Instructor remove check | Pat Kava | x | CRN changed with instructor xxxxx removed | Member list changed with kerb name xxxx removed | P |
Instructor change staff to instructor check | Pat Kava | x | changed staff for course 2006-10-10037 to instructor kerb name mfharris | Member list changed with kerb name xxxx removed | P |
results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-202
Recording Results
The test results will be recorded in Jira by QA testers. They can be of table form or text form.
Feedback Mechanism
The above results will be analyzed by the QA team, SmartSite programming team, and SmartSite management to determine the quality of code. Results of this test should be discussed as near to the Monday and/or Code Migration meetings when available.