TEST PLAN DATE
Testing date:
TEST PLAN OVERVIEW
Scope of testing
This document is designed to guide Quality Assurance in the functionality testing of the provider mechanisms within SmartSite. The functionality of the provider mechanisms, as detailed by the given test cases in this plan, are driven by the functional requirements at :http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/Sakai+Providers.
Additionally, the level of involvement by QA, system administrators, and programmers will be detailed for each test case.
Purpose for this testing
The purpose of this test plan is to regression test the provider functionality, which in SmartSite includes:
UserDirectoryProvider
Support for WebDav authentication (via Kerberos)
Provide user account information (e.g. first,last,email,type) from outside data source (e.g. feed from Mothra)
GroupProvider
Assign roles to users who log in to SmartSite that correspond to Sakai internal roles (e.g. Student, Instructor) when applicable. These roles need to be queried from an external data source (e.g. Banner mViews), so that authorization can be determined in real-time.
Define the sites a user belongs to when they log into SmartSite.
CourseManagementProvider
Retrieve dynamic rosters for courses, including updates within specific time interval. Roster updates from Banner need to be displayed in the SmartSite application within a pre-determined acceptable time period as defined by the user acceptance testing group.
Display current instructor(s) course sites and display them as members in the course with appropriate role
Display site membership changes in a course site, including instructor adds/removals/updates
KNOWN ISSUES / PREVIOUS BUGS FIXED
Previously determined bugs have been fixed for this testing, they were identified as the following. They are also summarized in the table of Jiras referenced in this test plan.
Realms not refreshing frequent enough
Rosters not accurate with Banner
Drops not showing in SmartSite UI
Known (not fixed) issue #1 for Resources tool (e.g. using WebDav), is that when a file is uploaded to the system, metadata is written to the database before the file is stored on the file system. This is a feature of Sakai, which was detected during some resource discussion re:AFS file storage. For this reason, a resource may "seem" to be updated correctly, and listed in the resouce tool however the content cannot be seen. An http 404 error is thrown (No page can be found). This error is NOT fixed for this testing, just a known issue for Case 1a.
Known (not fixed) issue #2 is that whatever user that creates the sites when logged in is placed as a member in the site. Therefore, if ssbatchadmin created sites then the roster for such sites may appear off by 1.
Known (not fixed) issue #3 is that the instructor presented in the member list is not presented first. This is a default sorting behavior within SmartSite and will not be fixed.
Known (not fixed) issue #4 http://jira.ucdavis.edu:8080/jira/browse/SAK-202 is that the instructor presented in the member list may be still listed even if they have been "replaced"
JIRAs that explain features and past bugs fixed, to be tested in this plan
Summary of the Jiras related to this test plan, which have been fixed:
BATCH TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-184 (1000 row limit on associations (realms) for sakai internal query)
BATCH TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-185 (term abbreviation and term description do not match in site information tool)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-197 Support and Training web link should be present, going to gateway site on sakai2
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-201 authz group refresh interfering with batching of courses, noticed in provider integration testing
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-124 (Banner rosters not in sync with SmartSite UI)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-125 (drop student from Banner and not reflected in SmartSite UI)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-132 (realms not refreshing fast enough in the providers)
TEST ENVIRONMENT
Hardware Setup
INTEGRATION TESTING | SETUP |
Server Type | Location |
|---|---|
database | bricker.ucdavis.edu with banner-test (external db), and sakai dev (internal db) |
application | hanley.ucdavis.edu, sakai-dev |
web server | url, |
population of data referenced | 2006 Fall Quarter Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created) |
QA TESTING | SETUP |
Server Type | Location |
|---|---|
database | bricker.ucdavis.edu with banner-test (external db), and sakai test (internal db) |
application | isaac.ucdavis.edu, sakai-test |
web server | url, |
population of data referenced | 2006 Fall Quarter Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created) |
Note: The application does not need to be stopped for these tests to occur
Software Setup
run test under https://mware.ucdavis.edu/svn/ucdsakai/branches/sakai-core-2.1.x branch, with mods from https://mware.ucdavis.edu/svn/ucdakai/sakai-core-2.1.x-scottsdev branch incorporated
Additional Setup Needed
**QA admin account (e.g. QAadmin)
**All login accounts that use kerberos names can be "provided" by the system (e.g. UserDirectoryProvider implemented).
**SU tool and Quartz tools are available in admin workspace/realm
**Testers have access to testable sites (via SU tool, admin workspace, as user QAAdmin, etc.)
**Run CourseManagement tests on existing batched sites perhaps? or run CourseManagementTests on new Fall Quarter Courses batched?mView refresh times must be recorded
Additional External-Banner Test Data Used for Provider Testing
A copy of test data that may be used for these test cases is attached: http://confluence.ucdavis.edu:8080/confluence/download/attachments/17096/200603TestData.xls
An example of a white list file to be used for course creation is: http://confluence.ucdavis.edu:8080/confluence/download/attachments/17226/white-list.txt
DEPENDENCIES
All UCDavis Providers (UserDirectoryProvider, GroupProvider, and CourseManagementProvider) will be enabled to allow user accounts to be looked up automatically via kerberos name
Distributed authentication using AFS should be enabled (SAK-143)
Sakai-test (Sakai-dev for integration testing) database will be a either a clean copy or a copy of existing production data, so sakai-test will have to be backed up prior to testing,
Pradhu or Geeta will be contacted to schedule this.We will be referencing sakai-test (banner test external, mViews..) for populating course data
ROLES AND RESPONSIBILITIES
Testers (QA,students etc)*
Admins and SmartSite Programmers
Any of the following persons that may be available:
***Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
***Database/system admins (Geeta Rayala, Chris Callahan)
***Banner admins (e.g. Libby Bullock)
*All of the above users will have access to the admin realm in Sakai, in addition to:
***QAadmin
PROVIDER DOCUMENTATION
Requirements
Provider requirements at: http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/UC+Davis+Data+Provisioning
General Provider and Batching interaction documented at https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Requirements+for+Data+Provisioning+using+Providers+and+Batch+Processes
UNIT AND INTEGRATION TEST RESULTS
Unit Test Results
The JUnit tests results that have been previously run on sakai-dev (isaac) are at:
http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/SmartSite+Provider+Unit+Test+Results%2C+run+Sep+11%2C+2006
In summary, all unit tests passed!!
Integration Test Results
TEST PLAN PROCESS
This SmartSite Provider QA Test Plan will be coordinated by Pat Kava, with assistance from the programming group (e.g. Scott Amerson), and will document the results of QA testing. It is assumed that all bugs have been documented by the programming group as issues before handing off to QA.
Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).
*NOTE: The order of the test cases will be as follows.
Preliminary Case (0a): Accessing server
Preliminary Case (0b): Login access via DistAuth
Preliminary Case (0c): Login outside of DistAuth (e.g. internal account)
Preliminary Case (0d): Accessing admin functionality
Case 1: User Directory Provider
1a) Web Dav authentication
1b) User directory lookup functionality
Case 2: Group Provider
2a) Roles in courses verified by tabs
2b) Realm refresh via browser refresh button
Case 3: Course Management Provider
3a) Rosters from Banner viewed by instructor and administrator role in site
3b) Adds/drops of students from Banner seen in SmartSite UI
3c) Roster details accurate (e.g. kname, role, etc)
3d) Instructor changes for a course seen in participants list for a site
TEST CASES
Key for Test Case Scenarios
Description = description of the test performed
Category of Testing = the described testing is at what level?
Prerequisite Steps = actions needed before performing any of the given tests
PRELIMINARY TESTS
Case 0 #Access,Login Test Cases
Case 0a
Setup Test Case Scenario 0a, Accessing test server
Description = Test the capability of the user to access the test server
Category of Testing = functionality,regression
Prerequisite Steps
None
Test Scenario 0a
go to http://sakai-test.ucdavis.edu in your browser
Test Cleanup
None
Test Results 0a
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
Access to SmartSite, | Pat Kava | mpkava | access | access | P |
Case 0b
Setup Test Case Scenario 0b, Logging into test server via SecureWeb (DistAuth)
Description = Test the capability of the user to log into the test server using their valid kerberos name
Category of Testing = functionality,regression
Prerequisite Steps
User has valid kerberos name
If not, go to http://computingaccounts.ucdavis.edu to set one up
Test Scenario 0b
go to http://sakai-test.ucdavis.edu in your browser
user will click on login link on rhs
Test Cleanup
None
Test Results 0b
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
Login to SmartSite, (DistAuth) | Pat Kava | mpkava | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Case 0c
Setup Test Case Scenario 0c, Logging into test server via SmartSite's internal accounts
Description = Test the capability of the user to log into the test server using a valid SmartSite account
Category of Testing = functionality,regression
Prerequisite Steps
User has valid account name, password known
Test Scenario 0c
go to http://sakai-test.ucdavis.edu/portal/xlogin in your browser
user will enter username and password given
User will use admin account name
User will use QAAdmin account name
User will use sakaistudent1, etc.??
Test Cleanup
None
Test Results 0c
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
Login to SmartSite, SmartSite account | Pat Kava | QAAdmin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Case 0d
Setup Test Case Scenario 0d, Logging into test server via SmartSite's internal admin account, seeing necessary tools
Description = Test the capability of the user to log into the test server using a valid SmartSite admin account and see SU Tool, and other admin functionality
Category of Testing = functionality,regression
Prerequisite Steps
User has valid admin account name, password known
Test Scenario 0d
go to http://sakai-test.ucdavis.edu/portal/xlogin in your browser
user will enter username and password given
User will use admin account name
View site information for a given site (e.g. see Sites link in Admin workspace)
User will logout of admin account
User will use QAadmin account name
Become a user using SU Tool
Logout of using SU Tool
Test Cleanup
None
Test Results 0d
For scenario 0d, the following actions:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
Login to SmartSite, SmartSite admin account | Pat Kava | admin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
Login to SmartSite, SmartSite admin account | Pat Kava | admin | see admin workspace | see admin workspace | P |
Login to SmartSite, SmartSite admin account | Pat Kava | admin | see Sites link | Sites link accessable | P |
Logout of SmartSite, using logout link | Pat Kava | admin | logout happens | logout happens | P |
Login to SmartSite, using QAadmin account | Pat Kava | QAadmin | logged in, user in "Users Present" | logged in, name in "Users Present" | P |
See SU page link | Pat Kava | QAadmin | SU page/tool available | SU page/tool accessable | P |
SU as some instructor using SU Tool | Pat Kava | QAadmin | SU by "View User Info" first, then "Become User" | SU happens correctly, become user | P |
Logout of SmartSite, using logout link | Pat Kava | SU'd person | logout happens | logout happens | P |
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
SmartSite admin account tools available | Pat Kava | admin,QAAdmin | administrative tools available | administrative tools work | P |
All Results for Case 0
Results for Setup Provider Tests
results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-XXX
USERDIRECTORYPROVIDER TESTS
Case 1 #UserDirectorProvider Test Cases
WebDav Testing, Case 1a
Description = Test the capability of the UserDirectoryProvider to provide Kerberos authentication for WebDav
Category of Testing = functionality,regression
Prerequisite Steps
One file must be uploaded via the Resources tool prior to using WebDav
Test Scenario 1a, WebDav
Log in as any user having access to any site
Follow WebDav instructions based on instructions in Resource area
* Use "Manage Multiple Resources" link in Resources tool to access WebDav
Data to Be Used:
QA determines
Test Cleanup
None
Test Results, Case 1a WebDav
Would yield an example Summary Test Result of:
Test | User | User Acting As | Result | Expected Result | Pass/Fail |
|---|---|---|---|---|---|
WebDav authentication | Pat Kava | mpkava | authentication happens, file uploaded available for viewing, etc. | authentication occurs, file access available for upload/download |