Versions Compared

Old Version 67

changes.mady.by.user Scott Amerson

Saved on

compared with

New Version 68

changes.mady.by.user Scott Amerson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

TEST PLAN DATE

Testing date:

TEST PLAN OVERVIEW

Scope of testing
This document is designed to guide Quality Assurance in the functionality testing of the provider mechanisms within SmartSite. The functionality of the provider mechanisms, as detailed by the given test cases in this plan, are driven by the functional requirements at :http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/Sakai+Providers.

Additionally, the level of involvement by QA, system administrators, and programmers will be detailed for each test case.

Tip
titlePurpose for this testing

The purpose of this test plan is to regression test the provider functionality, which in SmartSite includes:

UserDirectoryProvider

  1. Support for WebDav authentication (via Kerberos)
  2. Provide user account information (e.g. first,last,email,type) from outside data source (e.g. feed from Mothra)

GroupProvider

  1. Assign roles to users who log in to SmartSite that correspond to Sakai internal roles (e.g. Student, Instructor) when applicable. These roles need to be queried from an external data source (e.g. Banner mViews), so that authorization can be determined in real-time.
  2. Define the sites a user belongs to when they log into SmartSite.

CourseManagementProvider

  1. Retrieve dynamic rosters for courses
  2. Display current instructor(s) for courses
  3. Display Banner course details in the SmartSite UI
  4. Update roster dynamically for a course given a specific time interval. Roster updates from Banner need to be displayed in the SmartSite application within a pre-determined allowable time period as defined by the user acceptance testing group.

KNOWN ISSUES / PREVIOUS BUGS FIXED

Previously determined bugs have been fixed for this testing, they were identified as the following. They are also summarized in the table of Jiras referenced in this test plan.

  • Realms not refreshing frequent enough
  • Rosters not accurate with Banner
  • Drops not showing in SmartSite UI

*Known issue for Resources tool (e.g. using WebDav), is that when a file is uploaded to the system, metadata is written to the database before the file is stored on the file system. This is a feature of Sakai, which was detected during some resource discussion re:AFS file storage. For this reason, a resource may "seem" to be updated correctly, and listed in the resouce tool however the content cannot be seen. An http 404 error is thrown (No page can be found). This error is NOT fixed for this testing, just a known issue for Case 1a.

Info
titleJIRAs that explain features and past bugs fixed, to be tested in this plan

Summary of the Jiras related to this test plan, which have been fixed:
BATCH TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-184 (1000 row limit on associations (realms) for sakai internal query)
BATCH TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-185 (term abbreviation and term description do not match in site information tool)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-124 (Banner rosters not in sync with SmartSite UI)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-125 (drop student from Banner and not reflected in SmartSite UI)
PREVIOUS PROVIDER TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-132 (realms not refreshing fast enough in the providers)

Note
titleJIRAs that reference test scenarios, "QA Tests" described in this test plan

http://jira.ucdavis.edu:8080/jira/browse/SAK-155 When users log into Sakai, their user information is looked up externally in Mothra, LDAP, or some other central datasource for campus
http://jira.ucdavis.edu:8080/jira/browse/SAK-156 User logs into Sakai, and sees the tabs associated with their enrollment (roles) in courses from Banner, as well as sites internally defined in Sakai (e.g. projects)
http://jira.ucdavis.edu:8080/jira/browse/SAK-157 Course information must be pulled dynamically from Banner, within Sakai
http://jira.ucdavis.edu:8080/jira/browse/SAK-154 User is able to used WebDav in Sakai, with their Kerberos username and password

TEST ENVIRONMENT

  • Hardware Setup

    Server Type

    Location

    database

    bricker.ucdavis.edu with banner-test (external db), and sakai test (internal db)

    application

    isaac.ucdavis.edu, sakai-test

    web server

    url,

    http://sakai-test.ucdavis.edu

    population of data referenced

    2006 Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created)

  1. Note: The application does not need to be stopped for these tests to occur
  • Additional Setup Needed
    **QA admin account (e.g. QAadmin)
    **All login accounts that use kerberos names can be "provided" by the system (e.g. UserDirectoryProvider implemented).
    **SU tool and Quartz tools are available in admin workspace/realm
    **Testers have access to testable sites (via SU tool, admin workspace, as user QAAdmin, etc.)

DEPENDENCIES

  • All UCDavis Providers (UserDirectoryProvider, GroupProvider, and CourseManagementProvider) will be enabled to allow user accounts to be looked up automatically via kerberos name
  • Distributed authentication using AFS should be enabled (SAK-143)
  • Sakai-test database will be a copy of existing production data, so sakai-test will have to be backed up prior to testing
  • We will be referencing sakai-test external for populating course data

ROLES AND RESPONSIBILITIES

  • Testers (QA,students etc)*
  • Admins and SmartSite Programmers
    • Any of the following persons that may be available:
      ***Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
      ***Database/system admins (Geeta, Chris Callahan)

*All of the above users will have access to the admin realm in Sakai, in addition to:
***QAadmin

PROVIDER DOCUMENTATION

UNIT AND INTEGRATION TEST RESULTS

Unit Test Results

The JUnit tests results that have been previously run on sakai-dev (isaac) are at:
http://confluence.ucdavis.edu:8080/confluence/display/UCDSAKAI/SmartSite+Provider+Unit+Test+Results%2C+run+Sep+11%2C+2006
In summary, all unit tests passed!!

Integration Test Results

TEST PLAN PROCESS

This SmartSite Provider QA Test Plan will be coordinated by Pat Kava, with assistance from the programming group (e.g. Scott Amerson), and will document the results of QA testing. It is assumed that all bugs have been documented by the programming group as issues before handing off to QA.

Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).

Info

*NOTE: The order of the test cases will be as follows.

Setup Case (0a): Accessing server
Setup Case (0b): Login access via DistAuth
Setup Case (0c): Login outside of DistAuth (e.g. internal account)
Setup Case (0d): Accessing admin functionality

Case 1: User Directory Provider
1a) Web Dav authentication
1b) User directory lookup functionality

Case 2: Group Provider
2a) Roles in courses verified by tabs
2a,i) Roles in courses verified by system log output (system admins)
2b) Realm refresh via browser refresh button
2b,i) Realm refresh seen in system log output (system admins)

Case 3: Course Management Provider
3a) Rosters from Banner viewed by instructor and administrator role in site
3b) Adds/drops/updates from Banner seen in SmartSite UI
3c) Updates of course instructor information propogated into SmartSite UI
3d) Course Site information viewed by instructor seen in site summary details
3e) Roster details accurate (e.g. kname, role, etc)

TEST CASES

Info
titleKey for Test Case Scenarios

Description = description of the test performed
Category of Testing = the described testing is at what level?
Prerequisite Steps = actions needed before performing any of the given tests

PRELIMINARY TESTS

Case 0 #Access,Login Test Cases

Case 0a

Note
titleSetup Test Case Scenario 0a, Accessing test server

Description = Test the capability of the user to access the test server

Category of Testing = functionality,regression

Prerequisite Steps
None

Test Scenario 0a

  1. go to http://sakai-test.ucdavis.edu in your browser

Test Cleanup
None

Test Results 0a

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

Access to SmartSite,

Pat Kava

mpkava

access

access

P

Case 0b

Note
titleSetup Test Case Scenario 0b, Logging into test server via SecureWeb (DistAuth)

Description = Test the capability of the user to log into the test server using their valid kerberos name

Category of Testing = functionality,regression

Prerequisite Steps
User has valid kerberos name
If not, go to http://computingaccounts.ucdavis.edu to set one up

Test Scenario 0b

  1. go to http://sakai-test.ucdavis.edu in your browser
  2. user will click on login link on rhs

Test Cleanup
None

Test Results 0b

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

Login to SmartSite, (DistAuth)

Pat Kava

mpkava

logged in, user in "Users Present"

logged in, name in "Users Present"

P

Case 0c

Note
titleSetup Test Case Scenario 0c, Logging into test server via SmartSite's internal accounts

Description = Test the capability of the user to log into the test server using a valid SmartSite account

Category of Testing = functionality,regression

Prerequisite Steps
User has valid account name, password known

Test Scenario 0c

  1. go to http://sakai-test.ucdavis.edu/portal/xlogin in your browser
  2. user will enter username and password given
  3. User will use admin account name
  4. User will use QAAdmin account name
  5. User will use sakaistudent1, etc.??

Test Cleanup
None

Test Results 0c

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

Login to SmartSite, SmartSite account

Pat Kava

QAAdmin

logged in, user in "Users Present"

logged in, name in "Users Present"

P

Case 0d

Note
titleSetup Test Case Scenario 0d, Logging into test server via SmartSite's internal admin account, seeing necessary tools

Description = Test the capability of the user to log into the test server using a valid SmartSite admin account and see SU Tool, and other admin functionality

Category of Testing = functionality,regression

Prerequisite Steps
User has valid admin account name, password known

Test Scenario 0d

  1. go to http://sakai-test.ucdavis.edu/portal/xlogin in your browser
  2. user will enter username and password given
  3. User will use admin account name
  4. View site information for a given site (e.g. see Sites link in Admin workspace)
  5. User will logout of admin account
  6. User will use QAAdmin account name
  7. Become a user using SU Tool
  8. Logout of using SU Tool

Test Cleanup
None

Test Results 0d
For scenario 0d, the following actions:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

Login to SmartSite, SmartSite admin account

Pat Kava

admin

logged in, user in "Users Present"

logged in, name in "Users Present"

P

Login to SmartSite, SmartSite admin account

Pat Kava

admin

see admin workspace

see admin workspace

P

Login to SmartSite, SmartSite admin account

Pat Kava

admin

see Sites link

Sites link accessable

P

Logout of SmartSite, using logout link

Pat Kava

admin

logout happens

logout happens

P

Login to SmartSite, using QAAdmin account

Pat Kava

QAAdmin

logged in, user in "Users Present"

logged in, name in "Users Present"

P

See SU page link

Pat Kava

QAAdmin

SU page/tool available

SU page/tool accessable

P

SU as some instructor using SU Tool

Pat Kava

QAAdmin

SU by "View User Info" first, then "Become User"

SU happens correctly, become user

P

Logout of SmartSite, using logout link

Pat Kava

SU'd person

logout happens

logout happens

P

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

SmartSite admin account tools available

Pat Kava

admin,QAAdmin

administrative tools available

administrative tools work

P

All Results for Case 0

Tip
titleResults for Setup Provider Tests

results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-XXX

USERDIRECTORYPROVIDER TESTS

Case 1 #UserDirectorProvider Test Cases

Note
titleWebDav Testing, Case 1a

Description = Test the capability of the UserDirectoryProvider to provide Kerberos authentication for WebDav

Category of Testing = functionality,regression

Prerequisite Steps

  1. One file must be uploaded via the Resources tool prior to using WebDav

Test Scenario 1a, WebDav

  1. Log in as any user having access to any site
  2. Follow WebDav instructions based on instructions in Resource area
  3. * Use "Manage Multiple Resources" link in Resources tool to access WebDav
  • Data to Be Used:
    QA determines

Test Cleanup
None

Results, Case 1a WebDav

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

WebDav authentication

Pat Kava

mpkava

authentication happens, file uploaded available for viewing, etc.

authentication occurs, file access available for upload/download

P

results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-154

Note
titleWebDav Testing, Case 1b

Description = Test the capability of the UserDirectoryProvider to provide information about an externally provided user via the SU Tool

Category of Testing = functionality,regression

Prerequisite Steps

  1. SU tool available to tester

Test Scenario 1b, User lookup functionality

  1. Log in as any user having administrative access (access to admin realm)
  2. Use a given kerberos name available via test data (e.g. instructor)
  3. Enter kerberos name, and click "View User Info"
  4. Verify user information from whois.ucdavis.edu and/or email.ucdavis.edu
  • Data to Be Used:
    QA determines

Test Cleanup
None

Test Results Case 1b, User Lookup functionality

Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

User information verification

Pat Kava

QAAdmin

look up user with valid kerberos name

User's name, email address and "registered" type are valid

P

results: Recorded in Jira http://jira.ucdavis.edu:8080/jira/browse/SAK-155

GROUPPROVIDER TESTS

Case 2 #GroupProvider Test Cases

Recording Results

The test results will be recorded in Jira by QA testers. They can be of table form or text form.

Feedback Mechanism

The above results will be analyzed by the QA team, SmartSite programming team, and SmartSite management to determine the quality of code. Results of this test should be discussed as near to the Monday and/or Code Migration meetings when available.