Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

TEST PLAN DATE

Testing date: Scheduled 9/14/06, early morning

TEST PLAN OVERVIEW

Scope of testing
This testing is designed to guide Quality Assurance in the functionality testing of the batch creation mechanism of course sites in SmartSite. The functionality of the batching mechanism, which can be run at any time by an application administrator, is detailed by the given test cases in this plan. Additionally, the level of involvement by QA, system administrators, and programmers will be detailed for each test case.

...

Info
titleJIRAs that explain features to be tested in this plan

TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-184185 (1000 row limit on associations (realms) for sakai internal query)
TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-185 (term abbreviation and term description do not match in site information tool)
http://jira.ucdavis.edu:8080/jira/browse/SAK-23 (Add ability to customize pages and tools added to pages (both default and provisional) when provisioning course sites)
http://jira.ucdavis.edu:8080/jira/browse/SAK-25 (Remove admin user from sites, during course creation script)
http://jira.ucdavis.edu:8080/jira/browse/SAK-119 (Course White List)
http://jira.ucdavis.edu:8080/jira/browse/SAK-121 (Batch provisioning should make a toolset based on OOTB and other (provisional and contrib) tools)
http://jira.ucdavis.edu:8080/jira/browse/SAK-136 (Add Course title (from Banner) is the field presently mapped for course description information when provisioning sites. A course description (short and long) should be mapped from external data so that it can be referenced by certain course tools)
http://jira.ucdavis.edu:8080/jira/browse/SAK-140 (Course Black List)
http://jira.ucdavis.edu:8080/jira/browse/SAK-141 (Batching is done under a 'batch admin' user)
http://jira.ucdavis.edu:8080/jira/browse/SAK-146 (Set Provider Id For Manually Created Courses)

other non-batch related JIRAs covered:
http://jira.ucdavis.edu:8080/jira/browse/SAK-143 (enable distributed authentication using AFS file system)
http://jira.ucdavis.edu:8080/jira/browse/SAK-133 (AuthZ refresh test, https://jira.ucdavis.edu:8444/jira/browse/SAK-174)

Note
titleJIRAs that reference test scenarios, "QA Tests" described in this test plan

https://jira.ucdavis.edu:8444/jira/browse/SAK-119 = White List https://jira.ucdavis.edu:8444/jira/browse/SAK-140 = Black List https://jira.ucdavis.edu:8444/jira/browse/SAK-146 = Setting Provider Id

Warning
titleModifications to this Test Plan After Review with QA, 9/12/06

No Scenario 2b covered here, no Provider id consulting Black list functionality to be tested

TEST ENVIRONMENT

  • Hardware Setup

    Server Type

    Location

    database

    bricker.ucdavis.edu with banner-test (external db), and sakai test (internal db)

    application

    isaac.ucdavis.edu, sakai-test

    web server

    url,

    http://sakai-test.ucdavis.edu

    population of data referenced

    2006 Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created)

...

  • Additional Setup Needed
    **Batch admin account (e.g. ssbatchadmin)
    **QA admin account (e.g. QAadmin)
    **All login accounts that use kerberos names can be "provided" by the system (e.g. UserDirectoryProvider implemented).
    **SU tool and Quartz tools are available in admin workspace/realm
    **Pat Kava will be given access to admin realm
    **Testers have access to testable sites (via SU tool, admin workspace)

...

Info

*NOTE: The order of the test cases will be as follows.
Case 1: White List Testing,
Case 2 (a -and b-): Black List Testing,
Case 3 (a and b) Provider Id Testing

  1. Site Functionality Testing (Case 4), Course Management Testing (Case 5), and Provider Id (Case 3a) testing are each part of the White List testing scenario.
  2. In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.

...

Note
titleWhite List Testing

Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.

Category of Testing = unit, integration,functionality,regression

Prerequisite Steps

  1. Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
  2. Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
  3. Scott will give maintain permissions on the applicable sites, to the appropriate users
  • QA team = look up CRN as instructor for 2006-03-93158
  • QA team = look up CRN as instructor for 2006-06-60005
  • Scott will assist users in accessing the site(s) if needed
  • QA team = look for CRN 20061-06-111, 2006-03-6002499999
  • QA team = look up CRN 2006-sfs-sfsfdfsdfsfsfs
  1. Scott will advise testers that the batch has been run and sites have been created
  2. Scott and other programmers will monitor server activity while testers are using sakai-test.

Test Scenario

  1. Above users will perform functionality testing for each CRN, including :
    **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site+Functionality+Tests%2C+SmartSiteSite+BatchFunctionality+Integration+TestingTest+Results+Matrix for each site
    **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Results+Matrix for each site
    **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Results+Matrix for each site
  • Data to Be Used:

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    Invalid CRNs

    Invalid CRN (Number)

    20061-06-111

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Number)

    2006-03-6002499999

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Bogus Text)

    2006-sfs-sfsfdfsdfsfsfs

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    *additional data (see attachment),not may used for this testing* Invalid CRN (Number) available

    2006-03-67952

    10 (all valid knames)

    shershow (Scott Shershow)

    CRI 200B 002 Problems in Crit Theory

    Spring Quarter 2006

    White List

    white-list.txt

    Valid CRNs

    CRN with no enrollment

    2006-03-93158

    10 (all valid knames)

    klradke (Kathryn Radke)

    SAS 197T 001 Tutoring Sci & Society

    Spring Quarter 19952006

    White List

    white-list.txt

    CRN with enrollment

    2006-06-60005

    11 (all valid knames)

    acham (Christine Acham)

    AAS, 198, S01, Directed Gp Study

    Summer Special Session 2006

    White List

    white-list.txt

    CRN with enrollment

    2006-03-93307

    10 large enrollment (all valid knames)

    fzvasili (S Spyridakis)

    HIS 111C 001 Ancient History

    Spring Quarter 2006

    White List

    white-list.txt,black-list.txt

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

...

Update existing sites with appropriate provider id, enabling dynamic rosters (**
Note
titleBlack List Testing
Note
title

Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.

Category of Testing = unit, integration, functionality,regression

Prerequisite Steps

  1. Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
  2. Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
  3. Scott will que the batch job to run (White List), (5 min time elapse needed)
  4. Scott will advise QA team that the batch has been run
  5. Scott will monitor server activity while testers are using sakai-test.

-Prerequisite Steps needed for Scenario #2b only (in addition to Scenario #2a steps)-
-# Scott will update existing hand created sites manually (TBD site ids) with an associated CRN as the provider id-

Test Scenarios
*Scenario #2a = Black list consulted when white list batch run is executed*

  • QA team = will look for CRN 2006-06-60245 as user admin. This CRN will be listed in the White List files only
  • QA team = will look for CRN 2006-01-60001 that is listed both in the white-list.txt and black-list.txt files
  • QA team = will look for both CRN 2006-01-40018 that are listed both in the white-list.txt and black-list.txt files
  • QA team Scott = view logs, internal sakai db, will look for both CRN 2006-03-82882, 2006-03-83634 that are listed in the black-list.txt files only
  • Scott = help as needed
  • Data to Be Used:

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    Valid CRNs

    CRN with no enrollment

    2006-06-60245

    0

    fzduts (Dennis Dutschke)

    ITA 198 S01 Dir Group Study

    Summer Special Session 2006

    Black List

    white-list.txt

    CRN with large enrollment number

    2006-01-60001

    84 (all valid knames)

    ewlarsen (Eric Larsen)

    LDA, 190 , 001, ProSeminar

    Winter Quarter 2006

    Black List

    white-list.txt,black-list.txt

    CRN with enrollment

    2006-01-40018

    14 (all valid knames)

    calymos (Calvin Lymos)

    AAS, 154, 001, University Gospel Choir

    Winter Quarter 2006

    Black List

    white-list.txt,black-list.txt

    CRN with enrollment

    2006-03-82882

    10

    ltcmwc (Mark Connelly)

    MSC 141 002 Army Mngmt Systems

    Spring Quarter 2006

    Black List

    black-list.txt

    CRN with enrollment

    2006-03-83634

    10 (all valid knames)

    eschang (Ernest Chang)

    NPB 141P 001 Res In Physio Adapt

    Spring Quarter 2006

    Black List

    black-list.txt

*Scenario #2b = Black list consulted when existing site already has provider id that matches black list entry*

  • QA team = will look for CRN 2006-03-69857 as user ccjon. This CRN will be listed in the Black List, but have no existing site that has the provider ids matching the CRNs
  • QA team = will look for CRN 2006-03-65306,2006-06-60435 that is listed both in the provider id reference for sites ^^^^^^^^, and (((((((( and black-list.txt file
  • Scott = help QA
  • Data to Be Used:

    Scenario

    Course

    Site id

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    Valid CRNs

    CRN with enrollment

    2006-03-69857

    10

    jrlund (Jay Lund)

    ECI 299 019 Research

    Spring Quarter 2006

    Black List

    black-list.txt

    CRN with enrollment (large)

    2006-03-65306

    177 (all valid knames)

    fzsegel (Leigh Segel)

    BIS 103 002 Bioenergetics/Metabolism

    Spring Quarter 2006

    Black List

    white-list.txt,black-list.txt

    CRN with no enrollment

    2006-06-60435

    0

    fzblanch (Marc Blanchard)

    COM 152 S01 Literature Of Americas

    Summer Special Session 2006

    Black List

    white-list.txt,black-list.txt

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-140

Black List Test Results
Would yield an example Summary Test Result of:

Test

User

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxy (not created):Test Results

samerson

QA Admin

All tests pass, not provisioned

course not created, correct

P

Case 3 #Update Existing Site with Provider Id

Case 3 #Update Existing Site with Provider Id

Note
titleUpdate existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**)

Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to
test the ability to run this at any time.

Category of Testing = unit, integration, functionality,regression

Prerequisite Steps
Results Recording Steps summarized in:
**perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site

  1. Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file here
  2. Scott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)
  3. Scott will advise testers that the batch has been run
  4. Scott will monitor server activity while testers are using sakai-devtest.

*Data To Be Used:

Scenario

Course

SiteId or Title

Instructor Kerb Name

Course Details (Catalog Root, Suffix, Section, Description)

Term Identifier

Test Case Used In

File Associated

InValid CRNs

No CRN (null CRN value)

null

NA

NA

NA

NA

Provider Id

provider-id.txt

Valid CRNs and Sites

CRN valid

2006-03-93327

First Test Course To Be Updated By Batch

stenzel (John Stenzel)

MST 020C A02 Late Med & Early Modern (roster=10)

Spring Quarter 2006

Provider Id

provider-id.txt

CRN valid

2006-03-92472

[Second Test Course To Be updated By Batch

paully (Paul Manriquez)

WLD 910 001 Wkload - Math Pre 16A (roster=10)

Spring Quarter 2006

Provider Id

provider-id.txt

CRN valid

2006-03-83569

Third Test Course To Be Updated By Batch

goldberg (Jack Goldberg)

NPB 101 001 Systemic Physiology (roster=553)

Spring Quarter 2006

Provider Id

provider-id.txt

Scenario 3a (Batch updates existing sites provider id)
Test Scenario

  • QA team = will look for site First Test Course To Be Updated By Batch's providerid and roster, logging in as admin
  • QA team = will look for site Second Test Course To Be updated By Batch's providerid and roster, logging in as admin
  • QA team= will look for site Third Test Course To Be Updated By Batch's providerid and roster, logging in as admin
  • QA team = will validate provider ids and rosters
  • QA team = perform manual updates of provider ids
  • Scott = help as needed

Scenario 3b (Manual Process Check (update provider ids for sites manually)
Test Scenario
*Scott will have create 3 sites:
First Test Course To Have Manual Provider Id
Second Test Course To Have Manual Provider Id
Third Test Course To Have Manual Provider Id
-# After site First Test Course To Have Manual Provider Id,Second Test Course To Have Manual Provider Id, and Third Test Course To Have Manual Provider Id's provider ids are validated,-

QA will update site First Test Course To Have Manual Provider Id with:
providerId = 2006-03-73656 (10 enrolled)
and site Second Test Course To Have Manual Provider Id with:
providerId = Id with:
providerId = 2006-03-F0001 made up of 2006-03-62179,2006-03-62178,2006-03-65286 62180 (236 51 enrolled, multi-crn)
and site Third Test Course To Have Manual Provider Id with
providerId = 2006-03-80102 (177 178 enrolled)

  1. Scott and Pat Kava will check all sites rosters based on their associated provider id

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

...

Note
titleSite functionality

Description = Test the capability of the system to add sites with the correct pages, tools, tabs, and look and feel.

Category of Testing = unit, integration, functionality, regression

Prerequisite Steps
Results Recording Steps summarized in:
**perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site+Functionality+Tests%2C+SmartSiteSite+BatchFunctionality+Integration+TestingTest+Results+Matrix for each site
looking for expected results
Test Scenario

Panel
borderColor#ccc
bgColorF7D6C1
titleBGColor#efefef
borderStyledashed
titleIndividual Functionality TestsborderStyledashed
-

Look and Feel

- *


Pages

*


Tools on each page

*


Click-through on tools

*


Order

*


Tabs

*

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

...

Note
titleCourse Management data present

Description = Test the course management content displayed in admin tools.

Category of Testing = unit, integration, functionality, regression

Prerequisite Steps
Results Recording Steps summarized in:
**perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Results+Matrix for each site
looking for expected results

Panel
borderColor#ccc
bgColorwhite
titleBGColorefefef
borderStyledashed
titleTest Scenario
borderStyledashed
Site navigation *

Site Info tool
WorkSite Setup navigation

*Site Info tool *

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

...