TEST PLAN OVERVIEW
Scope of testing
This testing is designed to document the unit and integration test results of the programming team. For certain test cases, as described in the Test Cases section, the level of testing previously performed to handing off to QA will be documented. The level of involvement by system admins and programmers will be detailed for each test case.
Purpose for this testing
The automatic course creation mechanism for SmartSite. Course sites, when done automatically via chron jobs, can be created and/or updated in several ways
- On-Demand List of courses (White list)
- Will create only a pre-defined list of course sites
- Not specific to a Term
- Excluded List of courses (Black list)
- Will not be created by any mechanism
- Auto-update of existing course sites with group update (e.g. provider id)
- Sets existing sites with the assigned provider id value, and will not provision a duplicate site with same provider id from full other batch run
- Full run of course creation for a designated term (will not batch white or black lists, as well as not create sites with Provider Id to be set)
Additional Items slated for testing
- Manually update provider Id for a site
- Need to be done as a an admin user, or one that has access to admin realm
- See the results of the realms being refreshed (Jon's patch)
KNOWN ISSUES / PREVIOUS BUGS FIXED
None at this time.
JIRAs that explain features to be tested in this plan
http://jira.ucdavis.edu:8080/jira/browse/SAK-23 (Add ability to customize pages and tools added to pages (both default and provisional) when provisioning course sites)
http://jira.ucdavis.edu:8080/jira/browse/SAK-25 (Remove admin user from sites, during course creation script)
http://jira.ucdavis.edu:8080/jira/browse/SAK-119 (Course White List)
http://jira.ucdavis.edu:8080/jira/browse/SAK-121 (Batch provisioning should make a toolset based on OOTB and other (provisional and contrib) tools)
http://jira.ucdavis.edu:8080/jira/browse/SAK-136 (Add Course title (from Banner) is the field presently mapped for course description information when provisioning sites. A course description (short and long) should be mapped from external data so that it can be referenced by certain course tools)
http://jira.ucdavis.edu:8080/jira/browse/SAK-140 (Course Black List)
http://jira.ucdavis.edu:8080/jira/browse/SAK-141 (Batching is done under a 'batch admin' user)
http://jira.ucdavis.edu:8080/jira/browse/SAK-146 (Set Provider Id For Manually Created Courses)
other non-batch related JIRAs covered:
http://jira.ucdavis.edu:8080/jira/browse/SAK-143 (enable distributed authentication using AFS file system)
TODO http://jira.ucdavis.edu:8080/jira/browse/SAK-133 (AuthZ refresh test)
JIRAs that reference test scenarios described in this test plan
TEST ENVIRONMENT
- Hardware Setup
Server Type
Location
database
bricker.ucdavis.edu with banner-test (external db), and sakai dev (internal db)
application
isaac.ucdavis.edu, sakai-dev
web server
url,
http://sakai-dev.ucdavis.edu
- Software Setup
run test under https://mware.ucdavis.edu/svn/ucdsakai/branches/sakai-core-2.1.x branch, with mods from https://mware.ucdavis.edu/svn/ucdakai/sakai-core-2.1.x-scottsdev branch incorporated
- Additional Setup Needed
**Batch admin account (e.g. ssbatchadmin)
**Programmers have access to testable sites (via SU tool, admin workspace)
**SU tool and Quartz tools are available in admin workspace/realm
**All login accounts that reference kerberos names can be "provided"
- External-Banner Test Data to be Used As Representative Course Scenarios
TODO|CRN with large enrollment number|2006-03-xxxxx|NA|NA|NA|x|Provider Id|
Scenario
Course
Roster Count
Instructor Kerb Name
Course Details (Catalog Root, Suffix, Section, Description)
Test Case Used In
Invalid CRNs
Invalid CRN (Number)
20061-06-111
NA
NA
NA
NA
White List, Provider Id
Invalid CRN (Number)
2006-06-12345
NA
NA
NA
NA
White List
Invalid CRN (Number)
2006-03-60024
NA
NA
NA
NA
White List
*additional* Invalid CRN (Number) available
1995-03-76288
NA
NA
NA
NA
White List
Invalid CRN (Bogus Text)
2006-sfs-sfsfdfsdfsfsfs
NA
NA
NA
NA
White List
No CRN (empty file)
null
NA
NA
NA
NA
White List,Black List, Provider Id
No CRN (null CRN value)
null
NA
NA
NA
NA
Provider Id
Valid CRNs
CRN with no enrollment
1995-01-76288
0
fzthorn, (Robert Thornton)
PLB, 092, 012, Internship
Winter Quarter 1995
White List
CRN with no enrollment
2004-03-60097
0
dlmiller (Douglas Miller)
ECN, 298 , 026, Group Study
Spring Quarter 2004
Provider Id
CRN with enrollment
2006-06-60005
11 (all valid knames)
acham (Christine Acham)
AAS, 198, S01, Directed Gp Study
Summer Special Session 2006
White List,Black List,Provider Id
CRN with enrollment
2006-01-40018
14 (all valid knames)
calymos (Calvin Lymos)
AAS, 154, 001, University Gospel Choir
Winter Quarter 2006
Black List
CRN with large enrollment number
2006-01-60001
84 (all valid knames)
ewlarsen (Eric Larsen)
LDA, 190 , 001, ProSeminar
Winter Quarter 2006
White List
TODO|CRN with large enrollment number, some student kerbs don't exist|2006-03-xxxxx|NA|NA|NA|x|Provider Id|Valid Term
Term with large number CRNs
2004-01
NA
NA
32573 sites
Winter Quarter 2004
Full Batch Run
DEPENDENCIES
- At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
- Distributed authentication using AFS (SAK-143)
- Sakai-dev will be used in its current state
- We will be referencing sakai-test external for populating course data
ROLES AND RESPONSIBILITIES
- Unit/Integration Testers (Programmers, Managers, etc)*
which would include any of the following persons that may be available:- Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
- Database mViews admin (Brian Donnelly)
COURSE SITE BATCHING DOCUMENTATION
- Requirements
The requirements for batching are documented at https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batching+Strategy
JUNIT TEST RESULTS
The JUnit tests results are located on the server at:
/var/sakai/ucd-data/xxxxx
Also, the test results can be found here https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results
All unit tests passed!!
TEST PLAN PROCESS
This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.
Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).
*NOTE: The order of the test cases will be as follows. White List Testing, Black List Testing, Provider Id Testing *
Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White and Black List testing scenarios
TEST CASES
Key for Test Case Scenarios
Prerequisite Steps = actions needed before performing any of the given tests
Category of Testing = the described testing is at what level?
Description = description of the test performed
White List
White List Testing
Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.
Category of Testing = unit, integration
Prerequisite Steps
- Note: The application does not need to be stopped for this test to occur
- Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
- Scott will que the batch job to run, only for the above list (5 min time frame needed)
- Scott will give maintain permissions on the applicable sites, to the appropriate users
- Brian = look up CRN xxxxx
- Sandra = look up CRN yyyyy
- Scott will assist users in accessing the site(s) if needed
- Thomas = view logs, look up CRN zzzzz
- James = look up CRN wwwww
- Scott will advise programmers and Pat that the sites have been created
- Thomas and Scott will monitor server activity while testers are using SmartSite.
Test Scenario
- Above users will perform functionality testing for each CRN, including :
**perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site
**perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Matrix for each site
**perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site
Test Cleanup
- Scott will remove all sites created in this step when all test cases are completed.
White List Test Results
Would yield an example Summary Test Result of:
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site xxxxx (created): Functionality Results |
samerson |
fzolupon |
All tests pass |
batching of course succeeded |
P |
Site xxxxx (created): Course Management Test Results |
samerson |
QA Admin |
All tests pass |
course content correct |
P |
Site xxxxx (created): Provider Id Test Results |
samerson |
QA Admin |
All tests pass |
provider id correct, roster number correct |
P |
or..
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site xxxxy (not created):Test Results |
samerson |
QA Admin |
All tests pass, not provisioned |
course not created, correct |
P |
Test Results, White List
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site xxxxx (created): Functionality Results |
bdonnell |
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site yyyyy (created): Functionality Results |
smavocet |
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site wwwww (created): Functionality Results |
tpamsler |
Test |
User Role |
User Acting As |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|---|---|
Site zzzzz (created): Functionality Results |
jrenfro |
Black List
Black List Testing
Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.
Category of Testing = unit, integration
Prerequisite Steps
Test Scenario
Test Cleanup
Black List Test Results
Action |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|
colB1 |
colB2 |
col3 |
col4 |
Update Existing Site with Provider Id
Update existing sites with appropriate provider id, enabling dynamic rosters (**if CourseMgmtProvider enabled**)
Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to
test the ability to run this at any time.
Category of Testing = unit, integration
Prerequisite Steps
Test Scenario
Test Cleanup
Provider Id Update Results
Action |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|
colB1 |
colB2 |
col3 |
col4 |
Site Functionality
Site functionality
Description = Test the capability of the system to add sites with the correct pages, tools, tabs, and look and feel.
Category of Testing = unit, integration, functionality
Prerequisite Steps
Test Scenario
- Look and Feel *
- Pages *
- Tools on each page *
- Click-through on tools *
- Order *
- Tabs *
Test Cleanup
Action |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|
colB1 |
colB2 |
col3 |
col4 |
Course Management Data Presented in Admin Tools
Course Management data present
Description = Test the course management content displayed in admin tools.
Category of Testing = unit, integration, functionality
Prerequisite Steps
- Site navigation *
- WorkSite Setup navigation *
- Site Info tool *
Test Cleanup
Action |
Result |
Expected Result |
Pass/Fail |
|---|---|---|---|
colB1 |
colB2 |
col3 |
col4 |
Recording Results
The test results will be recorded in the above summary tables by QA, and a summary report of both Regression and QA Tests will be created by Scott Amerson.