...
- External-Banner Test Data to be Used As Representative Course Scenarios (see each test case)
Scenario | Course | Roster Count | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In |
---|
Invalid CRNs |
Invalid CRN (Number) | 20061-06-111 | NA | NA | NA | NA | White List, Provider Id |
Invalid CRN (Number) | 2006-06-12345 | NA | NA | NA | NA | White List |
Invalid CRN (Number) | 2006-03-60024 | NA | NA | NA | NA | White List |
*additional* Invalid CRN (Number) available | 1995-03-76288 | NA | NA | NA | NA | White List |
Invalid CRN (Bogus Text) | 2006-sfs-sfsfdfsdfsfsfs | NA | NA | NA | NA | White List |
No CRN (empty file) | null | NA | NA | NA | NA | White List,Black List, Provider Id |
No CRN (null CRN value) | null | NA | NA | NA | NA | Provider Id |
Valid CRNs |
CRN with no enrollment | 1995-01-76288 | 0 | fzthorn, (Robert Thornton) | PLB, 092, 012, Internship | Winter Quarter 1995 | White List |
CRN with no enrollment | 2004-03-60097 | 0 | dlmiller (Douglas Miller) | ECN, 298 , 026, Group Study | Spring Quarter 2004 | Provider Id |
CRN with enrollment | 2006-06-60005 | 11 (all valid knames) | acham (Christine Acham) | AAS, 198, S01, Directed Gp Study | Summer Special Session 2006 | White List,Black List,Provider Id |
CRN with enrollment | 2006-01-40018 | 14 (all valid knames) | calymos (Calvin Lymos) | AAS, 154, 001, University Gospel Choir | Winter Quarter 2006 | Black List |
CRN with large enrollment number | 2006-01-60001 | 84 (all valid knames) | ewlarsen (Eric Larsen) | LDA, 190 , 001, ProSeminar | Winter Quarter 2006 | White List |
TODO|CRN with large enrollment number|2006-03-xxxxx|NA|NA|NA|x|Provider Id|
TODO|CRN with large enrollment number, some student kerbs don't exist|2006-03-xxxxx|NA|NA|NA|x|Provider Id|
- A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.
DEPENDENCIES
- At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
- Distributed authentication using AFS (SAK-143)
- Sakai-dev will be used in its current state
- We will be referencing sakai-test external for populating course data
ROLES AND RESPONSIBILITIES
- Unit/Integration Testers (Programmers, Managers, etc)*
which would include any of the following persons that may be available: - Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
- Database mViews admin (Brian Donnelly)
COURSE SITE BATCHING DOCUMENTATION
...
- A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.
DEPENDENCIES
- At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
- Distributed authentication using AFS (SAK-143)
- Sakai-dev will be used in its current state
- We will be referencing sakai-test external for populating course data
ROLES AND RESPONSIBILITIES
- Unit/Integration Testers (Programmers, Managers, etc)*
which would include any of the following persons that may be available: - Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
- Database mViews admin (Brian Donnelly)
COURSE SITE BATCHING DOCUMENTATION
JUNIT TEST RESULTS
The JUnit tests results are located on the server at:
/var/sakai/ucd-data/course-batch-unit-tests-082006
Also, the test results can be found here https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results
All unit tests passed!!
TEST PLAN PROCESS
This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.
Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).
Info |
---|
*NOTE: The order of the test cases will be as follows. White List Testing, Black List Testing, Provider Id Testing, Full Batch Run * - Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White List testing scenario.
- In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.
|
TEST CASES
Info |
---|
title | Key for Test Case Scenarios |
---|
|
Prerequisite Steps = actions needed before performing any of the given tests Category of Testing = the described testing is at what level? Description = description of the test performed |
White List
Note |
---|
|
Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created. Category of Testing = unit, integration Prerequisite Steps - Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
- Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
- Scott will give maintain permissions on the applicable sites, to the appropriate users
- Brian = look up CRN xxxxx
- Sandra = look up CRN yyyyy
- Scott will assist users in accessing the site(s) if needed
- Thomas = view logs, look up CRN zzzzz
- Jon G. = look up CRN wwwww
- Scott will advise programmers that the batch has been run and sites have been created
- Thomas and Scott will monitor server activity while testers are using sakai-dev.
Test Scenario - Above users will perform functionality testing for each CRN, including :
**perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Matrix for each site **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/
|
...
JUNIT TEST RESULTS
The JUnit tests results are located on the server at:
/var/sakai/ucd-data/course-batch-unit-tests-082006
Also, the test results can be found here https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results
All unit tests passed!!
TEST PLAN PROCESS
This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.
Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).
Info |
---|
*NOTE: The order of the test cases will be as follows. White List Testing, Black List Testing, Provider Id Testing, Full Batch Run * - Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White List testing scenario.
- In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.
|
TEST CASES
Info |
---|
title | Key for Test Case Scenarios |
---|
|
Prerequisite Steps = actions needed before performing any of the given tests Category of Testing = the described testing is at what level? Description = description of the test performed |
White List
Note |
---|
|
Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created. Category of Testing = unit, integration Prerequisite Steps - Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
- Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
- Scott will give maintain permissions on the applicable sites, to the appropriate users
- Brian = look up CRN xxxxx
- Sandra = look up CRN yyyyy
- Scott will assist users in accessing the site(s) if needed
- Thomas = view logs, look up CRN zzzzz
- Jon G. = look up CRN wwwww
- Scott will advise programmers that the batch has been run and sites have been created
- Thomas and Scott will monitor server activity while testers are using sakai-dev.
Test Scenario - Above users will perform functionality testing for each CRN, including :
**perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Matrix for each site **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site
- Data to Be Used:
Scenario | Course | Roster Count | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In | File Associated |
---|
Invalid CRNs | Invalid CRN (Number) | 20061-06-111 | NA | NA | NA | NA | White List | white-list.txt | Invalid CRN (Number) | 2006-03-60024 | NA | NA | NA | NA | White List | white-list.txt | Invalid CRN (Bogus Text) | 2006-sfs-sfsfdfsdfsfsfs | NA | NA | NA | NA | White List | white-list.txt | *additional,not used for this testing* Invalid CRN (Number) available | 1995-03-76288 | NA | NA | NA | NA | White List | white-list.txt | Valid CRNs | CRN with no enrollment | 1995-01-76288 | 0 | fzthorn, (Robert Thornton) | PLB, 092, 012, Internship | Winter Quarter 1995 | White List | white-list.txt | CRN with enrollment | 2006-06-60005 | 11 (all valid knames) | acham (Christine Acham) | AAS, 198, S01, Directed Gp Study | Summer Special Session 2006 | White List | white-list.txt |
Test Cleanup - Scott will remove all sites created in this step when all test cases are completed.
|
White List Test Results
Would yield an example Summary Test Result of:
- SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site
- Data to Be Used:
Scenario | Course | Roster Count | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In | File Associated |
---|
Invalid CRNs | Invalid CRN (Number) | 20061-06-111 | NA | NA | NA | NA | White List | white-list.txt | Invalid CRN (Number) | 2006-03-60024 | NA | NA | NA | NA | White List | white-list.txt | Invalid CRN (Bogus Text) | 2006-sfs-sfsfdfsdfsfsfs | NA | NA | NA | NA | White List | white-list.txt | *additional,not used for this testing* Invalid CRN (Number) available | 1995-03-76288 | NA | NA | NA | NA | White List | white-list.txt | Valid CRNs | CRN with no enrollment | 1995-01-76288 | 0 | fzthorn, (Robert Thornton) | PLB, 092, 012, Internship | Winter Quarter 1995 | White List | white-list.txt | CRN with enrollment | 2006-06-60005 | 11 (all valid knames) | acham (Christine Acham) | AAS, 198, S01, Directed Gp Study | Summer Special Session 2006 | White List | white-list.txt |
Test Cleanup - Scott will remove all sites created in this step when all test cases are completed.
|
White List Test Results
Would yield an example Summary Test Result of:
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxx (created): Functionality Results | samerson | fzolupon | All tests pass | batching of course succeeded | P |
Site xxxxx (created): Course Management Test Results | samerson | QA Admin | All tests pass | course content correct | P |
Site xxxxx (created): Provider Id Test Results | samerson | QA Admin | All tests pass | provider id correct, roster number correct | P |
or..
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxy (not created):Test Results | samerson | QA Admin | All tests pass, not provisioned | course not created, correct | P |
Info |
---|
borderColor | #ccc |
---|
bgColor | efefef |
---|
titleBGColor | ffffcc |
---|
title | Test Results, White List |
---|
borderStyle | dashed |
---|
|
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxx (created): Functionality Results | bdonnell |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site yyyyy (created): Functionality Results | smavocet |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site wwwww (created): Functionality Results | tpamsler |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site |
|
...
or..
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxy (not created):Test Results | samerson | QA Admin | All tests pass, not provisioned | course not created, correct | P |
Info |
---|
borderColor | #ccc |
---|
bgColor | efefef |
---|
titleBGColor | ffffcc |
---|
title | Test Results, White List |
---|
borderStyle | dashed |
---|
|
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxx (created): Functionality Results | bdonnell |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site yyyyy (created): Functionality Results | smavocet |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site wwwww (created): Functionality Results | tpamsler |
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site zzzzz (created): Functionality Results | ccjon |
|
Black List
Note |
---|
|
Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created. Category of Testing = unit, integration Prerequisite Steps - Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
- Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
- Scott will que the batch jobs to run, first Black List, then White List, (5 min time elapse needed)
- Scott will advise programmers that the batches have been run
- Thomas and Scott will monitor server activity while testers are using sakai-dev.
Test Scenario - Jon G. = will look for CRN xxxxx, yyyyy, zzzzz as user ....... These CRNs will be listed in the Black List
- Brian = will look for CRN aaaaa that is listed both in the white-list.txt and black-list.txt files
- Thomas = view logs, internal sakai db
- Scott = help Jon
- Data to Be Used:
Scenario | Course | Roster Count | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In | File Associated |
---|
Test Cleanup - Scott will remove all sites created in this step when all test cases are completed.
|
Black List Test Results
Would yield an example Summary Test Result of:
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxy (not created):Test Results | samerson | QA Admin | All tests pass, not provisioned | course not created, correct | P |
Info |
---|
borderColor | #ccc |
---|
bgColor | efefef |
---|
titleBGColor | ffffcc |
---|
title | Test Results, Black List |
---|
borderStyle | dashed |
---|
|
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxx (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site yyyyy (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site zzzzz (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site aaaaa (not created):Test Results | bdonnell | x | x | site not created in UI or db | x |
*Note, pass/fail depends on db check also for this test! Jon and Brian will consult with Thomas and Scott about test case result. |
Update Existing Site with Provider Id
Note |
---|
title | Update existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**) |
---|
|
Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to test the ability to run this at any time. Category of Testing = unit, integration Prerequisite Steps Results Recording Steps summarized in: **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file hereScott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)Scott will advise programmers that the batch has been runThomas and Scott will monitor server activity while testers are using sakai-dev.zzzzz (created): Functionality Results |
|
...
samerson
...
fzolupon
...
All tests pass
...
batching of course succeeded
...
P
...
Site xxxxx (created): Course Management Test Results
...
samerson
...
QA Admin
...
All tests pass
...
course content correct
...
P
...
Site xxxxx (created): Provider Id Test Results
...
samerson
...
QA Admin
...
All tests pass
...
provider id correct, roster number correct
...
P
Black List
Note |
---|
|
Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created. Category of Testing = unit, integration Prerequisite Steps - Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
- Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
- Scott will que the batch jobs to run, first Black List, then White List, (5 min time elapse needed)
- Scott will advise programmers that the batches have been run
- Thomas and Scott will monitor server activity while testers are using sakai-dev.
Test Scenario - Jon G. = will look for CRN xxxxx, yyyyy, zzzzz as user ....... These CRNs will be listed in the Black List
- Brian = will look for CRN aaaaa that is listed both in the white-list.txt and black-list.txt files
- Thomas = view logs, internal sakai db
- Scott = help Jon
- Data to Be Used:
Scenario | Course | Roster Count | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In | File Associated |
---|
Valid CRNs | CRN with no enrollment | 2004-03-60097 | 0 | dlmiller (Douglas Miller) | ECN, 298 , 026, Group Study | Spring Quarter 2004 | White List | white-list.txt | CRN with large enrollment number | 2006-01-60001 | 84 (all valid knames) | ewlarsen (Eric Larsen) | LDA, 190 , 001, ProSeminar | Winter Quarter 2006 | White List | white-list.txt | CRN with enrollment | 2006-01-40018 | 14 (all valid knames) | calymos (Calvin Lymos) | AAS, 154, 001, University Gospel Choir | Winter Quarter 2006 | White List | white-list.txt,black-list.txt |
TODO, add more blacklisted here Test Cleanup
- Scott will remove all sites created in this step when all test cases are completed.
|
Black List Test Results
Would yield an example Summary Test Result of:
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxy (not created):Test Results | samerson | QA Admin | All tests pass, not provisioned | course not created, correct | P |
Info |
---|
borderColor | #ccc |
---|
bgColor | efefef |
---|
titleBGColor | ffffcc |
---|
title | Test Results, Black List |
---|
borderStyle | dashed |
---|
|
Test | User Role | User Acting As | Result | Expected Result | Pass/Fail |
---|
Site xxxxx (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site yyyyy (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site zzzzz (not created):Test Results | ccjon | x | x | site not created in UI or db | x | Site aaaaa (not created):Test Results | bdonnell | x | x | site not created in UI or db | x |
*Note, pass/fail depends on db check also for this test! Jon and Brian will consult with Thomas and Scott about test case result. |
Update Existing Site with Provider Id
Note |
---|
title | Update existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**) |
---|
|
Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to test the ability to run this at any time. Category of Testing = unit, integration Prerequisite Steps Results Recording Steps summarized in: **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site - Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file here
- Scott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)
- Scott will advise programmers that the batch has been run
- Thomas and Scott will monitor server activity while testers are using sakai-dev.
*Data To Be Used: Scenario | Course | SiteId | Instructor Kerb Name | Course Details (Catalog Root, Suffix, Section, Description) | Term Identifier | Test Case Used In | File Associated |
---|
InValid CRNs | No CRN (null CRN value) | null | NA | NA | NA | NA | Provider Id | provider-id.txt | Valid CRNs and Sites | CRN valid | CRN Value | xxx | NA | NA | NA | Provider Id | provider-id.txt | CRN valid | CRN Value | xxxx | NA | NA | NA | Provider Id | provider-id.txt | CRN valid | CRN Value | xxxxxx | NA | NA | NA | Provider Id | provider-id.txt |
Test Scenario - Jon G. = will look for site xxxxx's providerid and roster, logging in as admin
- Sandra = will look for site yyyyy's providerid and roster, logging in as ssbatchadmin
- Sandra = will look for site zzzzz's providerid and roster, logging in as ssbatchadmin
- Brian = will validate provider ids and rosters
- Thomas = view logs, internal sakai db
- Scott = help Sandra, Jon, and log in as himself for manual updates
Manual Process Check (update provider ids for sites manually): - After site xxxxx,yyyyy, and zzzzz's provider ids are validated,
Scott will update site xxxxx with: providerId = and.. site yyyyy with: providerId = - Thomas will check site xxxxx and site yyyyy's provider id and rosters
Test Cleanup - Scott will remove all sites created in this step when all test cases are completed.
|
...