Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • External-Banner Test Data to be Used As Representative Course Scenarios (see each test case)

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    Invalid CRNs

    Invalid CRN (Number)

    20061-06-111

    NA

    NA

    NA

    NA

    White List, Provider Id

    Invalid CRN (Number)

    2006-06-12345

    NA

    NA

    NA

    NA

    White List

    Invalid CRN (Number)

    2006-03-60024

    NA

    NA

    NA

    NA

    White List

    *additional* Invalid CRN (Number) available

    1995-03-76288

    NA

    NA

    NA

    NA

    White List

    Invalid CRN (Bogus Text)

    2006-sfs-sfsfdfsdfsfsfs

    NA

    NA

    NA

    NA

    White List

    No CRN (empty file)

    null

    NA

    NA

    NA

    NA

    White List,Black List, Provider Id

    No CRN (null CRN value)

    null

    NA

    NA

    NA

    NA

    Provider Id

    Valid CRNs

    CRN with no enrollment

    1995-01-76288

    0

    fzthorn, (Robert Thornton)

    PLB, 092, 012, Internship

    Winter Quarter 1995

    White List

    CRN with no enrollment

    2004-03-60097

    0

    dlmiller (Douglas Miller)

    ECN, 298 , 026, Group Study

    Spring Quarter 2004

    Provider Id

    CRN with enrollment

    2006-06-60005

    11 (all valid knames)

    acham (Christine Acham)

    AAS, 198, S01, Directed Gp Study

    Summer Special Session 2006

    White List,Black List,Provider Id

    CRN with enrollment

    2006-01-40018

    14 (all valid knames)

    calymos (Calvin Lymos)

    AAS, 154, 001, University Gospel Choir

    Winter Quarter 2006

    Black List

    CRN with large enrollment number

    2006-01-60001

    84 (all valid knames)

    ewlarsen (Eric Larsen)

    LDA, 190 , 001, ProSeminar

    Winter Quarter 2006

    White List

    TODO|CRN with large enrollment number|2006-03-xxxxx|NA|NA|NA|x|Provider Id|
    TODO|CRN with large enrollment number, some student kerbs don't exist|2006-03-xxxxx|NA|NA|NA|x|Provider Id|
  1. A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.

DEPENDENCIES

  • At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
  • Distributed authentication using AFS (SAK-143)
  • Sakai-dev will be used in its current state
  • We will be referencing sakai-test external for populating course data

ROLES AND RESPONSIBILITIES

  • Unit/Integration Testers (Programmers, Managers, etc)*
    which would include any of the following persons that may be available:
    • Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
    • Database mViews admin (Brian Donnelly)

COURSE SITE BATCHING DOCUMENTATION

...

  1. A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.

DEPENDENCIES

  • At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
  • Distributed authentication using AFS (SAK-143)
  • Sakai-dev will be used in its current state
  • We will be referencing sakai-test external for populating course data

ROLES AND RESPONSIBILITIES

  • Unit/Integration Testers (Programmers, Managers, etc)*
    which would include any of the following persons that may be available:
    • Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
    • Database mViews admin (Brian Donnelly)

COURSE SITE BATCHING DOCUMENTATION

JUNIT TEST RESULTS

The JUnit tests results are located on the server at:
/var/sakai/ucd-data/course-batch-unit-tests-082006

Also, the test results can be found here https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results

All unit tests passed!!

TEST PLAN PROCESS

This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.

Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).

Info

*NOTE: The order of the test cases will be as follows. White List Testing, Black List Testing, Provider Id Testing, Full Batch Run *

  1. Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White List testing scenario.
  2. In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.

TEST CASES

Info
titleKey for Test Case Scenarios

Prerequisite Steps = actions needed before performing any of the given tests
Category of Testing = the described testing is at what level?
Description = description of the test performed

White List

Note
titleWhite List Testing

Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.

Category of Testing = unit, integration

Prerequisite Steps

  1. Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
  2. Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
  3. Scott will give maintain permissions on the applicable sites, to the appropriate users
  • Brian = look up CRN xxxxx
  • Sandra = look up CRN yyyyy
  • Scott will assist users in accessing the site(s) if needed
  • Thomas = view logs, look up CRN zzzzz
  • Jon G. = look up CRN wwwww
  1. Scott will advise programmers that the batch has been run and sites have been created
  2. Thomas and Scott will monitor server activity while testers are using sakai-dev.

Test Scenario

  1. Above users will perform functionality testing for each CRN, including :
    **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site
    **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Matrix for each site
    **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/

...

JUNIT TEST RESULTS

The JUnit tests results are located on the server at:
/var/sakai/ucd-data/course-batch-unit-tests-082006

Also, the test results can be found here https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results

All unit tests passed!!

TEST PLAN PROCESS

This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.

Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).

Info

*NOTE: The order of the test cases will be as follows. White List Testing, Black List Testing, Provider Id Testing, Full Batch Run *

  1. Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White List testing scenario.
  2. In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.

TEST CASES

Info
titleKey for Test Case Scenarios

Prerequisite Steps = actions needed before performing any of the given tests
Category of Testing = the described testing is at what level?
Description = description of the test performed

White List

Note
titleWhite List Testing

Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.

Category of Testing = unit, integration

Prerequisite Steps

  1. Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
  2. Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
  3. Scott will give maintain permissions on the applicable sites, to the appropriate users
  • Brian = look up CRN xxxxx
  • Sandra = look up CRN yyyyy
  • Scott will assist users in accessing the site(s) if needed
  • Thomas = view logs, look up CRN zzzzz
  • Jon G. = look up CRN wwwww
  1. Scott will advise programmers that the batch has been run and sites have been created
  2. Thomas and Scott will monitor server activity while testers are using sakai-dev.

Test Scenario

  1. Above users will perform functionality testing for each CRN, including :
    **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site
    **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Matrix for each site
    **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site
  • Data to Be Used:

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    Invalid CRNs

    Invalid CRN (Number)

    20061-06-111

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Number)

    2006-03-60024

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Bogus Text)

    2006-sfs-sfsfdfsdfsfsfs

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    *additional,not used for this testing* Invalid CRN (Number) available

    1995-03-76288

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Valid CRNs

    CRN with no enrollment

    1995-01-76288

    0

    fzthorn, (Robert Thornton)

    PLB, 092, 012, Internship

    Winter Quarter 1995

    White List

    white-list.txt

    CRN with enrollment

    2006-06-60005

    11 (all valid knames)

    acham (Christine Acham)

    AAS, 198, S01, Directed Gp Study

    Summer Special Session 2006

    White List

    white-list.txt

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

White List Test Results

Would yield an example Summary Test Result of:

  1. SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site
  • Data to Be Used:

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    Invalid CRNs

    Invalid CRN (Number)

    20061-06-111

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Number)

    2006-03-60024

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Invalid CRN (Bogus Text)

    2006-sfs-sfsfdfsdfsfsfs

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    *additional,not used for this testing* Invalid CRN (Number) available

    1995-03-76288

    NA

    NA

    NA

    NA

    White List

    white-list.txt

    Valid CRNs

    CRN with no enrollment

    1995-01-76288

    0

    fzthorn, (Robert Thornton)

    PLB, 092, 012, Internship

    Winter Quarter 1995

    White List

    white-list.txt

    CRN with enrollment

    2006-06-60005

    11 (all valid knames)

    acham (Christine Acham)

    AAS, 198, S01, Directed Gp Study

    Summer Special Session 2006

    White List

    white-list.txt

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

White List Test Results

Would yield an example Summary Test Result of:

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxx (created): Functionality Results

samerson

fzolupon

All tests pass

batching of course succeeded

P

Site xxxxx (created): Course Management Test Results

samerson

QA Admin

All tests pass

course content correct

P

Site xxxxx (created): Provider Id Test Results

samerson

QA Admin

All tests pass

provider id correct, roster number correct

P

or..

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxy (not created):Test Results

samerson

QA Admin

All tests pass, not provisioned

course not created, correct

P

Info
borderColor#ccc
bgColorefefef
titleBGColorffffcc
titleTest Results, White List
borderStyledashed

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxx (created): Functionality Results

bdonnell

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site yyyyy (created): Functionality Results

smavocet

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site wwwww (created): Functionality Results

tpamsler

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site

...

or..

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxy (not created):Test Results

samerson

QA Admin

All tests pass, not provisioned

course not created, correct

P

Info
borderColor#ccc
bgColorefefef
titleBGColorffffcc
titleTest Results, White List
borderStyledashed

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxx (created): Functionality Results

bdonnell

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site yyyyy (created): Functionality Results

smavocet

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site wwwww (created): Functionality Results

tpamsler

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site zzzzz (created): Functionality Results

ccjon

Black List

Note
titleBlack List Testing

Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.

Category of Testing = unit, integration

Prerequisite Steps

  1. Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
  2. Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
  3. Scott will que the batch jobs to run, first Black List, then White List, (5 min time elapse needed)
  4. Scott will advise programmers that the batches have been run
  5. Thomas and Scott will monitor server activity while testers are using sakai-dev.

Test Scenario

  • Jon G. = will look for CRN xxxxx, yyyyy, zzzzz as user ....... These CRNs will be listed in the Black List
  • Brian = will look for CRN aaaaa that is listed both in the white-list.txt and black-list.txt files
  • Thomas = view logs, internal sakai db
  • Scott = help Jon
  • Data to Be Used:

    Scenario

    Course

    Roster Count

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

Test Cleanup

  1. Scott will remove all sites created in this step when all test cases are completed.

Black List Test Results
Would yield an example Summary Test Result of:

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxy (not created):Test Results

samerson

QA Admin

All tests pass, not provisioned

course not created, correct

P

Info
borderColor#ccc
bgColorefefef
titleBGColorffffcc
titleTest Results, Black List
borderStyledashed

Test

User Role

User Acting As

Result

Expected Result

Pass/Fail

Site xxxxx (not created):Test Results

ccjon

x

x

site not created in UI or db

x

Site yyyyy (not created):Test Results

ccjon

x

x

site not created in UI or db

x

Site zzzzz (not created):Test Results

ccjon

x

x

site not created in UI or db

x

Site aaaaa (not created):Test Results

bdonnell

x

x

site not created in UI or db

x

*Note, pass/fail depends on db check also for this test! Jon and Brian will consult with Thomas and Scott about test case result.

Update Existing Site with Provider Id

Note
titleUpdate existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**)

Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to
test the ability to run this at any time.

Category of Testing = unit, integration

Prerequisite Steps
Results Recording Steps summarized in:
**perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site

  • Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file here
  • Scott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)
  • Scott will advise programmers that the batch has been run
  • Thomas and Scott will monitor server activity while testers are using sakai-dev.

    zzzzz (created): Functionality Results

    ...

    samerson

    ...

    fzolupon

    ...

    All tests pass

    ...

    batching of course succeeded

    ...

    P

    ...

    Site xxxxx (created): Course Management Test Results

    ...

    samerson

    ...

    QA Admin

    ...

    All tests pass

    ...

    course content correct

    ...

    P

    ...

    Site xxxxx (created): Provider Id Test Results

    ...

    samerson

    ...

    QA Admin

    ...

    All tests pass

    ...

    provider id correct, roster number correct

    ...

    P

    ccjon

    Black List

    Note
    titleBlack List Testing

    Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.

    Category of Testing = unit, integration

    Prerequisite Steps

    1. Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
    2. Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
    3. Scott will que the batch jobs to run, first Black List, then White List, (5 min time elapse needed)
    4. Scott will advise programmers that the batches have been run
    5. Thomas and Scott will monitor server activity while testers are using sakai-dev.

    Test Scenario

    • Jon G. = will look for CRN xxxxx, yyyyy, zzzzz as user ....... These CRNs will be listed in the Black List
    • Brian = will look for CRN aaaaa that is listed both in the white-list.txt and black-list.txt files
    • Thomas = view logs, internal sakai db
    • Scott = help Jon
    • Data to Be Used:

      Scenario

      Course

      Roster Count

      Instructor Kerb Name

      Course Details (Catalog Root, Suffix, Section, Description)

      Term Identifier

      Test Case Used In

      File Associated

      Valid CRNs

      CRN with no enrollment

      2004-03-60097

      0

      dlmiller (Douglas Miller)

      ECN, 298 , 026, Group Study

      Spring Quarter 2004

      White List

      white-list.txt

      CRN with large enrollment number

      2006-01-60001

      84 (all valid knames)

      ewlarsen (Eric Larsen)

      LDA, 190 , 001, ProSeminar

      Winter Quarter 2006

      White List

      white-list.txt

      CRN with enrollment

      2006-01-40018

      14 (all valid knames)

      calymos (Calvin Lymos)

      AAS, 154, 001, University Gospel Choir

      Winter Quarter 2006

      White List

      white-list.txt,black-list.txt

      TODO, add more blacklisted here
      Test Cleanup
    1. Scott will remove all sites created in this step when all test cases are completed.

    Black List Test Results
    Would yield an example Summary Test Result of:

    Test

    User Role

    User Acting As

    Result

    Expected Result

    Pass/Fail

    Site xxxxy (not created):Test Results

    samerson

    QA Admin

    All tests pass, not provisioned

    course not created, correct

    P

    Info
    borderColor#ccc
    bgColorefefef
    titleBGColorffffcc
    titleTest Results, Black List
    borderStyledashed

    Test

    User Role

    User Acting As

    Result

    Expected Result

    Pass/Fail

    Site xxxxx (not created):Test Results

    ccjon

    x

    x

    site not created in UI or db

    x

    Site yyyyy (not created):Test Results

    ccjon

    x

    x

    site not created in UI or db

    x

    Site zzzzz (not created):Test Results

    ccjon

    x

    x

    site not created in UI or db

    x

    Site aaaaa (not created):Test Results

    bdonnell

    x

    x

    site not created in UI or db

    x

    *Note, pass/fail depends on db check also for this test! Jon and Brian will consult with Thomas and Scott about test case result.

    Update Existing Site with Provider Id

    Note
    titleUpdate existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**)

    Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to
    test the ability to run this at any time.

    Category of Testing = unit, integration

    Prerequisite Steps
    Results Recording Steps summarized in:
    **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site

    1. Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file here
    2. Scott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)
    3. Scott will advise programmers that the batch has been run
    4. Thomas and Scott will monitor server activity while testers are using sakai-dev.

    *Data To Be Used:

    Scenario

    Course

    SiteId

    Instructor Kerb Name

    Course Details (Catalog Root, Suffix, Section, Description)

    Term Identifier

    Test Case Used In

    File Associated

    InValid CRNs

    No CRN (null CRN value)

    null

    NA

    NA

    NA

    NA

    Provider Id

    provider-id.txt

    Valid CRNs and Sites

    CRN valid

    CRN Value

    xxx

    NA

    NA

    NA

    Provider Id

    provider-id.txt

    CRN valid

    CRN Value

    xxxx

    NA

    NA

    NA

    Provider Id

    provider-id.txt

    CRN valid

    CRN Value

    xxxxxx

    NA

    NA

    NA

    Provider Id

    provider-id.txt

    Test Scenario

    • Jon G. = will look for site xxxxx's providerid and roster, logging in as admin
    • Sandra = will look for site yyyyy's providerid and roster, logging in as ssbatchadmin
    • Sandra = will look for site zzzzz's providerid and roster, logging in as ssbatchadmin
    • Brian = will validate provider ids and rosters
    • Thomas = view logs, internal sakai db
    • Scott = help Sandra, Jon, and log in as himself for manual updates

    Manual Process Check (update provider ids for sites manually):

    1. After site xxxxx,yyyyy, and zzzzz's provider ids are validated,
      Scott will update site xxxxx with:
      providerId =
      and.. site yyyyy with:
      providerId =
    2. Thomas will check site xxxxx and site yyyyy's provider id and rosters

    Test Cleanup

    1. Scott will remove all sites created in this step when all test cases are completed.

    ...