Versions Compared

Old Version 12

changes.mady.by.user Scott Amerson

Saved on

compared with

New Version Current

changes.mady.by.user Scott Amerson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

TEST PLAN DATE

Testing date: Scheduled 9/14/06, early morning

TEST PLAN OVERVIEW

Scope of testing
This testing is designed to guide Quality Assurance in the functionality testing of the batch creation mechanism of course sites in SmartSite. The functionality of the batching mechanism, which can be run at any time by an application administrator, is detailed by the given test cases in this plan. Additionally, the level of involvement by QA, system administrators, and programmers will be detailed for each test case.

...

  1. There are some known issue for this testing, they are: a.
  • Related to the Home Page web content displayed for course sites. Currently, the Course Description information displays course title, which will be changed to description when the external data is mapped to a description field.

...

  • There were two bugs found during unit/integration testing that are presumed fixed.

      ...

        • The term information on the site tool didn't match the tab abbreviation for the term

      ...

        • When a user, especially an instructor, logged in (e.g. SU Tool) there is a limit of associations the instructor can have to courses. This limit is 1000, and since has been filtered down to pull back only terms that are <= year 2005.
          (These JIRAs (SAK-184, SAK-185) are listed below

      ...

        • in the summary of JIRAs covered in this test plan.)
      Info
      titleJIRAs that explain features to be tested in this plan

      TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-23 (Add ability to customize pages and tools added to pages (both default and provisional) when provisioning course sites)
      185 (1000 row limit on associations (realms) for sakai internal query)
      TESTING BUGS FIXED:http://jira.ucdavis.edu:8080/jira/browse/SAK-25 (Remove admin user from sites, during course creation script185 (term abbreviation and term description do not match in site information tool)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-119 (Course White List23 (Add ability to customize pages and tools added to pages (both default and provisional) when provisioning course sites)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-121 (Batch provisioning should make a toolset based on OOTB and other (provisional and contrib) tools25 (Remove admin user from sites, during course creation script)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-119 (Course White List)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-121 (Batch provisioning should make a toolset based on OOTB and other (provisional and contrib) tools)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-136 (Add Course title (from Banner) is the field presently mapped for course description information when provisioning sites. A course description (short and long) should be mapped from external data so that it can be referenced by certain course tools)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-140 (Course Black List)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-141 (Batching is done under a 'batch admin' user)
      http://jira.ucdavis.edu:8080/jira/browse/SAK-146 (Set Provider Id For Manually Created Courses)

      other non-batch related JIRAs covered:
      http://jira.ucdavis.edu:8080/jira/browse/SAK-143 (enable distributed authentication using AFS file system)
      -TODO http://jira.ucdavis.edu:8080/jira/browse/SAK-133 (AuthZ refresh test)-, https://jira.ucdavis.edu:8444/jira/browse/SAK-174)

      Note
      titleJIRAs that reference test scenarios, "QA Tests" described in this test plan

      TODO

      TEST ENVIRONMENT

      ...

      Server Type

      ...

      Location

      ...

      database

      ...

      https://jira.ucdavis.edu

      ...

      application

      ...

      isaac.ucdavis.edu, sakai-test

      ...

      web server

      ...

      url,

      ...

      1. Note: The application does not need to be stopped for these tests to occur
      • Additional Setup Needed
        **Batch admin account (e.g. ssbatchadmin)
        **QA admin account (e.g. QAadmin)
        **Testers have access to testable sites (via SU tool, admin workspace)
        **SU tool and Quartz tools are available in admin workspace/realm
        **All login accounts that reference kerberos names can be "provided"
      • External-Banner Test Data to be Used As Representative Course Scenarios (see each test case)
      1. A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.

      DEPENDENCIES

      • At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
      • Distributed authentication using AFS (SAK-143)
      • Sakai-test database will be a copy of existing production data, so sakai-test will have to be backed up prior to testing
      • We will be referencing sakai-test external for populating course data

      ROLES AND RESPONSIBILITIES

      • Testers (QA,students etc)*
      • Admins and SmartSite Programmers
        • Any of the following persons that may be available:
          ***Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
          ***Database/system admins (Geeta, Chris Callahan)

      *All of the above users will have access to the admin realm in Sakai, in addition to:
      ***ssbatchadmin
      ***QAadmin

      COURSE SITE BATCHING DOCUMENTATION

      ...

      :8444/jira/browse/SAK-119 = White List https://jira.ucdavis.edu:8444/jira/browse/SAK-140 = Black List https://jira.ucdavis.edu:8444/jira/browse/SAK-146 = Setting Provider Id

      Warning
      titleModifications to this Test Plan After Review with QA, 9/12/06

      No Scenario 2b covered here, no Provider id consulting Black list functionality to be tested

      TEST ENVIRONMENT

      • Hardware Setup

        Server Type

        Location

        database

        bricker.ucdavis.edu with banner-test (external db), and sakai test (internal db)

        application

        isaac.ucdavis.edu, sakai-test

        web server

        url,

        http://sakai-test.ucdavis.edu

        population of data referenced

        2006 Banner Test data (external), and clean sakai internal database with exception of necessary login accounts and manual sites created)

      1. Note: The application does not need to be stopped for these tests to occur
      • Additional Setup Needed
        **Batch admin account (e.g. ssbatchadmin)
        **QA admin account (e.g. QAadmin)
        **All login accounts that use kerberos names can be "provided" by the system (e.g. UserDirectoryProvider implemented).
        **SU tool and Quartz tools are available in admin workspace/realm
        **Pat Kava will be given access to admin realm
        **Testers have access to testable sites (via SU tool, admin workspace)

      UNIT AND INTEGRATION TEST RESULTS

      Unit Test Results

      The JUnit tests results are located on the server at:
      /var/sakai/ucd-data/course-batch-unit-tests-082006
      or..
      https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results
      = All unit tests passed!!

      Integration Test Results

      TEST PLAN PROCESS

      This Course Batching Unit/Integration Test Plan will be coordinated by Scott Amerson, and will document the results of unit testing and integration testing for the batching process before handing off to QA. All tests must pass before handing off to QA.

      Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).

      Info

      *NOTE: The order of the test cases will be as follows.
      White List Testing,
      Black List Testing,
      Provider Id Testing, and
      Full Batch Run

      1. Site Functionality Testing, Course Management Testing, and Provider Id testing are each part of the White List testing scenario.
      2. In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.

      TEST CASES

      Info
      titleKey for Test Case Scenarios

      Prerequisite Steps = actions needed before performing any of the given tests
      Category of Testing = the described testing is at what level?
      Description = description of the test performed

      White List

      ...

      titleWhite List Testing

      Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.

      Category of Testing = unit, integration

      Prerequisite Steps

      ...

      1. A few test course sites need to be created in advance so that the batch job can modify these sites with a respective provider Id. These sites are documented in the Provider Id Test Section.

      DEPENDENCIES

      • At least UserDirectoryProvider enabled to allow user accounts to be looked up automatically via kerberos name (Note:all Providers will be enabled)
      • Distributed authentication using AFS (SAK-143)
      • Sakai-test database will be a copy of existing production data, so sakai-test will have to be backed up prior to testing
      • We will be referencing sakai-test external for populating course data

      ROLES AND RESPONSIBILITIES

      • Testers (QA,students etc)*
      • Admins and SmartSite Programmers
        • Any of the following persons that may be available:
          ***Application admins (Programmers available, including Thomas, Lisa W., James, Brian, Jon G., Scott)
          ***Database/system admins (Geeta, Chris Callahan)

      *All of the above users will have access to the admin realm in Sakai, in addition to:
      ***ssbatchadmin
      ***QAadmin

      COURSE SITE BATCHING DOCUMENTATION

      • Requirements
        White List/Black List/Provider Ids setting documented requirements are in Jira

      Default Tool Set/Pages and Batching Strategy https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batching+Strategy

      Site Tab abbreviations documented at: https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Limitations+of+adding+courses+manually+to+SmartSite

      General Provider and Batching interaction documented at https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Requirements+for+Data+Provisioning+using+Providers+and+Batch+Processes

      UNIT AND INTEGRATION TEST RESULTS

      Unit Test Results

      The JUnit tests results are located on the server at:
      isaac/var/sakai/ucd-data/course-batch-unit-tests-082006
      or..
      https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Unit+Test+Results
      = All unit tests passed!!

      Integration Test Results

      https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Batch+Creation+Test+Plan%2C+Unit+and+Integration+Testing

      TEST PLAN PROCESS

      This Course QA Test Plan will be coordinated by Pat Kava, with assistance from the programming group (e.g. Scott Amerson), and will document the results of QA testing. It is assumed that all bugs have been documented by the programming group as issues before handing off to QA.

      Steps involved in the test process will be varied depending on the given test performed (as described below in the Test Cases section).

      Info

      *NOTE: The order of the test cases will be as follows.
      Case 1: White List Testing,
      Case 2 (a -and b-): Black List Testing,
      Case 3 (a and b) Provider Id Testing

      1. Site Functionality Testing (Case 4), Course Management Testing (Case 5), and Provider Id (Case 3a) testing are each part of the White List testing scenario.
      2. In order to test the Black List functionality, we will need to have a superset of CRNs that are trying to be added at the same time. To accomplish this, we will use the White List scenario with a same CRN that is included in the Black List file. See Black List Test Scenario for more information.

      TEST CASES

      Info
      titleKey for Test Case Scenarios

      Prerequisite Steps = actions needed before performing any of the given tests
      Category of Testing = the described testing is at what level?
      Description = description of the test performed

      Case 1 #White List

      Note
      titleWhite List Testing

      Description = Test the capability of the system to add "on-demand" course sites that are listed in a file. Courses are not required to have any enrollment in order to be created.

      Category of Testing = unit, integration,functionality,regression

      Prerequisite Steps

      1. Scott will add a list of CRNs (valid or not) as the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this file file to use here
      2. Scott will que the batch job to run, only for the above list (5 min elapsed time needed)
      3. Scott will give maintain permissions on the applicable sites, to the appropriate users
      • Brian QA team = look up CRN 1995-01-76288Sandra as instructor for 2006-03-93158
      • QA team = look up CRN as instructor for 2006-06-60005
      • Scott will assist users in accessing the site(s) if needed
      • Thomas QA team = view logs, look for CRN 20061-06-111, 2006-03-60024Jon G. 99999
      • QA team = look up CRN 2006-sfs-sfsfdfsdfsfsfs
      1. Scott will advise programmers testers that the batch has been run and sites have been created
      2. Thomas Scott and Scott other programmers will monitor server activity while testers are using sakai-devtest.

      Test Scenario

      1. Above users will perform functionality testing for each CRN, including :
        **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site+Functionality+Tests%2C+SmartSiteSite+BatchFunctionality+Integration+TestingTest+Results+Matrix for each site
        **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Results+Matrix for each site
        **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Results+Matrix for each site
      • Data to Be Used:
        2006-sfs-sfsfdfsdfsfsfs

        Scenario

        Course

        Roster Count

        Instructor Kerb Name

        Course Details (Catalog Root, Suffix, Section, Description)

        Term Identifier

        Test Case Used In

        File Associated

        Invalid CRNs

        Invalid CRN (Number)

        20061-06-111

        NA

        NA

        NA

        NA

        White List

        white-list.txt

        Invalid CRN (Number)

        2006-03-60024

        NA

        NA

        NA

        NA

        White List

        white-list.txt

        Invalid CRN (Bogus Text)

        Used In

        File Associated

        Invalid CRNs

        Invalid CRN (Number)

        20061-06-111

        NA

        NA

        NA

        NA

        White List

        white-list.txt

        *additional,not used for this testing* Invalid CRN (Number) available

        19952006-03-7628899999

        NA

        NA

        NA

        NA

        White List

        white-list.txt

        Valid CRNs

        CRN with no enrollment

        1995-01-76288

        0

        fzthorn, (Robert Thornton)

        PLB, 092, 012, Internship

        Winter Quarter 1995

        NA

        NA

        NA

        White List

        white-list.txt

        CRN with enrollmentInvalid CRN (Bogus Text)

        2006-06sfs-60005

        11 (all valid knames)

        acham (Christine Acham)

        AAS, 198, S01, Directed Gp Study

        Summer Special Session 2006sfsfdfsdfsfsfs

        NA

        NA

        NA

        NA

        White List

        white-list.txt

        CRN with enrollment*additional data (see attachment),may used for this testing* Invalid CRN (Number) available

        2006-03-9330767952

        10 (all valid knames)

        fzvasili shershow (S Spyridakis)HIS 111C 001 Ancient HistoryScott Shershow)

        CRI 200B 002 Problems in Crit Theory

        Spring Quarter 2006

        White List

        white-list.txt,black-list.txt

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      White List Test Results

      Would yield an example Summary Test Result of:

      Test

      User Role

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxx (created): Functionality Results

      samerson

      fzolupon

      All tests pass

      batching of course succeeded

      P

      Site xxxxx (created): Course Management Test Results

      samerson

      QA Admin

      All tests pass

      course content correct

      P

      Site xxxxx (created): Provider Id Test Results

      samerson

      QA Admin

      All tests pass

      provider id correct, roster number correct

      P

      or..

      Test

      User

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxy (not created):Test Results

      samerson

      QA Admin

      All tests pass, not provisioned

      course not created, correct

      P

      results: https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Batch+Course+Creation+Integration+Test+Results%2C+Sept+5%2C+2006

      Black List

      Note
      titleBlack List Testing

      Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.

      Category of Testing = unit, integration

      Prerequisite Steps

      1. Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
      2. Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
      3. Scott will que the batch job to run (White List), (5 min time elapse needed)
      4. Scott will advise programmers that the batch has been run
      5. Thomas and Scott will monitor server activity while testers are using sakai-dev.

      Prerequisite Steps needed for Scenario #2 only (in addition to Scenario #1 steps)

      1. Scott will update existing hand created sites manually (TBD site ids) with an associated CRN as the provider id

      Test Scenarios
      *Scenario #1 = Black list consulted when white list batch run is executed*

      • James. = will look for CRN 2004-03-60097 as user admin. This CRN will be listed in the White List files only
      • Brian = will look for CRN 2006-01-60001 that is listed both in the white-list.txt and black-list.txt files
      • Sandra = will look for both CRN 2006-01-40018 that are listed both in the white-list.txt and black-list.txt files
      • Thomas = view logs, internal sakai db, will look for both CRN 2006-03-82882, 2006-03-83634 that are listed in the black-list.txt files only
      • Scott = help as needed
      • Data to Be Used:

        Scenario

        Course

        Roster Count

        Instructor Kerb Name

        Course Details (Catalog Root, Suffix, Section, Description)

        Term Identifier

        Test Case Used In

        File Associated

        Valid CRNs

        CRN with no enrollment

        2004-03-60097

        0

        dlmiller (Douglas Miller)

        ECN, 298 , 026, Group Study

        Spring Quarter 2004

        Black List

        white-list.txt

        CRN with large enrollment number

        2006-01-60001

        84 (all valid knames)

        ewlarsen (Eric Larsen)

        LDA, 190 , 001, ProSeminar

        Winter Quarter 2006

        Black List

        white-list.txt,black-list.txt

        CRN with enrollment

        2006-01-40018

        14 (all valid knames)

        calymos (Calvin Lymos)

        AAS, 154, 001, University Gospel Choir

        Winter Quarter 2006

        Black List

        white-list.txt,black-list.txt

        CRN with enrollment

        2006-03-82882

        10

        ltcmwc (Mark Connelly)

        MSC 141 002 Army Mngmt Systems

        Spring Quarter 2006

        Black List

        black-list.txt

        CRN with enrollment

        2006-03-83634

        10 (all valid knames)

        eschang (Ernest Chang)

        NPB 141P 001 Res In Physio Adapt

        Spring Quarter 2006

        Black List

        black-list.txt

      *Scenario #2 = Black list consulted when existing site already has provider id that matches black list entry*

      • Jon G. = will look for CRN 2006-03-69857 as user ccjon. This CRN will be listed in the Black List, but have no existing site that has the provider ids matching the CRNs
      • Brian = will look for CRN 2006-03-65306,2006-06-60435 that is listed both in the provider id reference for sites ^^^^^^^^, and (((((((( and black-list.txt file
      • Thomas = view logs, internal sakai db
      • Scott = help Jon

        txt

        Valid CRNs

        CRN with no enrollment

        2006-03-93158

        10 (all valid knames)

        klradke (Kathryn Radke)

        SAS 197T 001 Tutoring Sci & Society

        Spring Quarter 2006

        White List

        white-list.txt

        CRN with enrollment

        2006-06-60005

        11 (all valid knames)

        acham (Christine Acham)

        AAS, 198, S01, Directed Gp Study

        Summer Special Session 2006

        White List

        white-list.txt

        CRN with enrollment

        2006-03-93307

        large enrollment (all valid knames)

        fzvasili (S Spyridakis)

        HIS 111C 001 Ancient History

        Spring Quarter 2006

        White List

        white-list.txt,black-list.txt

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-119

      White List Test Results

      Would yield an example Summary Test Result of:

      Test

      User Role

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxx (created): Functionality Results

      samerson

      fzolupon

      All tests pass

      batching of course succeeded

      P

      Site xxxxx (created): Course Management Test Results

      samerson

      fzolupon

      All tests pass

      course content correct

      P

      Site xxxxx (created): Provider Id Test Results

      samerson

      fzolupon

      All tests pass

      provider id correct, roster number correct

      P

      or..

      Test

      User

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxy (not created):Test Results

      samerson

      fzolupon

      All tests pass, not provisioned

      course not created, correct

      P

      Case 2 #Black List

      Note
      titleBlack List Testing

      Description = Test the capability of the system to not add course sites that are listed in a file. Courses in this list will never be batch created.

      Category of Testing = unit, integration, functionality,regression

      Prerequisite Steps

      1. Scott will add a list of CRNs (valid or not) to the white-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. This list will be different than that used for the White List Test Scenario. Use this white list
      2. Scott will add a list of CRNs (valid or not) to the black-list.txt file in /var/sakai/ucd-data folder on the server, in the form of TERMYR-TERMCD-CRN for each entry. Use this black list file file to use here
      3. Scott will que the batch job to run (White List), (5 min time elapse needed)
      4. Scott will advise QA team that the batch has been run
      5. Scott will monitor server activity while testers are using sakai-test.

      -Prerequisite Steps needed for Scenario #2b only (in addition to Scenario #2a steps)-
      -# Scott will update existing hand created sites manually (TBD site ids) with an associated CRN as the provider id-

      Test Scenarios
      *Scenario #2a = Black list consulted when white list batch run is executed*

      • QA team = will look for CRN 2006-06-60245 as user admin. This CRN will be listed in the White List files only
      • QA team = will look for CRN 2006-01-60001 that is listed both in the white-list.txt and black-list.txt files
      • QA team = will look for both CRN 2006-01-40018 that are listed both in the white-list.txt and black-list.txt files
      • Scott = view logs, internal sakai db, will look for both CRN 2006-03-82882, 2006-03-83634 that are listed in the black-list.txt files only
      • Scott = help as needed
      • Data to Be Used:

        Scenario

        Course

        Site id

        Roster Count

        Instructor Kerb Name

        Course Details (Catalog Root, Suffix, Section, Description)

        Term Identifier

        Test Case Used In

        File Associated

        Valid CRNs

        CRN with no enrollment

        2006-0306-6985760245

        10 0

        jrlund fzduts (Jay LundDennis Dutschke)

        ECI 299 019 Research

        Spring Quarter ITA 198 S01 Dir Group Study

        Summer Special Session 2006

        Black List

        blackwhite-list.txt

        CRN with large enrollment (large)number

        2006-0301-6530660001

        177 84 (all valid knames)

        fzsegel ewlarsen (Leigh Segel)

        BIS 103 002 Bioenergetics/Metabolism

        Spring Eric Larsen)

        LDA, 190 , 001, ProSeminar

        Winter Quarter 2006

        Black List

        white-list.txt,black-list.txt

        CRN with no enrollment

        2006-0601-60435

        0

        fzblanch (Marc Blanchard)

        COM 152 S01 Literature Of Americas

        Summer Special Session 40018

        14 (all valid knames)

        calymos (Calvin Lymos)

        AAS, 154, 001, University Gospel Choir

        Winter Quarter 2006

        Black List

        white-list.txt,black-list.txt

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      Black List Test Results
      Would yield an example Summary Test Result of:

      Test

      User

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxy (not created):Test Results

      samerson

      QA Admin

      All tests pass, not provisioned

      course not created, correct

      P

      results: https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Batch+Course+Creation+Integration+Test+Results%2C+Sept+5%2C+2006

      ...

      • CRN with enrollment

        2006-03-82882

        10

        ltcmwc (Mark Connelly)

        MSC 141 002 Army Mngmt Systems

        Spring Quarter 2006

        Black List

        black-list.txt

        CRN with enrollment

        2006-03-83634

        10 (all valid knames)

        eschang (Ernest Chang)

        NPB 141P 001 Res In Physio Adapt

        Spring Quarter 2006

        Black List

        black-list.txt

      Case 3 #Update Existing Site with Provider Id

      Note
      titleUpdate existing sites with appropriate provider id, enabling dynamic rosters (**When CourseManagementProvider enabled**)

      Description = Test the capability of the system to update existing course sites with associated provider ids. This functionality can be done either in the UI or batch, however we want to
      test the ability to run this at any time.

      Category of Testing = unit, integration, functionality,regression

      Prerequisite Steps
      Results Recording Steps summarized in:
      **perform provider id test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Provider+Id+Update+of+Existing+Sites%2C+Integration+Test+Matrix for each site

      1. Scott will add a list of sites and provider ids (valid or not) to the provider-id.txt file in /var/sakai/ucd-data folder on the server, in the form of site id, TERMYR-TERMCD-CRN for each entry. Use this file use file here
      2. Scott will que the batch jobs to run, just the provider id que batch run, (5 min time elapse needed)
      3. Scott will advise programmers testers that the batch has been run
      4. Thomas and Scott will monitor server activity while testers are using sakai-devtest.

      *Data To Be Used:

      Scenario

      Course

      SiteId or Title

      Instructor Kerb Name

      Course Details (Catalog Root, Suffix, Section, Description)

      Term Identifier

      Test Case Used In

      File Associated

      InValid CRNs

      No CRN (null CRN value)

      null

      NA

      NA

      NA

      NA

      Provider Id

      provider-id.txt

      Valid CRNs and Sites

      CRN valid

      2006-03-93327

      Scott's First Test Course To Be ProvidedUpdated By Batch

      stenzel (John Stenzel)

      MST 020C A02 Late Med & Early Modern (roster=10)

      Spring Quarter 2006

      Provider Id

      provider-id.txt

      CRN valid

      2006-03-92472

      A Second Test Course To Be updated By Batch

      paully (Paul Manriquez)

      WLD 910 001 Wkload - Math Pre 16A (roster=10)

      Spring Quarter 2006

      Provider Id

      provider-id.txt

      CRN valid

      2006-03-83569

      Third Course Site Example

      goldberg (Jack Goldberg)

      NPB 101 001 Systemic Physiology

      Spring Quarter 2006

      Provider Id

      provider-id.txt

      Test Scenario

      • Jon G. = will look for site Scott's Test Course To Be Provided's providerid and roster, logging in as admin
      • Sandra = will look for site A Second Course To Be updated By Batch's providerid and roster, logging in as ssbatchadmin
      • James= will look for site Third Course Site Example's providerid and roster, logging in as admin (after Jon G. done)
      • Brian = will validate provider ids and rosters
      • Thomas = view logs, internal sakai db
      • Scott = help Sandra, Jon, and log in as himself for manual updates

      Manual Process Check (update provider ids for sites manually):

      1. After site Scott's Test Course To Be Provided,A Second Course To Be updated By Batch, and Third Course Site Example's provider ids are validated,
        Scott will update site Second Course Site Example with:
        providerId = 2006-03-76517
        and.. site Third Course Site Example with:
        providerId = 2006-03-75796
      2. Thomas will check site Second Course Site Example and site Third Course Site Example's provider id and rosters

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      Provider Id Update Test Results
      Would yield an example Summary Test Result of (both batch and manual updates):

      Test

      User

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxx: Provider Id Update Results

      samerson

      admin

      provider id is ....., roster is same as CRN for provider id

      provider id updated and seen in site tool under "provider id", roster reflects provider id roster

      P

      results: https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Batch+Course+Creation+Integration+Test+Results%2C+Sept+5%2C+2006

      Site Functionality

      Note
      titleSite functionality

      Description = Test the capability of the system to add sites with the correct pages, tools, tabs, and look and feel.

      Category of Testing = unit, integration, functionality

      Prerequisite Steps
      Results Recording Steps summarized in:
      **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Course+Site+Functionality+Tests%2C+SmartSite+Batch+Integration+Testing+Matrix for each site
      looking for expected results
      Test Scenario

      Panel
      borderColor#ccc
      bgColorF7D6C1
      titleBGColor#efefef
      titleIndividual Functionality Tests
      borderStyledashed
      • -Look and Feel- *
      • Pages *
      • Tools on each page *
      • Click-through on tools *
      • Order *
      • Tabs *

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/Batch+Course+Creation+Integration+Test+Results%2C+Sept+5%2C+2006

      Course Management Data Presented in Admin Tools

      Note
      titleCourse Management data present

      Description = Test the course management content displayed in admin tools.

      Category of Testing = unit, integration, functionality

      Prerequisite Steps
      Results Recording Steps summarized in:
      **perform course management test

      CRN valid

      2006-03-83569

      Third Test Course To Be Updated By Batch

      goldberg (Jack Goldberg)

      NPB 101 001 Systemic Physiology (roster=553)

      Spring Quarter 2006

      Provider Id

      provider-id.txt

      Scenario 3a (Batch updates existing sites provider id)
      Test Scenario

      • QA team = will look for site First Test Course To Be Updated By Batch's providerid and roster, logging in as admin
      • QA team = will look for site Second Test Course To Be updated By Batch's providerid and roster, logging in as admin
      • QA team= will look for site Third Test Course To Be Updated By Batch's providerid and roster, logging in as admin
      • QA team = will validate provider ids and rosters
      • QA team = perform manual updates of provider ids
      • Scott = help as needed

      Scenario 3b (Manual Process Check (update provider ids for sites manually)
      Test Scenario
      *Scott will have create 3 sites:
      First Test Course To Have Manual Provider Id
      Second Test Course To Have Manual Provider Id
      Third Test Course To Have Manual Provider Id
      -# After site First Test Course To Have Manual Provider Id,Second Test Course To Have Manual Provider Id, and Third Test Course To Have Manual Provider Id's provider ids are validated,-

      QA will update site First Test Course To Have Manual Provider Id with:
      providerId = 2006-03-73656 (10 enrolled)
      and site Second Test Course To Have Manual Provider Id with:
      providerId = 2006-03-F0001 made up of 2006-03-62179,2006-03-62178,2006-03-62180 (51 enrolled, multi-crn)
      and site Third Test Course To Have Manual Provider Id with
      providerId = 2006-03-80102 (178 enrolled)

      1. Scott and Pat Kava will check all sites rosters based on their associated provider id

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: Recorded in Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-146, and is a part of White List Test https://jira.ucdavis.edu:8444/jira/browse/SAK-119

      Provider Id Update Test Results
      Would yield an example Summary Test Result of (both batch and manual updates):

      Test

      User

      User Acting As

      Result

      Expected Result

      Pass/Fail

      Site xxxxx: Provider Id Update Results

      samerson

      admin

      provider id is ....., roster is same as CRN for provider id

      provider id updated and seen in site tool under "provider id", roster reflects provider id roster

      P

      results: Recorded in Jira https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/

      ...

      Batch+Course+

      ...

      Creation+Integration+

      ...

      Test+

      ...

      Results%2C+

      ...

      Sept+

      ...

      Panel
      borderColor#ccc
      bgColorwhite
      titleBGColorefefef
      titleTest Scenario
      borderStyledashed
      • Site navigation *
      • WorkSite Setup navigation *
      • Site Info tool *

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: 5%2C+2006

      Case 4 #Site Functionality

      Note
      titleSite functionality

      Description = Test the capability of the system to add sites with the correct pages, tools, tabs, and look and feel.

      Category of Testing = unit, integration, functionality, regression

      Prerequisite Steps
      Results Recording Steps summarized in:
      **perform functionality test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/

      ...

      SmartSite+Course+Site+Tests%2C+Site+Functionality+Integration+Test+

      ...

      Full Batch Run on Term

      Note
      titleFull Batch Run

      Description = Test the batch method for a whole term

      Category of Testing = unit, integration, performance

      Prerequisite Steps

      • After all tests have been run, Scott will recompile the Batch code to run for 2004 Winter
      • Scott will add the black list file that contains one CRN from Winter 2004 into the /var/sakai/ucd-data directory.
      • This test will run in the evening of test day
      • Scott and Thomas will analyze the Sakai internal db to check for errors in creation
      • Data to Use:

        Scenario

        Course

        Roster Count

        Instructor Kerb Name

        Course Details (Catalog Root, Suffix, Section, Description)

        Term Identifier

        Test Case Used In

        File Associated

        Valid Term

        Term with large number CRNs

        2004-01

        NA

        NA

        32573 sites

        Winter Quarter 2004

        Full Batch Run

        NA

        Valid CRN

        2004-01-40200,2004-01-42003

        NA

        NA

        NA

        NA

        White List

        black-list.txt

      Test Cleanup

      1. Scott will ask Geeta to drop the devel schema when this test is completed.

      ...

      Results+Matrix for each site
      looking for expected results
      Test Scenario

      Panel
      borderColor#ccc
      bgColorF7D6C1
      titleBGColor#efefef
      borderStyledashed
      titleIndividual Functionality Tests

      Look and Feel
      Pages
      Tools on each page
      Click-through on tools
      Order
      Tabs

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: Recorded as part of White List Scenario, and Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-119

      Jiras covered in this test case:
      https://jira.ucdavis.edu:8444/jira/browse/SAK-121 = Tools and Pages
      https://jira.ucdavis.edu:8444/jira/browse/SAK-23 = Tools configurable

      Case 5 #Course Management Data Presented in Admin Tools

      Note
      titleCourse Management data present

      Description = Test the course management content displayed in admin tools.

      Category of Testing = unit, integration, functionality, regression

      Prerequisite Steps
      Results Recording Steps summarized in:
      **perform course management test https://confluence.ucdavis.edu:8443/confluence/display/UCDSAKAI/SmartSite+Course+Site%2C+Course+Management+Integration+Test+Results+Matrix for each site
      looking for expected results

      Panel
      borderColor#ccc
      bgColorwhite
      titleBGColorefefef
      borderStyledashed
      titleTest Scenario

      Site Info tool
      WorkSite Setup navigation

      Test Cleanup

      1. Scott will remove all sites created in this step when all test cases are completed.

      results: Recorded as part of White List Scenario, and Jira https://jira.ucdavis.edu:8444/jira/browse/SAK-119

      Jiras covered in this test case:
      https://jira.ucdavis.edu:8444/jira/browse/SAK-136 = Course Management Data (Description) details

      Recording Results

      The test results will be recorded in the above summary tables by integration testers, and a summary of this data will be ready for QA and the Sakai Development team to reviewJira by QA testers. They can be of table form or text form.

      Feedback Mechanism

      The above results will be analyzed by the QA team, SmartSite programming team, and SmartSite management to determine the steps needed and whether or not to hand off to QAquality of code. Results of this test should be discussed as near to the Monday meetings as possibleand/or Code Migration meetings when available.