INTEGRATION TEST DATE/TIME
September 15, 2006
Start time: 14:45
End Time: 16:25
Testers:
Scott
Thomas
Brian
James
MODIFICATIONS TO THE QA TEST PLAN
There programming team used the QA Test plan to test the different provider mechanisms. In some cases these modifications were because site setup didn't happen (e.g. there were no sites created for the last parts of scenario 3. These changes just resulted in a modification of the Banner updates, and didn't have a significant impact on the test results.
The test plan was modified to account for:
- 2006-10-10024, 2006-10-10025, 2006-10-10033, and 2006-10-10037 sites not created. They were accidentally excluded from the white list batch create file.
- additional courses were created in the white list batch file to allow us to check multiple tabs for instructors (e.g. group provider functionality). Specifically, kerberos names mfharris and wjhornof were used to validate multiple sites that one instructor could see.
SCENARIO CHANGES
Test case 3d (instructor changes) were modified to account for some sites not being created, and we added one additional step to 3d(3). That scenario is described below.
3d(1) = instructor add, didn't check scenario instructor uid (610902),kerberos name palmer to CRN (2006-10-)10033 oluwa prev inst because site wasn't created. However validated mfharris scenario (James)
3d(2) = instructor remove, didn't use these crns because sites not created. Instead, asked Libby to remove kerb name mfharris from crn 10004.
3d(3) = instructor change from staff to instructor name. changed scenario to add fzturner to crn 30220, and then additionally kerberos name bossler as the instructor (replacing fzturner). THIS SCENARIO is the only significant bug seen in this testing.
PROCESS
#~14:45
In the process of waiting for refresh times from the external database, I asked James to check scenario 3a. This is a basic roster check, and we didn't need to verify any roster changes. Next, I asked Brian and James to validate Jon Gorrono's authz refresh (group provider, case 2b). They did this by adding tpa and jrenfro as students to crn's 2006-10-18933 and 18945. Next, I asked Libby to start updating Banner data, based on using a 15 minute refresh rate and previous refresh times seen in testing (e.g. ~5MINS after quarter hour seeing changes).
#~14:53
Libby started on on 3b(1) and 3b(2) cases, while others were validating the previous scenarios
Thomas validated case 3b(1) at ~ 15:16 Libby done, shortly after
James validated case 3b(2) at ~ 15:16 Libby done, shortly after
I then asked Brian to check 3c, using both 2006-10-10001 (mfharris) and 2006-10-18954 (staff). These both verified with no glaring issues in the roster.
#~15:10
We then moved on to Cases 3d(1)= James, 3d(2)=James, and 3d(3)=James.
James noticed the sites missing, but also verified the first part of the case 3d(1) ,kerb name jordi was added to 2006-10-10013. There was no site 10033 created, but since the first verification was made we skipped the 2nd verification for kerb name palmer.
#For cases 3d(2) and 3d(3) we had Libby modify alternative records, so that we can see the effect. She modified:
#3d(2): mfharris removed from crn 10004, replaced with The staff (James verified this scenario ~16:10)
#3d(3): first had kerb name fzturner replace the staff for crn 30220, then had kerb name bossler replace fzturner. (Thomas looked at this scenario)
#Brian checked WebDav using connect to server on linux, and SU functionality verified User lookup functionality for all cases related to UserDirectoryProvider (e.g. 1a,b)
FINDINGS
The only significant findings, outside of the previous JIRAs known and/or reported in batching are:
- During the batching mechanism to create these courses (in order to see real provider activity), Jon G's refresh of groups interrupted the saving of sites. Essentially, sites were not saved correctly. We had to remove Jon's patch first, create the sites, recompile the code with the patch again, then check sites during testing.
- During the case 3d(3) we noticed that whenever an instructor was replaced (e.g. staff with someone on the banner side), or even instructor replaced by another in a course, all of the instructors still appear in the membership list of the course. Additionally, a few things to note here:
1. The remove functionality for members only appears for instructors, not students. This is when you are an admin of the site
2. Even upon logging out/logging in as a different instructor for CRN 30220, we saw the same result
It appears as though Sakai doesn't know how to remove instructors from a list once they've been replaced?