Most, if not all, of the issues we see with Samigo Tests & Quizzes can be assigned to one of the following categories.
- Authoring: the process of creating assessments within the Samigo tool
- Delivery: the process of administering an assessment to a group of students or other users
- Reports: collecting statistics on the assessments
- Importing/Exporting: the effort to put draw existing assessments into Sakai, and to output them in some format that other systems will be able to use
| Note | ||
|---|---|---|
| ||
Authoring
The authoring tool is already quite flexible. In addition to providing instructors with the ability to build assessments of six built-in types:
...
Importing/Exporting
...
Formative Assessment, Problem Set, Quiz, Survey, Test, and Timed Test, it allows for these types themselves to be fairly extensively customized, and for new assessment types to be created.
The instructor can control, for example, how many times an assessment can be submitted, whether late submissions will be accepted, whether to provide the user with feedback after an answer is submitted, even whether the program should only accept submissions from specific IP addresses. It's also possible to break up an assessment into discrete sections and to control how questions are presented to the user – i.e. one per page or in batches.
Unfortunately, the user interface itself is not exactly intuitive or streamlined. For example, in order to 'publish' an assessment – a necessary step if you want anyone to be able to use it – it's necessary to click on 'Settings' before you can find or select the 'Publish' button.
Also, the process of creating an assessment requires each question to be entered one-at-a-time and on its own screen. This means that there's no easy way for an instructor to simply copy and paste a list of questions that she's written up in a word processor.
Of course, there is still plenty of functionality that could be added to the authoring tool – and most of the effort out there in the community seems to be either fixing bugs with the existing authoring or adding new functionality, as you can see below.
Tasks/Feature Requests
- There's some interest in sharing assessment types so they could be modified on a departmental basis, for example SAK-5199 and SAK-3526
- Request for more flexibility to accommodate students with special needs - allow instructor to adjust time available on a student-by-student basis SAK-3427
Limitations
- Big thing is adding new question types – adding calculated/algorithmic questions
- Currently, to share question pools between users requires import/export - can't do it staying within the authoring tool itself
- Copying question pools doesn't actually copy questions, it creates links SAK-3526
- During authoring the list of question-types available is not context-sensitive SAK-1379 – this could be a feature or a limitation depending on POV.
Delivery
- Performance is the most fundamental issue here, both in terms of speed and reliability
- Most likely we will need some kind of load testing procedure in place to ensure that we don't repeat the problems of MyUCDavis, currently it doesn't seem like anybody is developing this
- The ability to administer high-stakes quizzes through Samigo is not yet supported in a realistic way (see Limitations below)
Limitations
- Timing of feedback may need to become more flexible SAK-4005
- Help pages are not yet context sensitive SAK-4103
- Need a high-stakes question type SAK-1749
- Generally, it doesn't seem like the test-taking is bulletproof enough yet, clicking too fast on buttons can cause problems, etc. SAK-6553 and SAK-6692
Reports
Minimal functionality at the moment.
- For example: School of Medicine wants to have "how long did student spend on each question"?
Importing/Exporting
Obviously, being able to bring existing quizzes and tests into SmartSite is crucial for all the current users of MyUCDavis Quizbuilder.
Currently, you can only import and export assessments to and from Samigo and MyUCDavis materials – even though it's QTI compliant, it's specific to Samigo. You can import assessments now from MyUCDavis to Samigo. You can also export/import assessments between sites within SmartSite.
...
Common Cartridge: Still unclear when this will be incorporated into Sakai Foundation core. Prob. sign. work to draw into our world. But, it could make it possible to bring in various outside data...
Outside tool from Sakai Foundation, does migration via SiteInfo, long-term effort to bring in textbook publisher data, etc.
Authoring
Tasks/Feature Requests
- Interest in sharing assessment types so they could be modified on a departmental basis, for example SAK-5199 and SAK-3526
- Request for more flexibility to accomodate students with special needs - allow instructor to adjust time available on a student-by-student basis SAK-3427
Limitations
- Big thing is adding question types – adding calculated/algorithmic questions
- Currently, to share question pools between users requires import/export - can't do it staying within the authoring tool itself
- Copying question pools doesn't actually copy questions, it creates links SAK-3526
- During authoring the list of question-types available is not context-sensitive SAK-1379 – this could be a feature or a limitation depending on POV.
Delivery
- Performance is the most fundamental issue here, both in terms of speed and reliability
- Most likely we will need some kind of load testing procedure in place to ensure that we don't repeat the problems of MyUCDavis, currently it doesn't seem like anybody is developing this
- The ability to administer high-stake quizzes through Samigo is not yet supported in a realistic way (see Limitations below)
Limitations
- Timing of feedback may need to become more flexible SAK-4005
- Help pages are not yet context sensitive SAK-4103
- Need a high-stakes question type SAK-1749
- Generally, it doesn't seem like the test-taking is bulletproof enough yet, clicking too fast on buttons can cause problems, etc. SAK-6553 and SAK-6692
Reports
Minimal functionality at the moment.
...