Authoring

Authoring

What's authoring?

To get a visual sense of how assessments are authored in Samigo Tests & Quizzes, it may be helpful to take a look at the Authoring Workflow.

In addition to providing instructors with the ability to build assessments of the six built-in types: Formative Assessment, Problem Set, Quiz, Survey, Test, and Timed Test, the authoring tool also allows for these types themselves to be customized, and for new assessment types to be created. Of course, there is still plenty of functionality that could be added – and most of the effort out there in the community seems to be either fixing bugs with or adding new functionality in this space.

Currently, the instructor can control, for example, how many times an assessment can be submitted, whether late submissions will be accepted, whether to provide the user with feedback after an answer is submitted, and even whether the program should accept submissions only from specific IP addresses. It's also possible to break up an assessment into discrete sections and to control how questions are presented to the user – i.e. one per page or in batches.

Unfortunately, the user interface itself is neither exactly intuitive or streamlined. For example, in order to 'publish' an assessment – a necessary step if you want anyone to be able to use it – it's necessary to click on 'Settings' before you can find or select the 'Publish' button.

Also, the process of creating an assessment requires each question to be entered one-at-a-time and on its own screen. This means that there's no easy way for an instructor to simply copy and paste a list of questions that she's written up in a word processor.

Tasks/Feature Requests

• There's some interest in sharing assessment types so they could be modified on a departmental basis, for example SAK-5199 and SAK-3526
• Request for more flexibility to accommodate students with special needs - allow instructor to adjust time available on a student-by-student basis SAK-3427

Limitations

• Big thing is adding new question types – adding calculated/algorithmic questions
• Currently, to share question pools between users requires import/export - can't do it staying within the authoring tool itself
• Copying question pools doesn't actually copy questions, it creates links SAK-3526
• During authoring the list of question-types available is not context-sensitive SAK-1379 – this could be a feature or a limitation depending on POV.

Delivery

• Performance is the most fundamental issue here, both in terms of speed and reliability
  • Most likely we will need some kind of load testing procedure in place to ensure that we don't repeat the problems of MyUCDavis, currently it doesn't seem like anybody is developing this
• The ability to administer high-stakes quizzes through Samigo is not yet supported in a realistic way (see Limitations below)

Limitations

• Timing of feedback may need to become more flexible SAK-4005
• Help pages are not yet context sensitive SAK-4103
• Need a high-stakes question type SAK-1749
  • Some clarification is needed about the distinction between a linear assessment and a high-stakes quiz SAK-3211
  • There are various security issues that may be prerequisites to high-stakes quizzes SAK-1905
• Generally, it doesn't seem like the test-taking is bulletproof enough yet, clicking too fast on buttons can cause problems, etc. SAK-6553 and SAK-6692

Reports

Minimal functionality at the moment.

• For example: School of Medicine wants to have "how long did student spend on each question"?

Importing/Exporting

Obviously, being able to bring existing quizzes and tests into SmartSite is crucial for all the current users of MyUCDavis Quizbuilder.

Currently, you can only import and export assessments to and from Samigo and MyUCDavis materials – even though it's QTI compliant, it's specific to Samigo. You can import assessments now from MyUCDavis to Samigo. You can also export/import assessments between sites within SmartSite.

One way: Samigo -> UCDavis.

Limitations:

1. Can't export question pools yet, except with workaround – create a quiz that pulls a random selection from a pool, when export it exports all questions. Bug fix may make that no longer work.
2. Calculated questions can't be imported from MyUCDavis
3. Also, some things lost - documented in this gap analysis
   • e.g. Matching questions in other systems have distractors may need a link/description
   • MyUCDavis allows you

• There seems to be an upper limit on the number of questions you can have in a question pool SAK-61 – this may be an authoring limitation as well

Some things that are ahead in the import/export space:

On the horizon:
First thing: import question pools, which will take exported quizzes from Samigo. Key thing is that you can export a quiz and hand it to somebody else (share with other faculty), import to pool. Main thing, initially.
Second thing: export question pools and collections from MyUCDavis (new feature for MyUCDavis).

Futuristic:
In design phase: importing Respondus quizzes or question pools – the advantage of R. is that it's a nice authoring environment - desktop app, less than $100 a copy.

QTI: Interoperability standard. Predecessor to Common Cartridge. Distinct from Zach's effort to implement CC. Part of Sakai's committment to IMS standards for compatibility.

Consider expanding QTI import/export to look at IMS Content Packaging – includes capability to import/export media with a quiz, possibly importing/exporting subpools. Still gathering use cases from the community for this.

Common Cartridge: Still unclear when this will be incorporated into Sakai Foundation core. Prob. sign. work to draw into our world. But, it could make it possible to bring in various outside data...
Outside tool from Sakai Foundation, does migration via SiteInfo, long-term effort to bring in textbook publisher data, etc.