Testing Criteria matrix
Testing Criteria matrix
Scoring criteria:
0 No
1 Undecided
2 Yes
Tally up scoring to determine PASS/FAIL
Deque Web Accessibility Testing Tool Evaluation Project Criteria (as of 11/16/09) |
Krist RAMP |
Jeremy RAMP |
Lisa RAMP |
Rick Worldspace |
Krist Total Validator |
---|---|---|---|---|---|
Validation for Standards Compliance
|
|||||
CSS |
0 |
0 |
0 |
0 |
0 |
HTML/XHTML |
0 |
0 |
0 |
0 |
2 |
Section 508 |
1 |
2 |
2 |
2 |
2 |
WCAG 1 |
2 |
2 |
2 |
2 |
2 |
WCAG 2 |
0 |
0 |
0 |
2 |
2 |
Features
|
|||||
Installation process and user interface are accessible |
|
|
- |
|
1 |
Can spider entire sites |
2 (fast) |
2 |
2 |
2 |
0 |
Saves page/site settings to reanalyze without re-spidering |
0 |
0 |
0 |
2 (buggy) |
0 |
Can include/exclude particular directories |
|
|
|
|
1 |
Tests static, dynamic, and password-protected sites |
|
|
|
|
2 |
Has a remediation wizard that describes issues and has examples for solutions |
2 good |
2 |
2 |
2 (buggy) |
1 |
Reports clearly indicate which pages pass or not |
2 |
2 |
2 |
|
1 |
Summary reporting: Are reports useful? |
2 good |
2 |
2 |
2 |
0 |
Summary reporting: Are they customizable? |
0 |
0 |
0 |
|
0 |
What else would you want in the reports? Multiple reporting formats (Xls, Word, PDF, etc.) |
HTML only |
HTML |
HTML |
|
HTML |
What else would you want in the reports? Reporting accuracy (did not report div id for skip nav) |
2 |
2 |
2 |
2 |
2 |
Ease of Use
|
|||||
Ease of installation - PC: Vista/7 |
0 (fail) |
0 |
0 |
N/A |
2 |
Ease of installation - PC: XP |
2 easy |
2 |
2 |
N/A |
|
Ease of installation - Mac |
- |
|
N/A |
|
|
Ease of installation - Linux |
- |
|
N/A |
|
|
Intuitive interface |
NO |
|
0 |
|
2 |
Estimated time to proficiency (in 1-2 hours hours) |
0 confusing |
2 not intuitive |
2 |
0 |
.5 |
Configurability of features |
1 |
1 |
1 |
2 (limited) |
1 (limited) |
Online documentation |
2 |
2 |
2 |
2 (limited) |
2 |
Vendor support |
|||||
Overall Value
|
|||||
Decreases the time/effort required to develop an accessible site |
2 |
|
2 |
2 |
2 |
Decreases the time/effort required to remediate a site |
2 |
|
2 |
2 |
2 |
UC Davis Additional Testing Criteria |
Krist RAMP |
Jeremy RAMP |
Lisa RAMP |
Rick Worldspace |
Krist Total |
---|---|---|---|---|---|
Browser compatibility |
- |
|
2 |
2 |
|
validates against CA Section 11135 (enforces Section 508) |
1 |
2 |
2 |
2 (508) |
2 |
Remediates PDF's |
0 |
0 |
0 |
|
0 |
Analyzes PDF's for accessibility |
0 |
0 |
0 |
|
0 |
Interactive issue remediation |
2 (setup) |
2 |
2 |
2 (setup) |
2 |
Spellchecker available |
0 |
0 |
0 |
0 |
2 |
Identifies broken links |
0 |
1 |
0 |
0 |
2 |
- documentation valuable |
2 |
2 |
2 |
2 (limited) |
1 |
Software up to date |
0 2004 |
0 |
0 |
|
2 |
Features/functionality readily available |
0 |
2 |
2 |
2 |
(pay) 0 |
History tracking |
0 |
2 - You can save |
0 |
2 |
0 |
Support timely and responsive |
1 |
1 |
1 |
2 (limited) |
|
Plug-in support (e.g., Dreamweaver) |
1 ? |
2 |
2 |
|
2 (Firefox) |