Testing Criteria matrix

Testing Criteria matrix

Scoring criteria:

0 No
1 Undecided
2 Yes

Tally up scoring to determine PASS/FAIL

Deque Web Accessibility Testing Tool Evaluation Project Criteria (as of 11/16/09)

Krist RAMP

Jeremy RAMP

Lisa RAMP

Rick Worldspace

Krist Total Validator

Validation for Standards Compliance

CSS

0

0

0

0

0

HTML/XHTML

0

0

0

0

2

Section 508

1

2

2

2

2

WCAG 1

2

2

2

2

2

WCAG 2

0

0

0

2

2

Features

Installation process and user interface are accessible

 

 

-

 

1

Can spider entire sites

2 (fast)

2

2

2

0

Saves page/site settings to reanalyze without re-spidering

0

0

0

2 (buggy)

0

Can include/exclude particular directories

 

 

 

 

1

Tests static, dynamic, and password-protected sites

 

 

 

 

2

Has a remediation wizard that describes issues and has examples for solutions

2 good

2

2

2 (buggy)

1

Reports clearly indicate which pages pass or not

2

2

2

 

1

Summary reporting: Are reports useful?

2 good

2

2

2

0

Summary reporting: Are they customizable?

0

0

0

 

0

What else would you want in the reports? Multiple reporting formats (Xls, Word, PDF, etc.)

HTML only

HTML

HTML

 

HTML

What else would you want in the reports? Reporting accuracy (did not report div id for skip nav)

2

2

2

2

2

Ease of Use

Ease of installation - PC: Vista/7

0 (fail)

0

0

N/A

2

Ease of installation - PC: XP

2 easy

2

2

N/A

 

Ease of installation - Mac

-

 

N/A

 

Ease of installation - Linux

-

 

N/A

 

Intuitive interface

NO

 

0

 

2

Estimated time to proficiency (in 1-2 hours hours)

0 confusing

2 not intuitive

2

0

.5

Configurability of features

1

1

1

2 (limited)

1 (limited)

Online documentation

2

2

2

2 (limited)

2

Vendor support

Overall Value

Decreases the time/effort required to develop an accessible site

2

 

2

2

2

Decreases the time/effort required to remediate a site

2

 

2

2

2




UC Davis Additional Testing Criteria

Krist RAMP

Jeremy RAMP

Lisa RAMP

Rick Worldspace

Krist Total

Browser compatibility

-

 

2

2

validates against CA Section 11135 (enforces Section 508)

1

2

2

2 (508)

2

Remediates PDF's

0

0

0

 

0

Analyzes PDF's for accessibility

0

0

0

 

0

Interactive issue remediation
(e.g., JavaScript that will always flag even if you told it not to, color usage on lables, etc.)

2 (setup)

2

2

2 (setup)

2

Spellchecker available

0

0

0

0

2

Identifies broken links

0

1

0

0

2

- documentation valuable

2

2

2

2 (limited)

1

Software up to date

0 2004

0

0

 

2

Features/functionality readily available

0

2

2

2

(pay) 0

History tracking

0

2 - You can save

0

2

0

Support timely and responsive

1

1

1

2 (limited)

 

Plug-in support (e.g., Dreamweaver)

1 ?

2

2

 

2 (Firefox)