Automated accessibility testing gets you on the way, but doesn’t find all the problems

The problem is that we’re not used to interacting with our products in others ways than our own. There is a high chance that we’re doing things wrong when we emulate how a certain group of users would interact with our product.

Assistive technology is complex. There are, for example, dozens of different keyboard interaction patterns to learn when you use a screen reader. This isn’t even an accessibility problem. It is incredible to see users in tests consume your products in ways you never thought of. And it is painful to see how often things that you consider “obvious” and “easy” are a barrier to a lot of people.

Demo page with some accessibility issues
Accessibility reports in the Developer Tools Issues Panel
Lighthouse accessibility audit
Accessibility insights report next to page with flagged up problems highlighted in the document
Forms in the demo page highlighted
HTML page source shown in the developer tools with the accessibility tree showing up that the role of a DIV is generic and that it has no name, regardless of its text content
Inspect tool overlay showing the information of the broken button
Issues panel of developer tools detecting 2 contrast issues in dark mode
Issues panel of developer tools detecting more contrast issues in light mode

This proves one thing: the earlier in the process of your product you take accessibility into consideration, the better the result will be. Any interaction is a possible accessibility barrier and needs to be flagged up for testing.

Experts know that. In a survey of Web Accessibility Practitioners 17.6% stated that only half the accessibility issues can be detected by automated testing.

Lighthouse asking you in a very timid way to also do some manual testin

The main thing to remember is that automated tests are a great way to get started and find the most obvious flaws. But they are not the end of the process if you really want to create accessible products.

Want to learn more? Check out the class on Skillshare where — amongst other things — I talk through this process of moving from automated to manual testing.

Screenshot of the Skillshare class on accessibility testing



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Christian Heilmann

Christian Heilmann

Maker of web things. Ponderer of new technology. Lover of simplicity. Portfolio: http://christianheilmann.com