Accessibility Testing

Accessibility (often shortened to a11y) means people can use your product regardless of disability: people who are blind or have low vision, who are deaf, who cannot use a mouse, who have cognitive or motor impairments, and many more, including temporary and situational ones. It is also a legal requirement in many markets. For testers, accessibility is a quality dimension with clear, testable criteria, which makes it an excellent place to add value.

WCAG 2.2 in brief

The Web Content Accessibility Guidelines (WCAG), published by the W3C, are the reference standard. WCAG 2.2 became a W3C Recommendation in October 2023. It is organised around four principles, often remembered as POUR:

• Perceivable: information can be perceived (text alternatives, captions, sufficient contrast).
• Operable: the interface can be operated (keyboard access, enough time, no seizure-inducing flashes).
• Understandable: content and behaviour are understandable (clear labels, predictable navigation, helpful errors).
• Robust: content works with assistive technologies (valid names, roles and states).

Each principle has guidelines, and each guideline has success criteria at levels A, AA and AAA. Most organisations and regulations target Level AA, which includes all A and AA criteria.

What changed in 2.2

WCAG 2.2 builds on 2.1 and added criteria including:

• 2.4.11 Focus Not Obscured (Minimum) (AA): a focused element is not entirely hidden, for example by a sticky header or cookie banner.
• 2.5.7 Dragging Movements (AA): anything done by dragging also has a single-pointer alternative.
• 2.5.8 Target Size (Minimum) (AA): pointer targets are at least 24 by 24 CSS pixels, with some exceptions such as adequate spacing.
• 3.2.6 Consistent Help (A): help mechanisms appear in a consistent place across pages.
• 3.3.7 Redundant Entry (A): users are not asked to re-enter information they already provided in the same process.
• 3.3.8 Accessible Authentication (Minimum) (AA): login does not rely on a cognitive function test, such as remembering or transcribing a code, without an alternative (allowing paste and password managers helps meet this).

It also removed 4.1.1 Parsing as obsolete. Several of these new criteria, such as dragging alternatives and redundant entry, are hard or impossible to check with automated tools.

Automated testing with axe-core

axe-core is an open-source accessibility rules engine from Deque. The @axe-core/playwright package runs it inside a Playwright page.

npm install --save-dev @axe-core/playwright

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('checkout page has no detectable WCAG A/AA violations', async ({ page }) => {
  await page.goto('/checkout');

  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
    .analyze();

  expect(results.violations).toEqual([]);
});

withTags limits the scan to rules mapped to those WCAG levels. Each entry in results.violations includes the rule id, an impact level, a help URL and the offending nodes with their selectors, so a failure message points directly at the problem.

Scanning states, not just pages

A page load scan only sees the initial state. Many issues live in dialogs, menus, error states and dynamically loaded content. Drive the UI into each state, then scan:

test('login error state is accessible', async ({ page }) => {
  await page.goto('/login');
  await page.getByRole('button', { name: 'Sign in' }).click(); // submit empty form

  await expect(page.getByRole('alert')).toBeVisible();

  const results = await new AxeBuilder({ page })
    .include('form')                 // scope the scan
    .analyze();

  expect(results.violations).toEqual([]);
});

.include() and .exclude() take CSS selectors. Use .exclude() for third-party widgets you cannot fix, and .disableRules(['rule-id']) only with a ticket that tracks the known issue. Silently disabling rules turns your gate into decoration.

Introducing a gate on an existing product

If a legacy app has hundreds of violations, a zero-violation assertion fails forever. Options:

• Start with the critical journeys (sign-up, login, checkout) and get those clean first.
• Gate on critical and serious impact violations first, then tighten.
• Fail on new violations by comparing against a reviewed baseline, while a backlog addresses the old ones.

Keyboard testing

Many users navigate with a keyboard alone, and keyboard support is a prerequisite for most assistive technologies. A manual keyboard pass is quick and finds serious bugs:

1. Put the mouse away. Start at the top of the page and press Tab repeatedly.
2. Can you reach every interactive element? Custom div buttons are often skipped.
3. Is focus always visible? You should always know where you are.
4. Is the order logical? It should follow the visual reading order.
5. Can you operate everything? Enter activates links and buttons, Space activates buttons and checkboxes, arrow keys move within menus, tabs and radio groups.
6. Do dialogs manage focus? Opening moves focus inside, Escape closes, focus does not escape behind the dialog, and closing returns focus to the trigger.
7. Any keyboard traps? You must always be able to move focus away.
8. Is focus hidden under sticky headers or banners (WCAG 2.2, 2.4.11)?

Parts of this can be automated as regression tests:

test('dialog traps and restores focus', async ({ page }) => {
  await page.goto('/settings');
  const trigger = page.getByRole('button', { name: 'Delete account' });
  await trigger.focus();
  await page.keyboard.press('Enter');

  const dialog = page.getByRole('dialog');
  await expect(dialog).toBeVisible();
  await expect(dialog.getByRole('button', { name: 'Cancel' })).toBeFocused();

  await page.keyboard.press('Escape');
  await expect(dialog).toBeHidden();
  await expect(trigger).toBeFocused();
});

Using role-based locators such as getByRole in your regular tests has a side benefit: if an element has no accessible role or name, the locator will not find it.

Screen reader testing

Screen readers announce the accessible name, role and state of elements. Common pairings are NVDA or JAWS with Chrome or Firefox on Windows, VoiceOver with Safari on macOS and iOS, and TalkBack with Chrome on Android. Learn one well rather than all superficially.

What to check:

• Images: meaningful images have useful alternative text; decorative images are ignored.
• Form fields: each announces its label, required state and any error message.
• Headings and landmarks: navigating by heading gives a sensible outline of the page.
• Dynamic updates: toasts, validation errors and cart counts are announced (live regions or focus management).
• Custom widgets: a custom dropdown announces as a combobox or listbox with its expanded state, not as “clickable”.

Treat screen reader results as findings that need judgment, not pixel-perfect scripts; announcements vary between screen reader and browser combinations.

What automation cannot catch

Automated tools cannot reliably tell you:

• Whether alternative text is accurate and meaningful (alt="image" passes a presence check).
• Whether the focus order makes sense or a keyboard user can complete the task.
• Whether labels and instructions are clear, or error messages explain how to fix the problem.
• Whether captions and transcripts are accurate.
• Whether content is understandable for people with cognitive disabilities.
• Whether dragging has an alternative or users are asked to re-enter data.
• How the experience feels with zoom at 200 to 400% or reflow on small screens.

The most reliable assurance comes from combining automated scans, manual expert testing and, where possible, usability sessions with disabled users.

Practice exercise

Your team ships a new “Filter products” side panel. It opens from a button, contains checkboxes and a price range slider that works by dragging, and has an “Apply” button. Write an accessibility test plan covering automated and manual checks, with at least one Playwright test sketch.

test('filter panel is keyboard operable', async ({ page }) => {
  await page.goto('/products');
  const open = page.getByRole('button', { name: 'Filter products' });
  await open.click();
  const panel = page.getByRole('dialog', { name: 'Filters' });
  await expect(panel).toBeVisible();

  const results = await new AxeBuilder({ page }).include('[role="dialog"]').analyze();
  expect(results.violations).toEqual([]);

  await page.keyboard.press('Escape');
  await expect(open).toBeFocused();
});

Key takeaways

• WCAG 2.2 is organised by POUR; most teams target Level AA.
• WCAG 2.2 added criteria on focus visibility, dragging, target size, consistent help, redundant entry and accessible authentication.
• @axe-core/playwright with new AxeBuilder({ page }).analyze() automates detectable issues; scan every important UI state.
• Keyboard and screen reader testing find issues that automated rules cannot.
• A passing automated scan means no detectable violations, not an accessible product.