Accessibility with Scenery

Scenery's Accessibility Features

Scenery has a variety of accessibility features available to use. This document describes how to use the Parallel DOM to support screen reader accessibility. Scenery supports other accessibility-related features too:

Parallel DOM - a parallel DOM tree that renders the Scenery scene graph into HTML for ingestion by a screen reader.
Pan and Zoom - zooming and panning within a scenery Display, with a variety of input modalities for controling it.
Voicing - Speech made directly by the application using SpeechSynthesis.

Background Information

Prior to reading through the following documentation, please familiarize yourself with this background information about accessible HTML, the accessibility tree, accessible names, and ARIA. The rest of this document will assume you have knowledge of these concepts.

Accessible HTML – This is an introduction to accessible, semantic HTML. This is important because much of Scenery's accessibility support is only as good as the accessible HTML you tell it to create.
The Accessibility Tree – It is good to understand how the browser interprets HTML into a rich data structure called the "Accessibility Tree." Most important is the notion of an "accessible name" and "role" that each element needs, at the very least, to be accessible.
Accessible Name – Identifying the multiple ways to provide an accessible name to an element is vital to web accessibility. Having multiple strategies allows for flexibility in code that you write.
ARIA – "Accessible Rich Internet Applications" is a way to supplement semantic HTML in order to provide further accessibility to elements and interactions. Much of the API for the Parallel DOM uses ARIA features. Remember that the first rule of ARIA is to not use ARIA!

The Parallel DOM

Scenery uses HTML5 technologies (svg, canvas, webgl) to draw the display. These have very little semantic data as to what is inside the rendered graphic. The PDOM (Parallel DOM (document object model)) pulls semantic data from the scene graph and adds it to a separate HTML structure that is available for assistive technologies (AT). When we say "PDOM", think "the HTML manifestation of the graphical Node.js content in the display."

The PDOM not visible on screen, but provides an interface to AT so that they have a representation of the display. The PDOM is dynamic and its DOM tree will update with changes to the scene. Any node that has accessible content specified will be represented in the PDOM. HTML is used so that scenery can rely on semantic HTML and accessibility conventions of the web. This way, scenery can push some of the accessibility work load to the browser and AT for providing keyboard navigation and auditory descriptions.

Any node in scenery can have accessible content. The accessible content will represent the node in the PDOM. The PDOM only contains Nodes that specify content.

Scenery's Accessibility API (powered by the PDOM)

Everything described in this document is accessed by creating a Display with the accessibility:true option passed to it. Most of Scenery's accessibility features are defined and implemented in ParallelDOM.js. ParallelDOM.js is a trait that is mixed in to Node.js. It adds getters and setters for properties related to the PDOM, so all we have to do is pass in PDOM specific options like normal into the super() or mutate() function calls.

The following explains how to use the accessibility functionality of Scenery. For more information and up-to-date API documentation, see the source code. On the side bar, options are categorized by where they are introduced and explained. In this file there is little "traditional" documentation, rather example-based explanation. The source code is the best place for specifics, documentation and implementation.

A Basic Example

The primary way that developers will implement accessibility is through options passed through to Node.js. First off, each node that wants content in the PDOM will need an HTML element in the PDOM to represent it. To do this, use the tagName option:

Above is a simple scenery Rectangle, that is represented as a paragraph tag in the PDOM. In addition to tagName, use innerContent to add text content inside the <p> element.

Multiple DOM Elements per Node

The basic example shows one DOM element representing a node in the PDOM, but ParallelDOM.js supports a richer HTML structure. Each node can have multiple DOM elements. To be represented in the PDOM, the node must have a primary element specified with tagName. It can optionally also have a label element specified with labelTagName, a descriptions element specified with descriptionTagName and a structural container element specified with containerTagName.

Terminology

Terminology is key in understanding the specifics of creating the PDOM. From here on, when speaking about "siblings," we are speaking about the relationship between HTML elements in the PDOM. These Elements are not "siblings to the node," but instead only siblings to each other, with an HTML Element parent called the "containerParent".

Summary: Each node has an PDOMPeer (or “peer”) that manages the HTMLElements (aka “elements”) that are related to that node in the PDOM. A node has one or more associated elements, one of which is the “primary element”, whose tag is specified by option tagName. There are two other optional “supplementary elements”, whose tags are specified via options labelTagName and descriptionTagName. If more than the primary element is specified, they are all referred to as “sibling elements” (including the "primary sibling") and can be optionally grouped together under a “container element”. The container element is given a tag name via containerTagName.

Example

Here is an example of a node that uses all of its elements to provide the fullest semantic picture of the sim component to the PDOM.

In this example, the rectangle's primary sibling is a button with an Accessible Name of "Grab Magnet". It has a label sibling with an h3 tag with inner content "Grab Magnet", and a description sibling with a tagName of "p" with the specified sentence.

A few notes here:

All supplemental HTML Elements (container, description and label) have default tags in the PDOM API. Specifying a "*Content" option without the "*TagName" option will produce an element of the default. See PDOMUtils for the default tag names.
By default the description and label tags come before the primary sibling, you can change this with appendLabel and appendDescription options.
The Text node "North South Magnet" has no accessible content and so it does not appear anywhere in the PDOM.

The Structure of the PDOM

By default, the PDOM hierarchy will match the hierarchy of the scene graph. This is an important feature to consider. If a parent node and child node both have accessible content, then, in the PDOM, the accessible HTML of the child node will be added as a child of the parent's primary sibling. In scenery code, this is managed by PDOMPeer, a type that stores and controls all HTML Elements for a given node.

Leveraging the Scene Graph

Consider the following example where we have a box filled with circles and the desired PDOM representation is an unordered list filled with list items.

In this example, scenery automatically structured the PDOM such that the list items are children of the unordered list to match the hierarchy of the scene graph.

Flexibility

The PDOM API can provide lots of flexibility in how to display content in the PDOM. Each sibling of the peer has a name (like label or description), but at its core it is still just an HTML element, and it can be any tag name specified. Below is an example of a node that is used just to add text content to the PDOM. In looking at the example, remember that there are default tag names for supplementary peer Elements. (Note: as of writing this, sibling tag names default to "p").

In this sense, the naming of the options to control each sibling in a bit "arbitrary," because you can use the API for what will work best for the situation. Every node does not necessarily require all four HTML Elements of its peer in the PDOM, use your judgement.

Keyboard Navigation

The PDOM API supports keyboard navigation only on the node's primary sibling. A general philosophy to follow is to have the DOM Element hold as much semantic information as possible. For example, if there is a button in the sim, it is an obvious choice to use a "button" element as the node's primary sibling tag. Another solution that works, although it is much worse, would be to choose a div, and then add listeners manually to control that div like a button. As a "div", an AT will not be able to tell the user what the element is. In general try to pick semantic HTML elements that will assist in conveying as much meaning as possible to the user. Although it is possible to use the ARIA spec to improve accessible experience, it should be used as little. Remember that the first rule of ARIA is to not use ARIA! Instead favor semantic HTML. Addressing semantics any further goes beyond the scope of this document.

Input types

If you specify a tagName: 'input', then use the inputType option to fill in the "type" attribute of the element. There are also inputValue and pdomChecked options to manipulate specific and common (that we found) attributes of input tags. If you need more control of the primary DOM element's attributes, see Node.setPDOMAttribute().

The above example is a node whose PDOM representation is that of a basic checkbox. In order to give it interactive functionality, use Node.addInputListener(). The function takes in type Object.<string, function> where the key is the name of the DOM Event you want to listen to. See Input.js documentation for an up-to-date list of supported scenery events, and the subset that come from the PDOM. This event is more often than not different than the listener needed for a mouse. Don't forget to remove the listener when the node is disposed with Node.removeInputListener().

Note that supplying a label tag name for the label sibling automatically set the for attribute, ensuring a proper Accessible Name, see this section for more details on the PDOM and setting accessible names.

Focus

All interactive elements in the PDOM receive keyboard focus, but not all objects in the display are interactive. For example, using PhET Interactive Simulations, the sweater in Balloons and Static Electricity is a dynamic content object because its electrons can be transferred to a balloon. Even so it is not directly interacted with by the user, thus the sweater never receives focus.

When an element in the PDOM is focused, a focus highlight is automatically rendered in the display to support keyboard navigation. This occurs when you specify a tagName that is inherently focusable in the DOM (like button). For more complex interactions, type input, or other native and focusable elements, may not work. Other tag names can be focused with the focusable option. If a specific focus highlight is desired, a Node or Shape can be passed into the focusHighlight option.

Visibility in the PDOM and the focus order is directly effected by Node.visible, but can also be toggled independently with the option Node.pdomVisible. When set to true this will hide content from screen readers and remove the element from focus order.

Interactive Content

The PDOM makes available the same input ability as is granted by HTML. Thus, a knowledge of HTML interaction can be quite helpful when designing an interaction in the PDOM. A precursor to any node with interactive content via the PDOM is the focusable option. Setting that option to true will allow keyboard (or AT) input from the PDOM to the node. From this point, the events that the primary sibling receives depends entirely its accessible name and role. See below for a dedicated section about setting the Accessible Name of a node. The ARIA attribute role can help inform the user to the custom interaction (use the ariaRole option). For example using the ARIA "application" role has worked well for freely moving, draggable objects. This will tell screen readers to pass all input events through to the DOM element in the PDOM, like "keyup" and "keydown" which aren't provided for buttons when using many AT (only "click"). Focusable elements can be manually focussed and blurred using the Node.focus() and Node.blur() functions.

Once the PDOM structure is as desired, and you know what events are expected from interacting with that node, use Node.addInputListener() to add event listeners to the PDOM events. See Input.js and its documentation for up-to-date notes on the events that are supported from PDOM elements. Among these are keydown, keyup, click, input, change, focus, and blur.

Manipulating the PDOM

Most properties of the ParallelDOM.js trait are mutable so that the PDOM can update with the graphical scene. Here are a few examples:

tagName: set/get the tag name of the primary DOM sibling of a node
labelTagName: set/get the tag name of the label DOM sibling for a node
descriptionTagName: set/get the tag name for the description DOM sibling of a node
innerContent: set/get the text content of primary sibling of a node
labelContent: set/get the text content of label sibling for a node
descriptionContent: set/get the text content of description sibling for a node
ariaRole: set/get the ARIA role for the primary DOM sibling for a node

Up to this point these have only been offered as options, but each of these can be dynamically set also. Setting any of the .*[tT]agName setters to null will clear that element from the PDOM. If you set the Node.tagName = null, this will clear all accessible content of the node.

Please see the ParallelDOM trait for a complete and up-to-date list of getters/setters.

A note about Accessible Name

The "Accessible Name" of an element is how AT identifies an element in the browser's accessibility tree. Diving into the nuance of this idea goes beyond the scope of this document, but understanding this is imperative to successfully creating an accessible PDOM. For more info see background reading about the topic.

Here is an overview about the various ways to set the Accessible Name via the Scenery PDOM API.

Element's Content: Example <button>My Button</button>. To accomplish this with the PDOM API use innerContent.
label Element: a label element can be associated with an interactive input type that does not have inner content in order to provide the input with an accessible name. A label is the preferred naming method when the display's interaction has visible text-based identifying it on screen. A label element can only be associated with "lable" elements like typical interactive HTML elements. To add an Accessible Name via a label, set the labelTagName to "label" and the "for" attribute will automatically be filled in to point to the primary sibling.
aria-label Attribute: an ARIA attribute that can provide an accessible name. For the pdom API use the ariaLabel option to set the value of said attribute on the primary DOM Element.
aria-labelledby Attribute: this can be used to associate an HTML element other than the label element to another element. The elements do not have to be right beside each other. This can be accomplished with addAriaLabelledbyAssociation(). This function takes an argument that looks like:
```
        
          this.addAriaLabelledbyAssociation( {
            thisElementName: PDOMPeer.PRIMARY_SIBLING,
            otherNode: nodeWithAccessibleName,
            otherElementName: PDOMPeer.PRIMARY_SIBLING
          } );
                  
```
This association object is used to specify what the other node is that is going to be this node's Accessible Name. It also allows you to choose which PDOM element specifically, though quite often it is the primary sibling. This method is also supported for aria-describedby, see addAriaDescribedbyAssociation

Ordering

To manipulate the order in the PDOM, use Node.pdomOrder = []. Scenery supports a fully independent tree of PDOMInstances to order the PDOM versus the ordering based on the nodes into the Instance tree. Because of this, you can use Node.pdomOrder to largely remap the scene graph (for rendering into the PDOM) without affecting the visually rendered output. Node.pdomOrder takes any array of nodes, even if the they aren't children to that node. Note that a node must be connected to the main scene graph (via children) in order to support being in a pdomOrder. Thus you cannot only add a node to a pdomOrder and expect it to render to the PDOM.

Interactive Alerts

All interactive alerts are powered with the aria-live attribute. PhET manages alerts in a custom queue, see utteranceQueue.js Each accessible display is outfitted with an UtteranceQueue that can be passed alerts to it. All PhET alerts should go through utteranceQueue, aria-live should not be added to elements in the PDOM.

Performance Considerations

Manipulating the DOM can be performance intensice. If the DOM is modified every animation frame, performance of the application can be reduced on slower devices like tablets and phones. Performance can be improved by limiting the frequency of setting accessible content and attributes where possible. So in general, it is good practice to set accessibility attributes as infrequently as possible. There is some work in progress for Scenery to batch the updates to the DOM so that the frequency of updates is reduced per animation frame. Please see the following issue for their status and potential work to be done:

Batching PDOM changes

Next Steps for Understanding

Please discuss developer related questions or problems with @jessegreenberg or @zepumph, and update this document accordingly to help those who follow in your footsteps. Also @terracoda is a great resource on questions about ARIA and web accessibility in general.

PhET Published Resources

Source Code

For up-to-date documentation and the latest API for accessibility in Scenery, please visit the source code.

Good luck and happy coding!