Voicing

Introduction

"Voicing" is an accessibility feature developed by PhET. The feature produces speech that comes directly from the browser instead of using third-party screen-reading software. Speech is generated using Speech Synthesis. It is intended to be useful for people that may benefit from described content that may not use or have access to other screen reading software. But there could certainly be other applications as well.

Speech is generally spoken in response to user input, and the API is designed with this in mind. The API allows you to define Voicing content on a node-by-node basis so that interactive components in the scenery application can create speech.

Responses

Voicing content is broken up into the following categories called "Responses", which heavily influence the API.

• Name Response: The name that labels the component using Voicing. Similar to the "Accessible Name" of digital accessibility.
• Object Response: The state information related to the component using Voicing. This is generally a description that directly describes the object that receives input.
• Context Response: The contextual response that describes the result of interaction with an object. For example, if an input element changes the properties of the application state that it not directly related to the input element it would be considered a context change.
• Hint Response: A response that guides the user toward interaction with the component using Voicing.

For example, lets say that we had a scenery object representing a magnet. We might assign responses to it like

• Name Response: Magnet
• Object Response: Weak strength.
• Context Response: Creating a weak magnetic field for other magnetic objects.
• Hint Response: Move me to modify the magnetic field.

As shown in this example, the "Name Response" labels the component, the "Object Response" describes the state of the component, the "Context Response" describes its impact on the surrounding application, and the "Hint Response" provides a guide to interact with the component.

Responses implemented with Voicing.ts

Voicing is implemented with a trait called Voicing.ts which can be composed with scenery's Node. It provides the ability to set the various responses on the Node and then make a request to speak one or more of them. The API of Voicing.ts is described in more detail later in this document.

Responses collected with responseCollector.js

The flow of responses is further controlled by a singleton called responseCollector.js. Responses of a certain type can be globally enabled or disabled. This supports (for example) the ability to add user preferences to the application that enable or disable certain responses if they are found to be too verbose or unhelpful. responseCollector.js has Properties that control the enabled states for responses, and contains utility functions for assembling final Voicing content depending on the state of these Properties.

Voicing.ts API

The following enumerates the Voicing.ts API.

voicingNameResponse

A getter/setter for the {string|null} name response for the Node.

voicingObjectResponse

A getter/setter for the {string|null} object response for the Node.

voicingContextResponse

A getter/setter for the {string|null} context response for the Node.

voicingHintResponse

A getter/setter for the {string|null} hint response for the Node.

voicingUtterance

A getter/setter for the {Utterance|null} Utterance used to speak Voicing content. By default, a unique Utterance is created and used when you use the Voicing trait. This means that speaking content through a Node with Voicing.ts will leverage the UtteranceQueue features of waiting to announce an Utterance until that Utterance has "stabilized". If you rapdily speak many responses only the last response will be spoken. But you can override this behavior by assigning a unique Utterance.

voicingUtteranceQueue

A getter/setter for the {UtteranceQueue|null} UtteranceQueue used to speak Voicing content. By default, a global singleton called voicingUtteranceQueue.js is used to speak content. But you may wish to use a different or multiple UtteranceQueues if you need more control over the output of speech.

voicingSpeakFullResponse

A function that requests speech of all responses for the Node. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingSpeakResponse

A function that speaks only the provided responses, specified through options. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingSpeakNameResponse

A function that speaks only the name response. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingSpeakObjectResponse

A function that speaks only the object response. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingSpeakContextResponse

A function that speaks only the context response. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingSpeakHintResponse

A function that speaks only the name response. Only the responses that are enabled by responseCollector.js will be spoken. This is generally called in response to some input or change in the application.

voicingFocusListener

A function is called whenever the Node that mixes Voicing receives focus. By default every response except the context response is spoken on focus. But this can be overridden if necessary.

voicingIgnoreVoicingManagerProperties

A {boolean} getter/setter that lets you ignore the Properties of voicingManager when you make requests to speak. If false, all responses will be spoken regardless of voicingManager Properties.

voicingResponsePatternCollection

Sets the collection of patterns to use for voicingManager.collectResponses. This lets you control the order of Voicing.ts responses, as well as customize punctuation and other formatting of the content. See ResponsePatternCollection.js for more information and how to create your own collection of patterns.

List out the API associated of ReadingBlock

Code examples

Simple Example

The following illustrates a basic example of using Voicing.ts with a Node. Click the Rectangle to hear speech.

Controlled Responses

The following is a more complicated example that demonstrates more direct control over speaking responses. In this example, you can drag the rectangle in the bounds. On pickup, the name response is spoken and on release, the horizontal position (object response) is spoken. This also demonstrates enabling/disabling the object response globally with the following checkbox.

Reading Blocks

"Reading Blocks" are Nodes that use Voicing, but have special behavior and a common set of Voicing options. Reading Blocks are generally used for graphical objects that are not otherwise interactive, but have some Voicing content (like Text, for example). Reading Blocks also have the following characteristics.

• Reading Blocks have a unique focus highlight to indicate they can be clicked to hear Voicing content.
• Reading Blocks are added to the focus order when Voicing is enabled.
• While Reading Block content is being spoken, a special highlight will appear around the content.

Reading Blocks are implemented with a Trait called ReadingBlock.ts, which extends Voicing.ts, and so it can be used with scenery Nodes.

Mouse Highlighting

The Voicing feature supports highlights that appear on mouse hover. This highlight generally indicates to the user that a component can receive input of some kind. This could include either components that are naturally interactive or it could include components that have become interactive purely to support Voicing content, like Reading Blocks.

Mouse Highlighting is implemented with a trait called InteractiveHighlighting.js which scenery Nodes can be composed with. InteractiveHighlighting.js will add an input listener to the Node to activate the Display's FocusOverlay when it is time to show a highlight. InteractiveHighlighting.js is extended by Voicing.ts, so all Nodes that use Voicing support Mouse Highlighting.