The IFTF logo

Accessibility Testing Report (2019)

Summary

Using a set of specially prepared games with attached player surveys, this IFTF program tested the current state of accessibility for IF game-play platforms. Based on several dozen responses from players with a range of disabilities and IF experience levels, the committee has generated a list of recommendations to improve the accessibility of interactive fiction technologies, as well as for individual IF games.

While the specific needs of players with disabilities drove our study, we believe that these recommendations can lead to better interactive fiction experiences for all players.

Table of Contents

Introduction

Project history and goals

Due to its reliance on text and its tendency to let people play at their own speed, interactive fiction (IF) has always enjoyed relatively high accessibility among digital games. Nobody really planned this outcome, though; it was a happy accident, based on the incidental accessibility of the form’s initial constituent ingredients. As such, IF-related technology developed in recent years has not always kept accessibility as a primary concern.

Recognizing this, the Interactive Fiction Technology Foundation (IFTF) launched its Accessibility Testing Project as one of its very first programs, coinciding with the organization’s public launch in mid–2016. It aimed to measure the state of accessibility within contemporary IF play platforms, and then report its findings and recommendations to the IF community.

This is that report, delivered to the public in the summer of 2019.

The program’s first chair, Deborah Kaplan, began work by gathering a team of experts in accessibility (herself included) and IF technology, with two others joining the team over the next couple of years. The committee’s final membership comprised the following individuals:

The program’s chairship passed from Deborah to Claire and on to Jason over the two and a half years of the committee’s work. Jason chaired the program at the time of this report, and serves as its primary author.

Notably, three of the committee’s members are themselves people with disabilities and assistive technology (AT) users. Deborah has mobility impairments that require the use of alternative personal-computer input methods, such as dictation software. Austin and Zach are both fully blind, and rely on screen readers and related technologies for their day-to-day digital interactions.

Sources for this report

This report, and the recommendations it provides, come from three primary sources.

  1. The response to surveys presented to a volunteer panel of players with disabilities, allowing them to describe their experiences playing the IF works prepared especially for this project. We ultimately received responses from thirty-four individuals. Even this relatively modest data-set held both meaningful patterns found across multiple responses as well as valuable insights from individual testers.

    We have attached an anonymized version of the survey responses to this report, as an appendix. The section titled “Testing method”, below, goes into more detail about how we set up and ran these tests.

  2. The expertise and personal experience of the committee members themselves. Every member of the committee is an expert on IF technology, accessibility technology, or both. Furthermore, three committee members themselves have disabilities – and use assistive technologies – that are in-line with the accessibility needs that our testing focused on.

    While our final recommendations use the survey results and the APX guidelines (described below) as their basis, committee members’ personal and professional backgrounds did play a significant role in steering this project’s ultimate outcome.

  3. The Accessible Player Experiences (APX) project by AbleGamers, a 2018 publication that codifies a relatively small, easy to articulate, and throughly proven set of accessibility guidelines for video games of all sorts. The membership overlap between AbleGamers and the IFTF Accessibility Testing project gave us immediate insight and access to these principles alongside their publication; as a result, our recommendations layer our own survey findings and committee-member knowledge upon these guidelines.

    The “Recommendations” section of this report contains further detail about APX, and our use of it.

Testing method

The program created two short IF games intended to function as accessibility obstacle courses for various gameplay platforms. Each game was created by a primary author deeply familiar with a particular IF style and its conventions, and both intentionally avoided using any extensions or techniques outside of the most basic, “out of the box” features of their respective creation systems.

After recruiting testers (see “Tester profile and recruitment”, below) and adding them to a private mailing list, the program created a website with instructions for participation, then directed testers to it. This website instructed testers to play one or both games, and for each game played, complete an online survey about their experience. Each survey asked the participant to specify their gameplay environment (including listing their OS, the browser or interpreter they used, and a description of their assistive-technology setup), and then proceeded to ask several questions about the accessibility of the game just played.

We have attached a copy of this website to this report as an appendix.

We originally gave testers around three and a half weeks, over January and February 2019, to return the surveys. We ended up extending that by a week to better accomodate later additions to the tester pool.

The games included:

We have attached to this report, in the “Appendices” section, everything that testers received in order to participate in this project: the website with full instructions, the surveys we invited them to fill out after play, and the games
themselves.

A noteworthy aside: Both of the games had a spiritual predecessor in Roger Firth’s Cloak of Darkness, a fact visible both in the theme of A Night Below of the Opera and the title of Twine of Access. Cloak is an abstract specification for a tiny parser-based IF, including a number of activities that a traditional model-world game system is expected to support: moving around among rooms, playing with light sources, hanging clothes on hooks, and so on. IFTF’s accessibility team drew inspiration from Cloak when developing the idea of a game-shaped “proving grounds” for IF accessibility.

Tester profile and recruitment

Our testing pool comprised a little more than fifty people, almost all of whom had visual or upper-body impairments, and many of whom used assistive technology – such as screen readers or magnifiers – to play the test games. We decided to focus our recruitment efforts on players with these particular disabilities due to the specific presentational style of most IF: text-centric, turn-based, and requiring keyboard-and-mouse input.

Thirty-four of our recruited testers completed at least one survey, and each of these people received a $10 Amazon gift certificate – a token of gratitude in-line with the compensation for similar tests that some members of the committee had previously participated in. (The money for these gift certificates came from IFTF’s general operating funds, and lay well within the program’s own chartered budget.)

We acquired testers via a recruiting process with two discrete rounds. First, we registered as a project with AbleGamers Player Panels program, which helps video game publishers organize accessibility testing efforts, and then connects these projects to volunteer players with disabilities. This provided us with around two dozen potential testers, all of whom we welcomed into the project.

This initial group included far fewer fully blind players than we wished to include, however, so the committee made a secondary recruitment effort that sought blind testers specifically. This involved an article on IFTF’s blog, and a call for volunteers on websites like audiogames.net. We received an enthusiastic response to this call, doubling the size of our tester-pool.

Survey responses

The committee had decided, at the start of our recruitment effort, to cap the size of the testing pool to fifty people; these two calls for players filled it out neatly. As such, we spent much of January and February of 2019 letting these testers play the games and complete the surveys.

In March, we reformatted the survey responses into the document attached to this report. We decided that the responses we received held enough varied, meaningful, and actionable information to help direct the recommendations that this report will now proceed to present.

Recommendations

Preface: APX Design Patterns

In late 2018, The AbleGamers foundation released Accessible Player Experiences (APX), a set of guidelines for video game designers interested in maximizing the accessibility of their work. Falling into the two categories of “Access” and “Challenge”, these twenty-two thoroughly researched and amply illustrated design patterns describe paths that allow creators to widen their games’ acessibility – and therefore increase their works’ reach – without diluting the experience that they wish to bring to all their players.

Our project took advantage of APX’s coincidental release just as our own testing efforts got underway. In the list of recommendations to the IF community that follow, we have paired – as much as possible – each recommendation with one or more APX guidelines. In this way, we set our IF-specific observations and recommendations into the deeply researched and thoroughly documented accessibility-improvement framework that AbleGamers has already established.

More generally, we recommend that all creators working with interactive fiction become familiar with the whole set of APX guidelies and principles in order to better understand the guidelines’ purpose, and the benefits that their application can bring to the overall quality and reach of any game. To aid with this, we hyperlink all mentions of relevant APX design patterns directly to their descriptive pages on the APX project’s own website.

We found that 18 of the 22 APX patterns matched the recommendations that we developed as a result of our testing. The remaining four do not apply to IF in most cases: these include “Improved precision” and “Slow it down”, more applicable to fast-paced action games, and “House rules” and “Play alongside”, which apply to multi-player games.

List of recommendations

These recommendations list several features that a work of interactive fiction should support in order to achieve better accessibility, based on the sources listed above (see “Sources for this report”).

We address these recommendations to the whole IF community – including authors, authoring-system creators, interpreter maintainers, and others. We acknowledge that their realization would require varying levels of planning and participation from all these community roles. Because implementation details lay outside the scope of this report, we elected to present all these recommendations as a single list, rather than attempt to divide it up into a series of sub-lists for authors, developers, and so on.

That said, we expect that IF authors can use many of these recommendations right away as design guidelines, implementable using current game-creation tools. Others may require changes in those tools, or in the programs people use to play IF, or both.

Some of these recommendations, particularly those pertaining to parser-based games, may seem aspirational, suggesting fairly significant changes to expectations about these interfaces that have held since the 1970s. This includes the familiar all-encompassing, ever-scrolling text view that offers little to no player assistance by default. We would ask designers to consider these as complementary improvements to parser IF interface standards, rather than (literally) game-changing disruptions. We do believe that changes informed by these recommendations would make parser IF more accessible to all players, including but not limited to players with disabilities.

This report avoids making even preliminary implementation recommendations by suggesting how the community should best approach these, or by labeling some recommendations as more important than others. Instead, we wish this report to serve as a starting point for community discussion, and – we hope – a way of sparking inspiration for new directions of accessible work and accessibility research from both IF authors and IF technologists.

Use alt text for all non-textual content (including “text art”)

APX Patterns: Second Channel

Every example of non-textual content that a player is expected to comprehend should have a textual equivalent. This includes the well-established web-based tradition of “alt text”: supplementary, author-provided text that does not display itself by default, but which is available on request. This request will usually come from an intermediate technology in between the game and the user. For example, a screen reader, used by a blind player, will always look for for alt text on embedded images. As another example, a parser-game interpreter might not be able to play sound at all, and should instead display alt-text for every sound effect the game wishes to present.

IF authoring systems should support methods for authors to add alt text to multimedia, much in the fashion of the alt attribute of the img HTML element. IF gameplay platforms should be able to read this information and reveal it to the player when appropriate. Finally, IF authors should always pair the use of multimedia with alt text.

Furthermore, authors should offer plain-prose alternatives to any information visually encoded with text-based effects. It is fine, for example, to draw a map in your game using text characters, or color-code certain words to have them convey additional meaning. However, a visually impaired player might not be able to comprehend this content as-is. If text-based effects contain information that player needs to proceed through the work, then the game should provide an alternate means for expressing it.

In our testing, the game A Night Below the Opera did intentionally include a puzzle that required the player to comprehend a simple map drawn with text characters, displayed as text within the main game window. While trivial for sighted players to understand, blind testers reported almost universally a complete inability to make use of it.

Our testing suggests that Inform and its various interpreters tend to have good support for displaying image alt text, but poor support for assigning or displaying alt text for audio. While Inform does allow authors to specify alt text for audio files, interpreters lack a way to access this information, instead displaying a generic placeholder message. Furthermore, interpreters that could not or did not play sound effects had a very uneven record with successfully displaying even this message. As a result, even players with full hearing ability often missed important in-game content.

Offer prominent volume / mute controls

APX Patterns: Clear Channels, Leave It There

All games should make clear, from the outset, whether they contain audio effects. This is of obliquely surprising relevance to IF, a game style which is often but not always totally silent.

Our two test games involved a parser IF with present but minimal use of sound effects, and a web-based hypertext game with no sound at all – neither particularly unusual cases for IF. Despite this, testers often found themselves confused about whether either game was supposed to feature any sound, and whether it was playing properly if so. The presence of even simple, ad-hoc sound controls (including the simplest control of all, a “This game has no sound effects” message) would have helped mitigate this confusion.

Games that do feature integrated sound should allow players to adjust the volume level independently of their system volume; this allows the sounds to play without interfering with a screen reader’s spoken output, for example.

Allow the player to set text preferences, including “dark mode”

APX Patterns: Clear Text; Leave It There

As text-centric work, every IF game should allow its players to adjust how text is displayed. Beyond letting players express aesthetic preferences, this can assist players with visual or other disabilities in making the game’s text easier to read and comprehend. At its most basic, this feature should allow players to adjust the size of the on-screen typeface. More advanced settings may involve the choice of fonts, margin and spacing settings, and so on.

Most modern interpreters and web browsers already allow players to set these sorts of display preferences, without authors needing to put in extra work. However, IF authors should test whether their game continues to work even if players increase the text size in this way.

Beyond allowing players to adjust the size and shape of type, IF games should also offer an inverted-contrast or “dark mode” option that displays light-colored text on a dark background. We were very interested to observe that many testers with vision impairments requested this feature, specifically.

Pair special updates with sounds

APX Patterns: Second Channel

Consider having your game pair important, recurring events – an increase in the player’s score, say, or the discovery of a new area, or even just picking up objects – with sound effects. The mapping between actions and sound effects should be consistent, within a given game. (An intriguing chime with a new discovery, a brass fanfare with a point scored, and so on.)

Per the APX “Second Channel” pattern, changes in game state that deserve special attention from the player benefit from having this information presented in more than one sensory channel. This presents an inexpensive and effective way to make important changes more obvious to all players, including those with an impairment involving one of those senses. (Or to those with a cognitive impairment, or even those who are simply playing the game in a distracting environment.)

Careful use of audio effects as a second information channel can especially help screen-reader users, who might miss text effects that a sighted player would more easily notice, whether they’re scrolling past (as in a parser game) or updating text already on-screen (as common in hypertext games).

We do not recommend reliance on an audio channel’s presence: some players are hearing-impaired, of course, and others will play in a noisy environment, or with their speakers shut off. The audio channel’s presence should be complementary to the game’s text, not interdependent with it.

Offer quick-travel and other in-game shortcuts

APX Patterns: Do More with Less

IF games tend to involve a lot of continuous mouse-and-keyboard input. Offering shortcuts that minimize especially repetitive input can help players with mobility impairments for whom typical mouse-and-keyboard use carries a higher time or effort cost than a typical able-bodied player would face.

Interactive fiction (and adventure games in general) already have a wealth of techniques along these lines, such as “fast travel”: clicking a region on a map to go there instantly, or (in a parser game) typing in the name of an already visited room, rather than manually entering all the steps required to get there. These serve as a welcome convenience to one’s entire audience – and to players with certain disabilities, they can make the difference between an accessible game and and inaccessible one.

Offer an Undo command

APX Patterns: Undo Redo

Parser-based IF has a deeply set tradition of making UNDO a standard verb recognized at the system level, rewinding the in-game clock, and thus allowing a player to change their mind about an already-issued command.

While benefiting all players, this provides additional assistance for players with certain cognitive or physical disabilities that might cause them to enter commands in error more often than a typical able-bodied player.

As such, every kind of IF game – parser-based or otherwise – should strive to include an undo feature of some kind.

Offer hints and walkthroughs

APX Patterns: Helping Hand, Bypass

Because of its foundational history involving devious puzzle games, IF already enjoys a rich tradition of sharing walkthroughs and other hinting systems for its work – often written by the games’ own authors. According to APX, giving players tools to un-stick themselves, when they find themselves unable to continue through a game’s content, adds significantly to a game’s accessibility.

In a turn-based text game, providing these tools can prove as simple as shipping a separate walkthrough document alongside the game. We encourage the community to continue considering this as bare-minimum but still appreciable effort. In-game hinting systems of various kinds can require significantly more work to produce, but can offer a much richer experience to the player than simply looking up the next action they need to perform.

Our test materials included a detailed walkthrough for A Night Below the Opera, and instructions that testers should feel unshy about using it. Many players reported making use of the walkthrough to proceed further in the game than they would have otherwise.

Notably, almost every blind player required the walkthrough to solve the “dance floor” puzzle, otherwise nearly insurmountable for any non-sighted person due to its reliance on a text-drawn map. This demonstrates how a thorough walkthrough can serve as an accessibility tool of last resort in many circumstances. While authors shouldn’t rely on walkthroughs to solve every outstanding accessibility issue in their games, they can nonetheless help players get past otherwise inaccessible (and, often, unintentional) challenges.

This recommendation also applies to subtler, “in-world” hinting and guidance that IF games can uniquely offer. A significant portion of the tester feedback for A Night Below the Opera resembled typical parser-game playtester feedback (even from folks who had not played a parser game before): suggesting synonyms for certain in-game objects or actions, or example, or stating that a certain puzzle lacked adequate in-game prompting as to its presence or potential solutions.

The good news, here, is that the standard playtest-and-improve iteration cycle for IF game development also help make a game more accessible. The APX “Helping Hand” pattern includes the signature of a thoroughly playtested IF game, where puzzle solutions and other intentionally obscured actions have a wide range of paths leading to them, with the game gently guiding and hinting the player towards the solution as they explore the space.

Parser IF should offer a separate “training ground” for new players

APX Patterns: Training Ground

Parser IF games have a rather deserved reputation for their steep initial learning curve, which can serve to frustrate and lock out many potential players – disabled or otherwise – from the get-go. The parser-game creation community has come up with many ways to address this over the years; common strategies include displaying short, generalized explanations for operating the parser, or revealing “tutorial messages” during gameplay when the player encounters various parser-game conventions for the first time.

The APX guidelines would suggest a somewhat different approach. Instead, consider offering a gated-off area where the player can freely experiment with the parser game’s model-world of interconnected rooms. This area can contain an assortment of interactive objects that demonstrate various properties typical to parser IF, such as portability, containment, or stackability. This would allow a new player to get hands-on familiarity with parser-game basics, and at their own pace, before confronting the wholly separate cognitive load of the game proper.

One could imagine that, at minimum, a game could link to a shared sandbox-game designed expressly for this purpose – perhaps hosted online, or shipped alongside the game, or even embedded within it by way of a code library.

List content warnings, when applicable

APX Patterns: Moderation in All Things

Games containing emotionally intense content – what APX lists as “surprise, violence, gore, or sexual themes” – should make the presence of this content known up-front. This can take the form of a detailed “content warning” list, or a simple one-line advisory. (e.g. “This game contains adult themes and may not be appropriate for young children.”) Authors can opt to hide warnings behind a separate command for the benefit of spoiler-averse players, but in any case the presence of warnings should still be obvious to all.

This practice benefits all kinds of players, including those with mental health difficulties who may wish to avoid certain kinds of content, and parents who want to play IF games with their children without exposing them to material intended only for grown-ups.

Interestingly, outside of a very few historical curiosities, IF lacks a tradition of offering self-filtered content, despite its text-based nature. For example, one could easily imagine a setting in a particular game that obscures or even rewords passages containing profanity, depending upon a player-controlled setting. APX shows how such settings are not uncommon in commercial video games; IF authors may wish to consider a similar approach in their own work, for the sake of making it more accommodating to players who, for whatever reason, would rather avoid reading explicit language.

An aside: one tester did state that Twine of Access should have had a “food” trigger warning – for the benefit of players with eating disorders – based on a scene where the player can sit at a banquet table and consume an entire, deliciously described turkey. While we don’t necessarily advocate this content-warning approach over others, it does illustrate another example of how playtesting a game with a diverse field can yield unexpected and interesting feedback.

List obvious exits and objects

APX Patterns: Distinguish This From That; Total Recall

Games should display a list – separately from the main display, ideally – of all the important interactive elements within the player’s current in-game content. This includes exits they can pass through, objects they can pick up or examine, and characters they can interact with.

The list can update itself as the player’s knowledge of the area changes: for example, with an addition to the exit-list appearing if the player solves a puzzle to reveal a secret door. (See also “Pair special updates with sounds”.)

A game can offer a toggle that hides this display from players who do not wish to have this information so explicitly listed out for them. However, the game should make this information visible by default, and let players seeking a more “traditional” experience shut it off. This approach serves up welcome assistance to players with disabilities that need it, such as those with memory impairments, while also providing a friendly aid to all players new to interactive fiction.

Prominently display a command guide

APX Patterns: Total Recall

IF games that expect user input not self-evident to a brand-new player – and this includes all parser-based games – should prominently display a simple guide to common commands. This can complement deeper guides and tutorials that might otherwise accompany the game.

In the case of a parser game, this might show only a handful of the most fundamental actions for movement, examining things, and interacting with objects or characters, along with any additional commands of special interest to the particular game at hand. It can also end with a pointer to further assistance the game offers (e.g. “Type HELP for more detailed instructions”).

Players can toggle this display off if they choose, but games should display it by default, for the same reasons as given for the “List obvious exits and objects” recommendation. It is a boon to players with cognitive disabilities especially, but also to all new-to-IF players generally.

List frequently accessed information in separate panes

APX Patterns: Personal Interface; Distinguish This From That; Total Recall; Leave It There

Information that players must to refer back to frequently while exploring – such as the full (“verbose”) description of the current room, or their inventory list – should appear in panes separate from the main interaction view (and may also appear as usual in that main scroll).

This serves a purpose similar to the previous two recommendations, helping to keep the player continuously “grounded” with a sense as to their current location, nearby (or possessed) objects to interact with, and available actions to try.

Games should furthermore allow the player to adjust the shape, size, location, and text-display options of these panes, and remember these settings in between play-sessions.

Avoid assuming able-bodied life experience

APX Patterns: (None in particular)

A game should not hang progress on the player’s having knowledge that would be immediately obvious to an able-bodied person based on life experience, but potentially more obscure to a person with a disability.

Our test game A Night Below the Opera intentionally involved one such situation, with a simple puzzle whose solution depends on the player’s foreknowledge that the numeral “9” looks like an inverted “6”. More than one blind tester found themselves stymied by this puzzle, with one player calling it out as potentially unfair to blind people.

This recommendation has kinship with Graham Nelson’s “Not to need to be American” rule, stated in his “Bill of Player’s Rights” essay. This rule advises creators to avoid assuming that the player has some specific cultural knowledge before playing your game. The example he named was the infamous “diamond maze” from Zork II, which a player can intentionally solve only if they recognize a certain explorable area as a baseball field, a fact hidden from the player through the use of obfuscated language.

Almost anyone regardless cultural background of can learn the rules of baseball, just as almost anyone regardless of visual ability can learn the relative shapes of digits. But an author should take care when assumptions about what knowledge, based on life-long personal experience, the player brings with them.

Longer games should offer saving

APX Patterns: Save Early Save Often

Games offering a challenge of any sort – whether of dexterity, puzzle-solving, or even difficult emotional navigation – should offer players the option of permanently saving their progress once they have completed that challenge. This allows players to continue exploring the work without making them repeat something that may have proved especially taxing for them, possibly due to disability.

Design systems for traditional parser-based games, such as Inform, already offer rich support for player-directed game saving at the core level. Newer systems – or “hand-rolled” games that use purpose-built engines – should keep this need in mind, for anything other than the shortest work.

Add accessibility awareness to creation tools

APX Patterns: None in particular

IF creation tools should themselves be aware of accessibility best practices, and treat creators’ disregard for them as errors. For example, if an author places multimedia in a work and does not provide any alt text for it, the creation tool should display a warning – or even a work-stopping error – at compile-time.

Followup actions

This section lists some next actions that we recommend the IF community to consider undertaking, separately from the game-design and technological recommendations above.

Further testing with parser-game screen effects

While quite short, A Night Below the Opera used a number of screen effects common to parser IF – and, in this case, supported at core by the Inform authoring system. According to our testing, two of them caused especial confusion with screen-reader users: box quotes, and full-screen menus (such as those created with Inform 7’s standard Menus extension). Some players also reported difficultly noticing status-bar updates.

Confusion around these effects was not universal among screen-reader users, and we suspect that players more experienced with parser IF conventions had less trouble with them. Determining this properly would involve the work of more focused testing with screen readers, involving both veteran IF players and newcomers.

Notably, if our suspicion is correct, then new players who use screen readers might have trouble using menus – and since menus very often serve as a game’s primary channel for delivering tutorial information, this may uniwittingly lock these players out of the game entirely.

Further testing with non-Latin Unicode characters

Many screen reader users reported problems reading a set of signs in A Night Below the Opera that featured the same message printed in different languages – including “emoji language” – but the survey responses themselves aren’t enough bring us to any firm conclusions about the issue.

First, at least one response made clear that the player didn’t realize that the signs were interactive and examinable; this might account for at least some of the other testers’ reports that they never saw the signs in any language other than English.

Another tester did examine all the signs, but couldn’t read the non-English signs well, and chalked it up to a lack of optional language support-packs installed on their screen reader – this, too, may have accounted for some share of testers who couldn’t read the other signs, and raises interesting questions about whether a game ought to, for example, advise players ahead of time whether to expect certain non-Latin character sets to appear.

Finally, two testers reported that their screen readers announced the emoji sign to them, but they seemed to only recognize the more common characters in the message – the grinning face and a thumbs-up, but not the shower-head or water-drop symbols.

As the practitioners of a text-centric medium, the IF community ought to have a deeper understanding of how well commonly used screen-reader technology renders the Unicode characters that look so beautiful to sighted players on modern play-platforms. We would argue this especially for emoji, that character set increasingly used by writers, readers, and game-makers across all languages.

Make a habit of accessibility-testing new work

The interactive fiction community has a rich and deeply ingrained tradition of thorough pre-release playtesting for new work. As already noted in this report, this alone benefits a game’s accessibility. We recommend that game creators improve this tradition even further by making an effort to include assistive-technology users among their playtesters.

A significant amount of the survey response from the two test games doesn’t lend itself to generalized recommendations so much as highly specific advice regarding various details of the games – for instance, the screen layout of the panes in Twine of Access received comment from multiple screen-reader users who would have preferred a slightly different presentational order. This exemplifies the kind of advice that a game creator would receive only through adding players with disabilities to their testing-pool.

Test the accessibility of IF creation tools

This program launched with the intent to measure the accessibility both IF game-play platforms and IF creation tools, including the Twine and Inform software applications themselves. It became clear after the fact that this represented two related but discrete projects, and we opted to focus entirely on the field of playing IF, rather than that of its creation.

We acknowledge that assuring the accessibility of IF creation tools remains an open and important question. Another testing and reporting effort – which, we humbly suggest, might take some lessons from this one – would help address this.

Other notes and observations

Twine scored better than expected

Before running this test, we expected Twine of Access to present serious accessibility challenges to players who rely on screen readers. It contains various features common to Twine work that we considered obstacles to good accessibility, including hyperlinks whose text changes when the player clicks on it, or which replaces the link itself with other non-interactive text.

To our surprise, few blind or visually impaired players seemed to experience any particular difficulty with the Twine game, and on average described it as far more accessible than the Inform-based test game.

Our hypothesis: After many years of a web dominated by sites filled to the brim with embedded scripts that cause all manner of effects, blind computer users are simply acclimated to having websites do strange things with their content from time to time. Having a clicked link change its text may, in the bigger picture, be small potatoes compared to al the challenges posed by the whole wide world of varyingly screen-reader-friendly web pages.

We recommend that creators using Twine – or other IF technologies that base themselves on web standards – build on this (perhaps back-handedly) good base-position to maximize the reach and accessibility of their work, by way of the advice and guidelines this report presents.

An observation about screen reader varieties

Notably, the most popular screen-reader among our testers was NVDA; of the fifteen testers that reported using screen readers, eight said that they used NVDA, and at least two mentioning third-party add-ons specifically useful for playing IF games. If future accessibility work by the IF community seeks a de facto standard screen-reader technology to test against, this might prove a good initial candidate.

Appendices

The instructional website

During testing, IFTF hosted the instructional website itself, on its own servers. Here is an archived version of that website.

Note that the surveys it links to no longer work. However, you can find archived versions of them below.

The surveys

We created the surveys using Google Forms, after determining that this service itself featured adequate accessibility for our purposes.

Here are copies of those surveys, converted into PDF documents.

The responses

This section groups all the survey responses we received by game, and then by question within each game’s associated survey. We’ve replaced each response’s identifying information with a parenthesized “tag” summarizing that respondent’s gameplay environment.

The games

These games are also linked, with more contextual information and instructions, from the instructional website, above.

List of testers

This list includes all testers who completed at least one survey, and shows the name they provided while doing so.