Breaking the Sound Barrier: How Game Audio Can Improve Accessibility

Breaking the Sound Barrier: How Game Audio Can Improve Accessibility

Adriane Kuzminski: All right everyone. My name is Adriane Kuzminski,
and today I would like to talk to you about the role of audio in accessibility. I am just going to move
this down a little bit. There we go. All right.
What my talk will cover is what is game audio, who can benefit from accessible audio, and the methods and challenges
of making games more accessible. A little bit about me. I am a sound designer
with a passion for accessibility. I had first heard about accessibility
when I came across the AbleGamers booth at PAX East in 2013. After that, I read about color blind
filters in World of Warcraft. This made me think,
but what about blind gamers? So I looked into the blind accessible games that were available,
and I noticed a few things. One is that there was a huge lack
of mainstream blind accessible games. Second of all, I realized that
a lot of blind accessible games made by the community had
kind of poor quality audio. However the audio games made by
sound designers were inaccessible. So this showed me that we need more people with technical backgrounds who are passionate about accessibility and can promote
conversation between gamers and developers. So, a little bit about what I
used to learn all of this stuff. I read different theses, I went to a lot of different
conferences like CSUN, AES, NAMM, and GDC
of course, accessibility meetups, NFB and game conventions, Abilities Expos,
conversations with friends, and of course, with the blind community. I also learned from my involvement
with a few blind accessible indie games. One is Frequence Missing. Some of you may have heard
about this game before. Eric Robinson gave a talk about it at the last GDC called Audio Driven Gameplay. There is also Earplay,
Steno Hero, and Sight Unseen. All these games are still being developed. With these games, I was not a developer, but rather like an accessibility consultant. I would help test for the
screen reader compatibility, help them find what audible information they needed for players, and finding out different ways to express player orientation through sound. And of course, I advocated that they
should test with blind gamers, not me. Not just me, but with blind gamers. Just a little warning. It is going to be walls of text
throughout my PowerPoint. The great part was that with these games, I was able to see a lot of
different ideas in action. This is why I want to help many
accessible games get out there. They help show these accessibility methods
in an attractive way. So rather than creating
little demos that aren’t that interesting
to show blind accessible methods, here is a full product that
people can actually play. This is the best way to inspire developers, other than personal stories of course,
which are the most powerful. These also help developers
get off “square one” with collaboration and foresight. This helps to solve the tale
of the sighted developer who says, “Wow, accessible games! Cool. I want to make one. It is going to be like new and revolutionary. Wait? What is voice over?” [Audience laughing] With developers who understand
what is possible, with accessibility and with audio, those people are more likely to succeed in creating high quality
blind accessible games. We are talking about accessibility
and game audio. So what is game audio? Typically, it is defined as sound effects, ambience, music, and dialogue. The purpose is to make the games more realistic, more immersive, and to enhance the emotion. However, I will not be talking
in these four terms. I will instead be talking about instructional,
locational, and emotional sound. As a warning, screen readers though
they are an audible function, they are not in the sound designers’
control, typically. However, I will still be talking about them. So who can benefit from accessible audio? Well, there is the obvious group which is those who are affected by all visual elements of a game. People who are blind. This makes up 39 million people. For the purpose of this talk, I am actually separating people
who are blind and low vision, even though that I understand of course,
that blindness is a spectrum. There are those people
who are affected by text. Perhaps they can see the UI,
and they can see their location, but they just can’t read what is on the screen. This would make up people
who have low vision, which would be 285 million people. This is according to the
World Health Organization in 2012. We have a much bigger population since then. This also affects aging gamers,
which we just had a whole talk about. Again, this is a huge population
of gamers at 27%. That is huge. There are also gamers with dyslexia. This makes up 6-17% of
the world’s population. That is a huge number of people. This will be outside the scope of my talk, but there are also those who are
learning a second language. This makes up almost 500 million people just for those who are learning
English and Spanish alone. And finally, there are those who can benefit from controlling the volume
of certain sounds. This would make up gamers with autism and sensory processing disorder or SPD. One percent of the world
population has autism, and 5-16% of children have SPD. So we talked about what is game audio, who benefits from audio,
and the role of audio in games which would be realism,
immersion, and emotion. But what is audio’s role in the gameplay? There is an initial issue
that I saw with this because a lot of user interface and game designers are very visual thinkers. Because audio focuses on the
realism, immersion, and emotion, audio is often limited to just aesthetic information instead
of gameplay information. However if we approach audio similarly to user interface design by focusing
on user control, user memory loads, and consistent interfaces, then we can start to translate
visual information audibly. I believe this is part of the key
to accessible audio design. So how do we do this? First of all, players have questions
that we need to answer. One would be, what should I do? This would be instructional
gameplay information. There is also where am I? Where should I go? This would be locational
gameplay information. Even though this is more
aesthetic and perhaps it could be seen as gameplay
in more of an abstract way, we also can’t forget the question
which is what should I feel? Which would be emotional information. With these three types of audio,
we can help break down accessible audio. A warning, I could not fit
the whole framework, groundwork, all the academics, and everything into this talk, of course,
because it is only a half hour. I am actually going to skip
all of that and move onto just the practical information, and what you can do. First of all, there is the question
of what should I do? This is instructional information. Like I said, wall of text. The purpose of this would be to
help players learn the game through audible text icons and actions. With text, the obvious answer
is that menus and tutorials need to be read by a screen reader
or by voice actors. Some examples of this would be in King of Dragon Pass, Skullgirls,
and games made by Choice of Games; those all use
text-to-speech, or screen readers. In Girl to the Thief and Frequency Missing,
those use voice menus. There is also the user interface. These icons need to be audible,
whether this is verbally or symbolic. You could take the approach of displaying
both at the same time, having verbal and symbolic sound
at the same time. With this, the player could hear both, and then once the UI sound is memorized,
then you don’t need the text anymore. This would be similar to Hearthstone,
where you have player cards. They have a picture, and they have
the description of what the card can do. Well, the players don’t end up
reading the card every single time. Eventually they just see the picture on it, and they already know what to do
just from that image. There are also actions. This would go along the lines of the
user interface design. Using clearly, easily recognized
or competitive dialogue, and sound effects. With Mortal Kombat and Injustice, they did this by limiting
the number of hit, punch, kick, and yelling sounds;
and made those unique to each character so they were
very easy to memorize. We also had this in Overwatch. The ultimate abilities are always the same. There is just the friendly version,
and the enemy version. This limited variation, like I said, allows for faster memorization,
immediate confirmation of the player’s actions,
and promotes quick reactions. You don’t always have to have
just one sound effect because of course,
it starts to get a little old and a little annoying sometimes,
if it is not necessary. An example of this would be in Overwatch. The characters have unique footsteps. Rather than the same footstep sound
every single time, there is a range, but it easy to memorize who that character is
from the qualities. You could also have a
customized player experience. Again in Overwatch, the game audio is mixed by the threat level instead of realism. They don’t use it exactly for accessibility. They are focusing more on
player speed and recognition, but like I said, that is a form of accessibility, even though
it is not intended that way. The sounds from enemies are
more noticeable than sounds from teammates, even though that is not
technically realistic. Now you have locational information, which answers where am I,
and where should I go? The purpose of this is to
make sure player orientation is clear, and that they can confirm
their location and direction whenever they need to. I believe those are two
very important qualities of this. We have the simulated acoustics, like which was mentioned before,
especially with VR now. You have all the nerdy stuff
like 5.1 imaging, HRTF, obstruction occlusion, etc. However, physics based reverb models and detailed positioning
aren’t the complete answer because the player
needs to know the direction. They need to know their direction,
and not just their location. So for some games,
this is not actually that complicated. In Mortal Kombat,
they just used stereopanning. That is it, just left and right. This tells a player not only
what side of the screen they are on, it also tells them what side
of the enemy they are on. Like I said if direction is then the issue, if that is the focus and the hard part, then how do we show it? You can use different types of sound cues. This would be adaptive music,
which is used in just about every game. This would be when the music changes
if you are going the wrong way, or if you are coming up on an enemy. You can have assisting characters. This was used in the game A Blind Legend. Even in Dishonored,
you have a character who says, “Hey, follow me. Follow me.” You follow that person. You could also do this
rather than just with characters, but with also sound effects.
Kind of like sound beacons. This is used in Papa Sangre and Sight Unseen where you can follow or avoid certain sounds. You can also have memorable sound effects. An example of this would be if you have a dungeon crawler,
each door could maybe have a unique sound so they can know
if they have been to that door before. Also with memorable ambiences, this happened to me
when I was playing Silent Hill 2. I like to kind of wander around and not really play the game
like it is supposed to be played. I got lost, but I knew I had heard the ambience that I was hearing before. So I knew I was going backwards. There are also experimental ways,
such as digital white canes, which was used in the Gone Home mod. There is sonar. I put Alien: Isolation up there,
but that is at least an example of sonar. You also have… This has not been used,
but right or wrong footsteps. If they are going on a linear path, there is the right footstep sound
or the wrong footstep sound that tells them if they are going
in the correct way. You can also have verbal cues. This could be as simple as
cardinal directions. This was used in the game Swamp,
which is a zombie shooter audio game. They had a shortcut so you could know if you were going North,
East, South, or West. You could even use these for
northwest, southwest, etc. You could have audio description
of the environments. In Frequency Missing
when you come into a room, the room is actually described to you
when you first walk in. You can have audio descriptions
from touch activation. This is used more on phone and tablets
with the screen readers. You basically put your finger down, and then you will be told
what is underneath your finger. This is also used in the game
Frequency Missing. Finally, there is emotional information. What should I feel? The purpose of this is
players should not be distracted from the task or cause unintentional anxiety. You could use relaxing music
or sound effects. This could help gamers with autism and SPD. An example of this would be the music in Civilization or music
in many puzzle games. You could also have audio sliders. This would also help gamers
with autism and SPD. This would allow players to
remove certain sounds that they are sensitive to, and it would allow them to reintroduce them later. So they could turn it all the way down,
and then maybe start turning it up a little as they get used to it. Then, raise it all the way up
once they are reintroduced. At first when I heard about this idea from somebody who works with
a lot of kids with autism I thought, we already have music,
sound effects, and all these other sliders. I don’t really know how we could do this. Then I started to play the Pit People beta, and they have an
additional slider for UI sounds. Just like that, it was something
I had never even thought of before. Finally, we have audio description. This would allow the characters
to be described, and their expressions
to be described in the cut scenes. So rather than just being
locational information, it could be the emotional information of the characters. This would be similar to what is being used
in television, of course. What is great is that audio description
has really started to take off a whole lot more than it has ever been. On Netflix, all original Netflix shows
have audio description. All Pixar films now have audio description. So what are some of the challenges? For one, sighted people hate screen readers. [Audience Laughing] They hate screen readers. You can go to Youtube and type in “how to turn off voice over”, and you will find about fifteen videos
of people who are so frustrated and so relieved
that they figured this out. You could look at the comments,
and everybody is going to be saying, “I love you, I love you, I love you. I was going to throw
my computer out the window.” I think this is a huge challenge. But with personal assistants like Echo Dot, like the Google Home,
and even with driving apps I think that is going to start kind of softening sighted people up
to the use of screen readers, especially after the voices become
more natural too. There is also lack of in-game
screen reader support on consoles, lack of in-game screen reader
support on popular engines, and lack of screen reader accessible clients
in online stores. You can’t buy the game, how
are you supposed to play the game? There is also consistency. Developing for Talkback isn’t the same
for VoiceOver. Those are both screen readers for phones. Also even though text-to-speech
is an audible function, it is just not the sound designer’s job. Therefore, you are separating it
from the people who really care about audio
and you are giving it to somebody who may not care as much, about the design of it anyway. Also, you have the other option,
which would be voice actors. However, voice actors are expensive. This could raise the budget,
especially given how much additional text you would have to have read out loud. The challenge with audio sliders is that it is hard to standardize exactly what sounds are more likely to trigger SPD. Of course, people are individuals. This is different for everybody. We really just need more
research on this area. There are also challenges
with the audible UI. Such as repetition is annoying,
especially to sound designers. UI sounds can be hard to memorize. Expectations for UI sounds are far behind the GUI,
the graphical user interface. And, sound designers
don’t control the UI design. There are also issues with location. Like I said before, realistic sound
doesn’t mean accessible sound. Instead of tackling every single game, I think it is good to start
focusing on the games that lend themselves most
to blind accessibility. These would be games that allow
for once choice at a time, and they give the player enough time
to make an informed decision. Such examples would be
point and click adventures like Frequency Missing,
or games by Telltale Games. However, they would have to remove
the Quick Time Events, of course. There are also turn based strategy games like the Pokemon, Persona,
and Final Fantasy series. Of course, you would need to make sure that has fully spoken dialogue,
that the combat options are spoken, and that there is an
audible navigation system. You also have trading card games
like Hearthstone or the Pokemon trading card game online. However the way I would see them anyway, they would have keyboard support and no time limit. You have side view fighting games, which are the most accessible of AAA games: Mortal Kombat, Injustice, Skullgirls, and like it was mentioned
before too, is Killer Instinct. However, they would need to
make sure that they have audible tutorials, voice menus, and no Quick Time Events. Then, there are also interactive story games, like Her Story was mentioned earlier too. There are also interactive stories for Alexa devices and Earplay, which was
one of the games I mentioned. Finally, then you have visual novels
like Steins;Gate and Naruto. However, these would need to be
fully voiced or have screen reader support. For a few non-audio based solutions, blind accessibility doesn’t just
depend on audio. However, some of these solutions don’t just help people with blindness, or dyslexia, or everybody else I have mentioned;
they also help people with mobility, hearing, and
intellectual disabilities too. Such examples would be tweakable font sizes, haptic feedback like the Rumble Paks. Using these for gameplay information not just like, “Whoa, you are getting shot at,
and it is rumbling! LFO!” It could also be seen as a way of tactile enjoyment for gamers with autism. That was one thing that was mentioned to me. Also, allow players to slow the game down. This was mentioned before. There is also auto-assist,
or even the VATS system in Fallout 3, and all the other Fallout games. There is also keyboard accessibility,
and like I mentioned before, skippable Quick Time Events. So here are a few thoughts. Accessibility is not about
limiting your creative vision. But instead, it is about removing
unintentional design disabilities so more people can play games. Also, don’t sacrifice accessibility
for immersion sake. To have more than one way to
access the gameplay information, I think that is very key too. Also, look at new technology. Look at VR, personal assistants,
voice recognition, etc. Also look at the tech gamers use
like screen readers, white canes, the smart glasses
that are starting to come out, and Braille devices. You can even bring some of these ideas
into your own game. That would actually make it more immersive. Also, look at UI design.
Like I said before, with user control, user memory, and consistent interfaces. Finally, get feedback from gamers with disabilities. Don’t pretend that closing your eyes
makes you a blind gamer. This happens. I have seen this happen more than once. It is just something I think
a lot of people wanted to actually do. But even the way that you grab your phone
and hold your phone to your face, you are already starting off
on the wrong path. Finally, I could literally give a whole talk about this, but why consider accessibility in the first place? Well first of all with gamer quality of life, games promote technology independence. Especially when it comes to iPads,
the way that we interface with technology so often, those are being used for job skills too. You also have job placement in the games industry by promoting blind programmers. Also there is social interaction,
inclusion, and the ability to relate, which is especially important for children. I know a lot of people
who want to play Minecraft. Minecraft is not blind accessible. Also, you can give the chance
to experience higher level cultural expression and art in games. I see right now that we have novels and films. That is the way we often express
our cultural expression, our humanity. But it is starting to be in more games more and more and more. Pretty soon, games might be the main way that we show that level of expression. One game I think of is Papers, Please. I think that is a perfect example of it. Also, there are technological advancements. Like I think everybody has said here, game accessibility makes games
better for everyone. Finally for your company,
you reach more people. It is as simple as that. All right, so that finishes my talk. My email, if you would like to get in touch with me, is [email protected] My first name is spelled Adriane and Smash Clay is spelled smashclay. You can also go to my website My Twitter is @smashclayaudio. And also, reach out to AbleGamers
at Or you can email me at
[email protected] I want to thank everybody
who is making more games blind accessible, and to my husband for getting this talk under 30 minutes. [Audience Laughing] For the bibliography and further reading,
that is on the site. Some upcoming blind accessible games to look out for are A Hero’s Call of course, Frequency Missing, Earplay,
Steno Hero, and Sight Unseen. Some of the other blind accessible games I mentioned were Swamp, A Blind Legend,
and Girl to the Thief. Thank you. [Audience applause]


2 thoughts on “Breaking the Sound Barrier: How Game Audio Can Improve Accessibility”

Leave a Reply

Your email address will not be published. Required fields are marked *