Bilingual AAC

In the beginning

When Augmentative and Alternative Communication (AAC) systems first became available, most could only be used with one language. A few that supported multiple languages, but not as integrated systems. You had to choose one language or the other. It was either impossible or extremely complex to switch over to the other language. This did not provide a great experience to someone who needed to use two languages on a daily basis.

One of our core beliefs about AAC is that it should be a tool that allows the user to explore the reality of language use. It’s not a strict rule book that dictates how you can or should use language. With this in mind, we wanted to make a system that reflected real bilingual use. We needed to support the actual vocabulary, voices, and use patterns of someone who is bilingual or multilingual. We also decided early on not to make this a premium feature or add-on that costs extra. Our users shouldn’t have to pay more to speak the languages that are part of their life.

A multilingual text-to-speech app

Our first step on this road came when we released Proloquo4Text. We decided to build it as a multilingual app from the very beginning. You don’t need to turn on special settings or make additional purchases - it’s all right there. Multilingual use is as seamless as possible. The app detects the language you’re typing, and uses the proper Text-to-Speech voice. This is even possible on a sentence by sentence level. You can also switch the language of the interface, stored phrases and prediction with just a few taps. It’s as easy as changing language keyboards. Today almost 20% of people using Proloquo4Text do so with two or more languages.

There’s a large number of bilingual Spanish and English speakers in the United States, so it made sense for us to offer a comprehensive bilingual experience with the Spanish version of Proloquo2Go.

We also wanted to offer personally relevant experiences for AAC users. AAC has to be available throughout the day for an AAC user to start communicating with it. What do you think will happen if someone can only use AAC in English, even though they speak Spanish with the most important people in their life? The likelihood that they’ll abandon the system increases.

We also think it is important for users to express their own unique identity and heritage. Including more languages and voices that users can identify with helps. Their AAC reflects who they are, no matter the time of day or conversation partner. This was already a reality for some, with regional accents in English.

Taking on the challenge

Creating a bilingual version of Proloquo2Go with European and American variants of Spanish was the logical next step for us. Making text-based AAC multilingual is one thing, but doing the same for a symbol-based system introduces a new level of complexity.

When we decided to do this with Proloquo2Go, our symbol-based AAC solution, we were setting out into a great unknown. There was some theoretical research which suggested that it would be valuable. But there were no real-life examples to base our work on – we essentially had to learn by doing.

While there was extensive existing research of core words in English, research about core words in Spanish at that time was very limited. Most Spanish systems inspired by core words were just direct translations of English. They didn’t actually reflect the words used most frequently in Spanish. So AssistiveWare’s first step was to carry out months of research, to create a high-quality Spanish core word vocabulary.

The next step was figuring out how to make the English and Spanish systems work in tandem. One of the biggest challenges was the vocabulary layout. Crescendo is based on motor planning. As much as possible, words are kept in the same place throughout the system to aid learning. We wanted to do this, where possible, across both languages. This is easier said than done. English is a Germanic language and Spanish is a Romance language. Their grammars are fundamentally different. We had to carefully plan the Spanish system so that it would work by itself as well as side by side with English. We maintained common points where possible, but still left room for each language’s individuality.

Designing the vocabulary was just one step. We also created bilingual children’s voices in collaboration with Acapela Group and Tobii Technology. Creating bilingual Text-to-Speech voices is even more complicated than monolingual ones. When choosing voice talents for our bilingual children’s voices, we needed to find the perfect balance of factors. We needed actors with perfect diction and enunciation in both languages. But we also wanted their accents to sound bilingual – with a hint of Spanish to their English. Both actors also needed to be mature enough to spend several days in a studio, while still having a young-sounding voice.

When it was all said and done, we had Emilio and Valeria, children’s voices that speak both Spanish and English. Using the same voice creates a more seamless experience than changing voices between languages.

What makes our system unique

The core word vocabulary and bilingual voices are essential. Yet, they also need a flexible system to really make them shine. Other AAC systems can be used in multiple languages. Proloquo2Go stands out, however, because it really reflects the nuances of bilingualism. For example, you can switch languages several times in a sentence, to say something like “Make sure que quede bien tasty” or “Has pagado los bills?”

Bilingual people often use a particular word in a certain language, even if they’re speaking in a different one. For example, American Spanish speakers may use “vacuum” instead of the Spanish word, even in a Spanish sentence. Proloquo2Go’s button customization takes care of this. You can change the name, language and voice of each individual button. This allows you to use language this way on your AAC system too.

The impact

In 2015 a free Proloquo2Go update included a bilingual Spanish-English vocabulary and voices. The bilingual voices, which are also available in Proloquo4Text and Pictello, have been downloaded over 11,000 times. Close to 5% of our Proloquo2Go users take advantage of the bilingual functionality.

We didn’t stop there. After English-Spanish, we also worked on a similar seamless English-French bilingual experience for Canadian users. More recently we added support for bilingual Dutch-French Belgian users.

For bilingual users, it makes a big difference to have access to an AAC system which is useful throughout the day. Access to languages and voices that they identify with greatly increases the chance that they will remain engaged and become successful communicators.

This innovation has had a wider impact as well. When bilingual AAC was purely theoretical, there was a sort of a Catch-22. Because there was no bilingual AAC, there couldn’t be any kind of bilingual AAC instruction. In taking the initiative, we created an opportunity to prove that bilingual AAC use was possible and valuable. This helped draw more attention to it in schools.

At AssistiveWare, we invest in these kinds of innovations because we want to deliver value to the AAC world and to our user’s lives. They help recognize the diversity of AAC users throughout the world. We want to make this technology more inclusive for everyone.