Problem Statement and Background

Communication is a medium for many different social interactions such as expressing
emotions, asking for assistance, and building relationships, which makes it a fundamental tool for intellectual and social growth. About 1 in 100 people experience severe communication impairments that require communication aids [1]. Augmentative and alternative communication (AAC) is any method which facilitates communication for individuals who experiences challenges expressing themselves using speech [2]. Individuals with disabilities due to stroke, Cerebral Palsy, developmental delays, or autism are primary users of these methods as they may be non-verbal, lacking fine motor control of upper extremities, or both [1].

There are various types of AAC methods, ranging from no-tech solutions such as facial expressions or gestures, low-tech solutions such as pen and paper or photo boards, to high-tech solutions such as tablets with eye gaze cameras [2]. The current state of high tech AAC devices for individuals with motor impairments makes it difficult to hold a conversation with an AAC user and for AAC users to express themselves effectively. The rate of communication using an AAC device is on average ten times slower than the average rate of verbal speech [3][4]. Additionally, AAC devices don’t come preprogrammed with every possible word a user may need, resulting in a large amount of time dedicated to manually programming new words into the device as the user builds their vocabulary. This is often done by a caregiver, such as a parent, Occupational Therapist (OT) or Speech Language Pathologist (SLP).

Smart devices can recognize where we are, who is important to us, and what we like to do, in order to provide us with tailored information. This information suggests the question: why can’t AAC devices do the same?

In summary, current AAC devices are lacking in two main domains: 1) the rate of communication and 2) available vocabulary. Thus, this project aims to improve the rate of communication for the user, and present dynamic and contextual vocabulary on the device based on information unique to them and their environment.

Methods, Approach, and Solutions Considered

Prior to solution conception, a list of existing products on the market and their respective shortcomings was gathered. These shortcomings and the pain-points of the users observed during user research were compared. The rate of communication was one of the primary shortcomings experienced by users. An important contributor to the low communication speeds were the input device employed by users. In order to synthesize a solution, a co-creation session was organized

with the Reka team and OTs from Kidsability. A rapid prototyping technique, commonly referred to as Crazy 8’s, was used to develop potential solutions [5]. Heat-mapping was then used on the drafted solutions to identify features that the brainstorming team found would satisfy some of the needs identified earlier.

Following this initial co-creation session, smart glasses were explored as a potential input method for the AAC system. Market research was performed on various smart glass solutions and North’s Focal glasses were identified as a suitable contender due to its sleek design which would ideally fit seamlessly with users lives. Professionals at North were contacted and the Focals were user-tested by the Reka team to identify fit. During discussions with a Software Display Lead at North, it was identified that the viewing range of the glasses were limited to 12 degrees and would not have enough space to display enough words on the device to make it useful for regular use.

Further exploration into the AAC space revealed that access to the necessary words in a timely fashion was at the root of the communication speed problem. Nested menus and folders and the need to establish a mental mapping for where words were within this folder structure created high cognitive overhead for users. As a result of this analysis, it was established that efficient word access in specific environmental contexts were lacking from current solutions. A list of useful features within the environmental and social context were brainstormed and the feasibility of implementing each of these features were considered. Through procedural feasibility analysis, certain features were deemed out of scope and thus eliminated from the solution space.

Description of Final Approach and Design

The team has created Reka, a device which will dynamically provide vocabulary to the AAC user’s software based on their physical location, communication partner, and distance to their recipient. Currently, in order to select a word, a user might have to travel through up to four nested menus. By dynamically suggesting relevant vocabulary to the user however, the number of selections required to vocalize a word could be minimized. For example, if a user were at school, Reka would recognize this and relay it to the user’s AAC software. The user’s software could then display all the words relevant for school on the front page of the interface, since the likelihood the user will need the words is much higher at school than if the user was at the park. This idea can be extended to recipients as well – if the user is talking with their mother for example, they might use different vocabulary than when talking with a friend. Reka would also grow with the user – for example, if a user commonly orders a burger at a specific restaurant, Reka will recognize this fact and display the word ‘burger’ near the top of the suggested word list next time the user visits that restaurant.

Figure 1 depicts the system architecture for Reka. The solution consists of a hardware module with the following sensors: GPS, Bluetooth and camera. The hardware module was deemed necessary as most tablets used currently by AAC users don’t have the necessary sensors that Reka leverages to provide useful suggestions. The sensors relay raw data to the iOS application and Reka Library which communicate with the server and database, and consequently send data to APIs that return contextual information about the user’s location, such as location tags using the Google Places API and menu items from restaurants using the Locu API. This facilitates the process of determining which words are most relevant to the user in a specific scenario and allows for them to be suggested on the user interface. The Reka application is currently supported to be run on Apple devices, such as an iPad. Next steps include extending this functionality to the Microsoft Surface which is another commonly used device for AAC.

Figure 2 shows a snapshot of the designed solution using a specific scenario to demonstrate functionality. The image on the left shows a default word board with no suggestions provided for the user. The middle image shows the suggestions that appear when a user goes to the movie theatre - Reka recognizes the user’s location using the GPS sensor onboard the hardware module. The image on the right shows the suggestions that appear when the user’s friend shows up at the movie theatre - Reka recognizes the user’s friend using Bluetooth beacon technology while continuing to provide vocabulary based on location information and user preferences.

Figure 3 shows a representation of the physical prototype that has been built. It consists of a tablet running the Reka software application and a hardware module encased in a 3D printed box that mounts to the iPad using a 3D printed bracket.

Outcomes and User Testing

The Reka team is currently partnered with Grand River Hospital in Waterloo and KidsAbility Waterloo. These partnerships have assisted the team in thoroughly understanding the problem space as well as validating that the solution approach will effectively meet user needs. Reka has been demonstrated to several OTs and SLPs from these organizations who have all responded to the prototype extremely well. These individuals have commented on how they can see their clients using Reka and directly benefiting from the value that it brings.

When presenting the solution concept to several AAC device users and their families, they all expressed interest in trying out the device once it was ready for use. Next steps include conducting pilot user testing on these AAC device users through our partnership connections and collecting feedback that will drive the next iteration of the prototype. Collecting real user data will help make the machine learning algorithm for word suggestions based on preferences more robust.

Cost to Produce and Expected Pricing

It is acknowledged that 1 in 5 families with children with disabilities fall under the low-income classification and thus it is a critical requirement for Reka to be approved by the Assistive Devices Program (ADP) [6]. As a part of the ADP program, the government will pay 75% of the product cost, making Reka affordable for users at only 25% of the cost.

Due to the underserved nature of the market, competitor products for current software AAC solutions that can be downloaded from the Google Play and Apple app stores are priced at approximately $975 [7].

The current cost to produce the hardware component of Reka is roughly $700. The team plans to make it available to customers at a price of $850 dollars. Additionally, the software component of Reka will be made available on the app store at $350. Users would be able to purchase both the hardware and software for only 25% of the cost with ADP subsidization, resulting in a cost of $300 for the complete package solution.

Future versions of Reka will phase out the hardware component and the software component will integrate directly with the users tablet granted that it has all the necessary sensors to enable full Reka functionality. At this point, the software will be available to users at a subsidized price of $87.50. Purchases may be facilitated through local hospitals and centres for child development with which Reka has partnership.

Significance

In North America alone, there are 2.2 million individuals who face communication impairments [1]. Additionally, the communication aid market is worth four billion dollars [2][7], and continues to grow, providing evidence for opportunity in this space. Reka provides value to the user by improving the rate of communication and automatically providing dynamic and contextual based vocabulary. This not only alleviates pains for the primary AAC device user, but reduces the amount of manual programming required by the caregiver.