From Voiceme To Autonome To “Ok Google” – An Eadl Case Study Narrative

Seth R. Hills, BSME, CPO

Hunter-Holmes McGuire VAMC, Assistive Technology Department (Richmond, VA)


Among dependent populations, Veterans are some of the most vulnerable, and Veterans with spinal cord injury or other severely disabling conditions are even more so [1].  With the loss of independence, quality of life suffers. Of the many proven methods to improve quality of life, one of the more significant is the provision and use of electronic aids to daily living (EADLs). Not only are they beneficial for the patient by increasing independence, but also for the caregiver in freeing up their time and responsibilities. as well as the societal benefits of lower costs for care. [2]

This case study chronicles the prescription and use of several different forms of EADLs for a client at the McGuire VA Medical Center.  Certainly, clinicians will periodically be presented with challenging cases.  This is an account of a care team, a Veteran who has a desire to stay current and connected, the various solutions attempted to provide access to his environment, and the obstacles encountered along the way.

The McGuire VAMC has acute, outpatient, inpatient, surgical, hospice, and long-term care centers.  This Veteran has been beating the odds since his admission to the hospital.  He suffers from various maladies that have left him weak, and unable to move or care for himself.  He relies on 24hour nursing care for everything.  He has great support from his family, and is often seen being wheeled through the canteen and exchange with a family member. 

The patient’s goals are to be able to call his family, control the television, listen to music, and stay current with the news. He also requires a way to access caregiver assistance.

The hospital room he lives in is equipped with a television, standard nurse call, and hospital bed.  He does not own a cellphone; however, his room does have a landline phone. Movement is limited to some head and gross right upper extremity (UE) movement with limited range of motion (ROM).  His right UE has some glenohumeral flexion, the ability to abduct the upper arm, and to squeeze his hand.  However, his hand grip is rather contracted and requires splinting to prevent further contracture. His speech is somewhat garbled with weak phonation and projection that varies throughout the day, and from day to day.

Initial treatment goals were to provide adaptive access to phone (make and receive calls), television (channel up/down, volume up/down, power), and nurse call.

The biggest challenges in this quest have been lack of consistency in the patient’s abilities, and caregiver technology fatigue.


In May 2015, the assistive technology team at McGuire VAMC was consulted to provide adaptive nurse call access. A rehab engineer outfitted his bed with an EZcall switch next to his head, however, false and inadvertent activations led to the signal being ignored as it got pushed too often.  Results were similar with a sipnpuff nurse call switch and the switch or straw was often found on the floor in the patient’s room with no reason given for it being discarded other than indications from caregivers that he was activating it too often for no reason. 

VoiceMe ECU

The next attempt was to utilize the VoiceMe environmental control unit (ECU) for TV (Channel Up/Down) and nurse call control.  We were able to successfully train his voice to the device, but he had some trouble with command structure on his own. He was able to move between channels with some help, and he was able to remember the nurse call commands in two instances 3 hours apart.  On a follow up, the patient demonstrated that he remembered the correct commands to use the VoiceMe EADL, but indicated the system was not working. The patient was confused by false positives from the device and inadvertently disabled the voice activation keyword.

AutonoMe ECU

Figure 1: http://asi-autonome.com/environmental-control-units-ecu/autonome-hospital/
The VoiceMe approach was abandoned due to the potential for unintended self-sabotage. The AutonoMe ECU with a simple four button pageset (TV Power, Channel Up / Down, and Nurse call) was setup on a floor stand next to his bed to replace the VoiceMe [The Quartet ECU was not considered an option at this time due to the complexity of voice training, command structure, and lack of visual feedback]. It was selected in hopes that a more intuitive interface and responsive voice control would compensate for the failings of the VoiceMe ECU. The patient was able to use voice commands, but inconsistently.  While he was not able to mistakenly disable the new system, he was unable to maintain intelligible pronunciation.  His voice is often garbled and difficult to understand which makes voice recognition challenging.  

An external microphone was connected to the AutonoMe to determine if the voice recognition would work better with the microphone being positioned closer to the patient’s mouth.  The microphone was positioned at the bedside, and the patient demonstrated using all 4 commands successfully. The external microphone was an improvement. 

Despite having visual feedback as design criteria, the patient’s vision is somewhat limited, and he could not read the command words to help cue his memory.  Since the patient was not relying on the display for cueing or feedback it was moved off to the side of the bed to improve caregiver access. 

Voice recognition continued to be a challenge and tended to increase the patient’s frustration when it did not function as he expected it to.  We installed an auxiliary switch based nurse call for hand or head usage as an interim nurse call method for times when the EADL-voice recognition system was not functioning adequately. 

Figure 2: Quartet Simplicity, http://www.qtiusa.com/
The patient expressed interest in making phone calls to family members, this required some creative connections to the landline in his room- a first attempt was made with the Sero Phone (ablenet, sero!), then a Konftel conference phone with IR remote (Konftel) was successfully integrated. Unfortunately, the whole system was removed not too long after due to problems with IR transmission to the phone and tv, caregiver technology fatigue, and excessive nurse call activations.

At this point it became clear that any system setup for the patient needed to be fairly simple for nursing staff to deal with (should ideally not be mounted to the bed as staff are not setting the equipment up after bed transfers). Voice control is very intermittent, and the microphone is often out of reach.  The patient needs another solution that is simpler to use and more reliable. 

Quartet ECU

After consulting with colleagues at a national Assistive Technology Conference, the suggestion was made to trial the Quartet (Figure 2) with single switch scanning.  We setup the quartet with a sipnpuff straw to activate the single switch scanning.  Options were refined to nurse call, television control, and phone calls.  After training, repeat coaching, and follow-ups, the patient was able to make some phone calls, change channels, and activate the nurse call.  However, he often forgot the order of operations for making phone calls, and attempted to incorrectly make calls from the speed dial list.  He would also leave the phone line “off the hook” and neither the staff nor the patient recognized that fact until the rehab engineer came and “hung up” the line.  The Quartet was left as the final solution for some time until it became clear that it too was no longer being utilized by the patient, and not supported by nursing staff.

Nurse call light

Figure 3: Nurse Call Light -Hunter Holmes McGuire VAMC, Richmond, VA

During the patient’s use of the Quartet ECU, he was still activating the nurse call excessively, so we hypothesized that perhaps the fact that he cannot see whether the nurse call has been activated or not is why he keeps activating it.  To test the hypothesis, and in an attempt to relieve some caregiver burden, we designed and built a simple nurse call light indicator (figure 3) to light up whenever the nurse call is activated.  The only problem we did not consider is that this was just one more item mounted to the bed that gets in the way of transfers, patient care, and adds another burden to the caregivers!  A future version will have to be either floor mounted or placed somewhere that will not interfere with bedside care and transfers. 

Consumer smart personal assistant

Figure 4: https://www.ablenetinc.com/technology/ speech-generating-devices/littlemack
A family member to the patient requested we trial the Amazon Echo, so an Amazon Echo Dot was set up and worked some of the time.  However, due to the patient’s inconsistent voice, he was often unable to articulate the command word “Alexa”.  The same problem was encountered with the Google Home.  In order to bypass the command phrase, we tested whether the echo would respond to the patient’s speech once the command button was activated or if someone else activated the command phrase.  Indeed, it did, so we recorded the “Alexa” command phrase on a Little Mack (figure 4) voice recorder switch, with some success. However, we were back to the problem we have had all along of the patient perseverating on the switch.


Problems along the way

 Throughout our attempts, the patient was moved several times within the hospital, which changed the requirements of how to interface to the television, phone, and nurse call.  This is a challenge when attempting to program IR codes to non-traditional television and phone systems.

Extended setup and training sessions occasionally resulted in the patient becoming frustrated and uncooperative.

The nursing staff raised privacy concerns given that the Echo has the ability to record audio surreptitiously, and stemming from the patient’s comments that he was going to turn in staff for no apparent reason.  In reality, this is a potential issue with any device capable of recording audio (cellphones, tablets, etc.), and, if there is a concern, a dialogue needs to be opened with the patient to discuss privacy concerns.

Communication with care team

Nursing staff were trained on each new system, and asked to share that information with the care team.  On occasions where problems occurred with equipment being taken down and disconnected, the Rehab engineer met with the nurse manager to find solutions.

It is important to keep the care team abreast of developments, and to ensure them that EADL’s for the patient are also in their best interests to free up caregiver time and resources.  Sometimes low-tech or no-tech is good enough: On several occasions, Nursing staff pointed out that the patient was able to get their attention by calling out “nurse” if he needed assistance.

Psychologists work closely with treatment teams to optimize the patient’s functional capabilities, and are key players when dealing with cases where cognition is an obstacle.  It is advisable to seek their input when considering the probability of success with proposed interventions at any given time.

Future considerations

The bottom line is that unless a patient has stable cognition, consistent switching ability, or phonation, and there is adequate Buy-in from Nursing/Caregiver/Family/Therapy, it is likely that any attempted intervention will fail.

Going forward, a goal of our team is to develop a switching mechanism that adapts to the patient.  Thus, eliminating a lot of the false positives.  We also would like to hone in on a grip switch that the patient can hold and activate as needed since this is one of his remaining capabilities.  It would be a simple task to have caregivers place a switch in the patient’s hand, and would not overly clutter the bedside.  It is also possible that we could program an Alexa skill to remind the patient they just called the nurse, and do they really need to call the nurse right now?

The consumer solutions are great for intuitive control, there is no need to memorize a hierarchical command structure, or key phrases (aside from the initial command words).  However, finding a way for our patient to be able to consistently activate a switch without perseverating on it is our main issue, and that will likely require a closer working relationship with the treating psychologist. 


Journal articles: 

[1] Shivayogi, P. (2013). “Vulnerable Population and Methods for Their Safeguard.”. Perspectives in Clinical Research, 53-57.

[2] Ashley Craig, Y. T. (2005). The efficacy and benefits of environmental control systems for the severely disabled. Med Sci Monit; 11(1), RA32-39.