Dynamic Text In Visual Scene Displays: Supporting Word Reading In A Preschooler With ASD

Kelsey Mandak1, Janice Light1, David Mcnaughton1

1Pennsylvania State University



Autism spectrum disorder (ASD) is a developmental disorder that is often associated with complex communication needs (CCN). A large portion of children with ASD have communication impairments so severe that they cannot meet their daily communication needs with speech alone. [1] These children are considered potential candidates for augmentative and alternative communication (AAC), which can include signs and gestures, communication boards, speech-generating devices (SGDs), or mobile technologies. [2]

While learning how to communicate effectively is significantly important, the ability to read and write is also an essential skill for individuals with ASD to develop in order to participate fully in life.  It is a skill that is increasingly recognized as critical for individuals with developmental disabilities. [3] For individuals with ASD with CCN, literacy plays an even bigger role as it is the gateway to communication.  Literacy skills support the development of communicative competence in these individuals and provide the easiest access to generative language. [4] Despite this importance, the lack of literacy skills in individuals with CCN has been repeatedly documented.  Current research demonstrates that up to 90% of individuals with CCN enter adulthood without functional literacy skills. [5] This heightens the risk of poor outcomes for these individuals in education, employment, and communication with others.

There is an urgent need to reduce poor outcomes for individuals with CCN and to find a way for young communicators to transition to literacy in an effective and efficient manner.  Currently, beginning communicators typically use graphic AAC symbols to communicate, either through the use of visual scene displays (VSDs) or grid displays. VSDs are electronic photographs of meaningful events in an individual’s life with vocabulary programmed under certain parts of the photo (“hot spots”).  When a child touches a hot spot, the device produces programmed spoken output.  Typically, there is no text associated with VSDs and thus no way for a child to transition to literacy.  On the contrary, traditional grid displays of graphic symbols typically have orthographic text associated with the symbols.  The text is usually written above the picture symbols in a small, static, and permanent fashion.  Although orthographic text is present in these displays, there is no evidence to suggest that children acquire these sight words when exposed to this text. [5]

Despite the importance of literacy acquisition, there are currently no evidence-based AAC apps that can effectively support the transition from communicating through graphic symbols to the use of orthographic text. The purpose of this study was to investigate the effects of a new software feature to support the transition to literacy in a pre-literate preschooler with ASD. The software feature (termed “Transition to Literacy” or the “T2L” feature) provides dynamic presentation of text and speech output when a hot spot is activated (for a demonstration of the feature, visit: https://rerc-aac.psu.edu/research/r2-investigating-aac-technologies-to-support-the-transition-from-graphic-symbols-to-literacy/). Dynamic text differs from static text in that it uses motion to grab the attention of the learner. When a child touches an image or storybook page with the dynamic text feature, the associated written words appear dynamically on the screen, in close proximity to the picture selected, pause for 3 seconds, and then fade away.

The development of the T2L feature was based on scientific principles from the field of visual cognitive science and instructional design. The feature was designed to: (a) actively match text with its referent (i.e., pairing speech output with presentation of text; [6]) (b) use motion to draw attention of the learner (i.e., dynamic presentation of text; [7]) and (c) focus on individualized and motivating vocabulary (i.e., responding to the interests of learners; [4]).

Early studies investigating the T2L feature have provided positive results across a range of ages and disabilities. [8-10] The purpose of this study was to investigate the effects of the T2L feature on the acquisition of sight words by a pre-literate preschooler with ASD during shared book reading experiences. Specifically, the research question was: What is the effect of the T2L feature on the acquisition, maintenance, and generalization of single word reading by a pre-literate preschooler with ASD during shared book reading experiences? By dynamically displaying text and actively pairing text with its referent, it is hypothesized that the preschooler with ASD will successfully acquire the target sight words.


This pilot study included one 4-year old male, Matthew (pseudonym), who was diagnosed with ASD.  Matthew was pre-literate and attended a LEAP preschool (i.e. each classroom had 4 children with ASD and 8 children who are typically developing).  A single-subject design was used with four distinct phases: baseline, intervention, generalization, and maintenance. 


The stimuli in this study were 10 words from the book “Brown Bear, Brown Bear, What do you see?”, which was considered a motivating and age-appropriate book for preschoolers. All stimuli were the same across all phases of the study and included: bear, cat, bird, dog, duck, sheep, horse, fish, frog, and teacher. The words ranged from 4 (e.g., duck) to 7 letters (e.g., teacher), were all imageable, and were not already in the participant’s sight word vocabulary.

The stimuli were presented on an app called EasyVSD, which was designed by Invotek and housed on a Samsung GALAXY Tab Pro Tablet. The VSD app displayed the individual pages of the book, which each displayed one of the stimuli words.



Baseline measures (probes) were taken prior to the start of the intervention phase in order to establish the participant’s current level of performance on the dependent variable (correct identification of sight words). For the probes, the participant was presented with four images from “Brown bear, brown bear” representing the targeted sight words and one of the target sight words in text form. There was a total of 10 probes.  A minimum of five baseline points was required before the intervention phase began.


Once the baseline phase was complete, intervention began. Each intervention session started with probes (i.e., same procedures as baseline) followed by exposure to the T2L VSD app during a storybook reading.  The VSD software was housed on a Samsung GALAXY Tab Pro Tablet.  The software displayed the individual pages of “Brown bear, brown bear” in electronic form. Each individual animal page included dynamic text and speech output when selected. No other instruction was provided during intervention.


Generalization measures were taken during baseline, prior to the start of the intervention phase in order to establish the participant’s current level of performance on the dependent variable with different representations (i.e., real photographs of the target words instead of images from the book) of the targeted words. Generalization measures were additionally taken after intervention, once the participant met criterion for all sight words. All generalization probes followed the same procedures as the baseline probes (i.e., “Read the word, give me the picture that goes with the word”).


Long-term maintenance measures (probes) were taken eight weeks following the completion of intervention. The maintenance probes followed the same procedures as the baseline and intervention probes.


e 1 shows the percentage of sight words read correctly by Matthew during each phase of the study (that is, baseline, intervention, maintenance, and generalization).   The graph shows that during baseline, Matthew ranged from 0% to 50% over 5 sessions and averaged 26% accuracy.  Matthew started intervention following the 5th baseline session.  The graph shows Matthew’s accuracy during each intervention session.  Matthew required 26 intervention sessions to acquire all 10 sight words, and demonstrated an overall gain of 74% from baseline. Following the last intervention session, Matthew achieved 100% accuracy with the generalization probes.   Eight weeks after the last intervention session, Matthew achieved 100% accuracy during both maintenance and generalization probes.
Figure 1. Percentage of sight words read correctly, out of 10 trials.
A stable baseline was achieved by Matthew over 5 probe sessions, with an average of 26% accuracy (range: 0% to 50%; see Figure 1). Matthew met criterion for the first pair of words (i.e., bear and cat) after 13 sessions, which resulted in 65 exposures per word (3 minutes and 15 seconds of total exposure time; see Table 1). After the initial pair of words were acquired, Matthew acquired the remaining words with less total exposures. Matthew acquired all 10 target words after 26 intervention sessions (i.e., 2 hours and 10 minutes of intervention). Matthew demonstrated an overall gain of +74% and Tau-U score of 0.63 (i.e., large effect).

Generalization probes were completed during baseline, following the last intervention session, and 8-weeks post intervention. Matthew achieved an average of 25% accuracy during baseline (range: 20% to 30%), 100% accuracy following the last intervention session, and 100% accuracy 8-weeks post intervention. Matthew achieved 100% accuracy during the maintenance probes which were taken 8-weeks following the last intervention session.

Matthew’s performance provides preliminary evidence that a software feature for AAC apps, including the dynamic presentation of text paired with graphics and speech output, positively impacts the single-word reading of pre-literature preschoolers with ASD.

Table 1.  Intervention Exposure Data for Matthew

Length of Exposure to Dynamic Text and Number of Sessions
Word Set Length of Exposure # of Sessions
bear, cat 195s 13
bird, dog 45s 3
duck, sheep 30s 2
horse, fish 60s 4
frog, teacher 60s 4
Total 390s 26


This study provides a new and unique way to complement current AAC devices and literacy instruction in children with ASD who are pre-literate.  Increasing literacy skills in these children may lead to positive outcomes long-term in terms of participation in education and society. It is important to also note that this improvement in technology is not intended to replace any formal literacy instruction, but rather complement current communication apps by infusing literacy into places where it is currently absent.


 [1]Matson, J., Kozlowski, A., & Matson, M. (2012). Speech deficits in persons with autism: Etiology and symptom presentation. Research in Autism Spectrum Disorders, 6, 573–577.

[2] Beukelman, D., & Mirenda, P. (2013). Augmentative and alternative communication: Supporting children and adults with complex communication needs (4th ed.). Baltimore, MD: Brookes.

[3] Fossett, B., Smith, V., & Mirenda, P. (2002). Facilitating oral language and literacy development during general education activities. In D. Ryndak & S. Alper (Eds.), Curriculum and instruction for students with significant disabilities in inclusive settings (2nd ed., pp. 173– 205). Boston, MA: Allyn and Bacon.

[4] Light, J., & McNaughton, D. (2013). Literacy intervention for individuals with complex communication needs. In D.R. Beukelman and P. Mirenda (Eds.), Augmentative and alternative communication: Supporting children and adults with complex communication needs, 4th edition (pp. 309-351). Baltimore: Paul H. Brookes Publishing Co.

[5] Foley, B. E., & Wolter, J. (2010). Literacy intervention for transition-aged youth: What is and what could be. In D. McNaughton & D. Beukelman (Eds.), Transition strategies for adolescents and young adults who use AAC (pp. 35–68). Baltimore, MD: Brookes.

[6] Browder, D. M., & Xin, Y. P. (1998). A meta-analysis and review of sight word research and its implications for teaching functional reading to individuals with moderate and severe disabilities. Journal of Special Education, 32, 130-153.

[7] Wilkinson, K., & Jagaroo, V. (2004). Contributions of principles of visual cognitive science to AAC system display design. Augmentative and Alternative Communication, 20, 123–136.

[8] Boyle, S., McCoy, A., McNaughton, D., Light, J. (2017). Using digital texts in interactive reading activities for children with language delays and disorders: A review of the research literature and pilot study. Seminars in Speech and Language, 38(4), 263–275.

[9] Caron, J., Holyfield, C., Knudston, C., Light, J., & McNaughton, D. (2016, November). Grid displays to literacy: effect of dynamic text on word reading for individuals with ASD. Presentation at the American Speech-Language Hearing Association Convention, Philadelphia, Pa.

[10] Holyfield, C., Pope, L., Light, J., McNaughton, D., & Drager, K. (2016, November). Visual scene displays (VSDs) with dynamic text: supporting early reading in adults with Intellectual Disabilities (IDD). Presentation at the American Speech-Language Hearing Association Convention, Philadelphia, Pa.