RESNA Annual Conference - 2012


Raja Kushalnagar, Brian Trager and Karen Beiter
Information and Computing Studies, National Technical Institute for the Deaf Rochester Institute of Technology, Rochester, NY 14623


Deaf students often miss information in lectures. We used smart phones to record and stream the lecture slides in near-real-time and developed an application to enable students to rewind this video in real-time and to review information in real-time that they otherwise would completely miss. Students reported higher satisfaction and more accurate answers in using this approach, which can be used in virtually any gathering for real-time review.


Although modern classroom technology has contributed to the visual dispersion of the multiple information sources, technology can reduce these barriers, and benefit everyone, not just deaf participants. Marschark et al. [1] noted deaf students spent at least as much time watching the interpreter as compared with hearing students watching the instructor. Also, deaf students spent much less time watching course materials, e.g., slides. As a result, hearing students gain more information in class than deaf students [2]. Another disadvantage is that deaf students often rely solely on what they see to gather information. Instructors often do not allow enough time for students to see both the interpreter and the active lecture visual information [3]. Often, hearing students depend on auditory cues from the instructor to shift their attention from the instructor to the slides and vice-versa [4]. Unlike hearing students, deaf students cannot depend on auditory cues to decide when to switch from the overhead slides to the interpreter or vice-versa. They also report frustration in having the presenter and participants to accept the presence of the interpreter in highly visible spaces [5].

Figure 1: A deaf student’s view of the interpreter, white board and slides respectively. d

A final issue is that deaf students have to split attention between multiple visual streams in the classroom. For instance, teachers typically display overhead slides. The deaf student has to decide whether to read the information on the screen or to watch the interpreter translate the teacher’s explanation as shown in Figure 1. Regardless of the choice made, some information will be lost. They are unable to obtain equal access to classroom information, even when provided with access services that translate the audio into a visual representation, called mediated instruction. As a result even though deaf students can participate in almost any learning activity, they still lag behind hearing peers, and miss a substantial amount of information. Therefore, there is a need for new approaches to reduce information loss for deaf students.


Given the fact that federal law imposes responsibility on educational institutions for providing equal learning access to deaf and hard of hearing students [6], accessible technology research for higher education has focused on utilizing institutional and classroom infrastructure to provide equal access.

Figure 2: Accessible Viewing by the deaf student d

Deaf students tend to be thinly and evenly spread as deafness is a low incidence disability. Most deaf students often are the only student in their class [7]. For a single enrolled deaf student, it is cumbersome for institutional technical staff to support specific classroom or distance learning solutions. Also, deaf students typically attend lectures in multiple classrooms, yet throughout the semester, most classrooms in a given institution will not host a deaf student. Therefore, from both an institutional and student perspective, in terms of effort, availability and cost, the accessible technology should be centered on the student rather than the classroom. Therefore we chose to use consumer mobile devices to enhance classroom and lecture accessibility as shown in Figure 2.


We recorded and streamed a lecture using smart phones. The reason for picking smart phones is because they have built in video cameras and can capture and stream video. The popularity of smart phones has surged, and now constitutes a majority of new phone purchasers [8]. We chose to record and stream a two minute long lecture about the human visual system. We chose this specific kind of lecture because it is not very technical, yet is highly visual. We presented this lecture in two different ways to the students. In the first approach, the instructor simultaneously spoke while referring to information on the slides. We recorded and streamed the lecture slides, and also recorded the sign language interpreter close up.

Figure 3: Rewinding a video to the previous slide d

In the second approach, after each time the instructor talked about the slide information, we rewound the video to the previous slide and replayed the video segment as shown in Figure 3. We compared participant responses to the questions after they watched these lectures. We recruited 17 deaf participants ages 20-45 (9 female) for the study that typically requested accommodations in the classroom. Each participant watched both the simultaneous presentation and the simultaneous-rewind presentation. After viewing each presentation, each of the participants was asked to respond to two preference questions using a Likert scale from 1 to 5. Each participant was asked to rate the views they watched on the basis of the following questions: 1) “What is your rating for the simultaneous versus the sequential views?”, and 2) “Do you feel confident you can sit at the back of the classroom?” Students were also asked to answer two questions about the content of the lecture slides. The first question asked about information that was presented on the slides after the presenter stopped talking; the second question asked about information that was presented simultaneously with the slides.


Respondents gave similar ratings for both questions. For the question “What is your rating for the simultaneous presentation versus the sequential presentation?” there was a significant preference in favor in viewing the simultaneous with rewind presentation versus the simultaneous presentation: χ2 = 17.21, p < 0.001. For the question on whether students felt confident of sitting in the back of the classroom, again there was a significant preference difference of χ2 = 10.49, p < 0.005. For the content question about a slide in which the presenter simultaneously spoke, only 3 of 17 students answered it correctly. For the content question about a slide in which the presenter simultaneously spoke and then followed by a video rewind of the slide, 11 of 17 students answered this question correctly.


The real-time rewind feature significantly aided students in lecture content capture and recall. The students also liked the ability to of sitting anywhere in the classroom and still being able to view classroom information clearly. These study findings highlight the fact that students prioritize personal qualities such as independence and usability. Use of smart phones in optimal locations and reviewing of personal phones appears to improve visual access and seating options for all students, especially those students who are deaf and hard of hearing. Moreover the ability to capture, record and stream online presentations from preferred locations to students in the classroom enables everyone to share visually accessible of the classroom. This benefits deaf students who otherwise would not get more inclusive access in typical classrooms. The study suggests that deaf students prefer a record and rewind approach over a single recording approach.


  1.  M. Marschark, J. B. Pelz, C. Convertino, P. Sapere, M. E. Arndt, and R. Seewagen, “Classroom Interpreting and Visual Information Processing in Mainstream Education for Deaf Students: Live or Memorex(R)?,” American Educational Research Journal, vol. 42, no. 4, pp. 727-761, Jan. 2005.
  2.  M. Marschark, P. Sapere, C. Convertino, and J. Pelz, “Learning via direct and mediated instruction by deaf students.,” Journal of Deaf Studies and Deaf Education, vol. 13, no. 4, pp. 546-561, Jan. 2008.
  3. R. S. Kushalnagar and J.-F. Pâris, “Empowering Deaf Students with Multiple Views,” in Proceedings of the 2010 Technology and Deaf Education: Exploring Instructional and Access Technologies, 2010.
  4. M. Marschark, P. Sapere, C. Convertino, and R. Seewagen, “Access to postsecondary education through sign language interpreting.,” Journal of Deaf Studies and Deaf Education, vol. 10, no. 1, pp. 38-50, Jan. 2005.

  5. K. Woodcock, M. J. Rohan, and L. Campbell, “Equitable representation of deaf people in mainstream academia: Why not?,” Higher Education, vol. 53, no. 3, pp. 359–379, 2007.

  6. R. S. Kushalnagar, “Balancing Perceptually Disabled Consumers’ Rights Against Copyright Holders' Rights,” Journal of Disability Law, pp. 1-35, 2009.

  7. U. S. Office of Special Education Programs (OSEP), “Annual Report To Congress On The Implementation Of The Individuals With Disabilities Education Act (IDEA), 2007,” 2007. [Online]. Available: ndex.html.
  8. NielsenWire, “Android Soars, but iPhone Still Most Desired as Smartphones Grab 25% of U.S. Mobile Market,” NielsenWire, 2010. [Online]. Available: le/android-soars-but-iphone-still-most-desired- as-smartphones-grab-25-of-u-s-mobile-market/. [Accessed: 08-Feb-2010].