Eye movements while viewing narrated, captioned, and silent videos

Mon, 02 Dec 2013 19:00:00 -0500

Discovery: The highlighted research advances our understanding of saccadic strategies in dynamic scenes. Furthermore, it introduces narration as a powerful and useful tool for studying eye movement strategies, which may shed light on ways to optimize instructional material presented in video form. By examining the characteristics of narration (text or audio) that lead to higher retention and better understanding of visual information, we can improve not only educational videos, but also signage used on roadways and GPS systems. This work also has the potential to improve devices for the hard-of-hearing by suggesting that the position of the caption in the frame is important for being able to efficiently switch between narration and important parts of the scene. As new technologies emerge, such as gaze-controlled interfaces, it is important to be aware of preexisting biases that may weaken the usefulness of such devices in certain settings (e.g., when a lot of text is present).

Learning: This project required the use of complex computer vision algorithms as well as a basic understanding of the linguistic aspects of the project. Thus the project fostered interdisciplinary collaboration and learning, preparing the next generation of scientists to embrace and contribute to interdisciplinary projects in the future.