My capstone project involves the development of a dynamic multisensory interface to provide accessible biological aiagrams for blind and low vision students.

Users load an image from the web by entering a short code generated by the web-based diagram creation/upload interface.

An example of a loaded diagram. This application is currently designed for use with high-contrast diagrams.

According to the Census Bureau’s 2013 American Community Survey, there are approximately 543,893 children with significant visual impairment in the U.S. For these children much of the mainstream classroom learning materials are inaccessible, and adapted educational materials are required to meet these students’ needs. Producing appropriate accessible materials for STEM fields is particularly challenging due to the prevalence of visual diagrams in scientific texts. Tactile diagrams or physical models are commonly used to present this type of information, although these modalities are limited by their production time, cost, mobility and ease of use in mixed ability classrooms. The objective of this research project is to develop a haptic-audio interface compatible with a commercially available device that can be used to present educational diagrams to students studying life sciences. The proposed interface would serve as a multi-sensory educational tool for classroom and independent use.

The proposed system will be implemented on a widely available commercial device that features haptic capabilities. This interface will use a combination of audio elements and haptics (vibration) to present diagrams and information. Haptics will be used to establish the location and shape of on-screen elements, and audio output will provide labels and additional information. An increase screen size is preferable, since the resolution of touch is extremely limited and diagrams must be significantly enlarged to be tactilely decipherable. For development, the iPhone 6 Plus will be used. This device was chosen due to its general availability, its large screen and haptic motor. Additionally, while Android phones are often cheaper and easier to develop for, the iPhone was chosen because it is the device most widely supported by the blind community. The larger screen real estate provided by the iPhone 6 Plus will allow for complex diagrams to be displayed without the need for scrolling, which has been shown to disorient non-visual users.

An early version of this mobile application has been produced, and a web-based diagram upload and create interface has been developed. The application is currently capable of loading a diagram from the web and provided audio and haptic feedback when different areas of the diagram are touched.

The online diagram creation interface can be access here (password is just the letter “a” to prevent unsolicited uploads during the testing phase). Please note that this is an early prototype focusing on establishing functionality, and will be reformatted in the coming months.