Home Page

The amount of information available electronically has increased dramatically over the past decade. The challenge is to develop techniques for providing effective access so that all individuals can benefit from these resources and so that information is readily available when needed. Unfortunately, many knowledge sources are provided in a single format and thus are not accessible to everyone. For example, individuals with impaired eyesight have limited access to graphical displays, thus preventing them from fully utilizing available information resources. Although research has investigated alternative modes of presentation of graphical information for people who have visual impairments, such as converting a video image to a soundscape or by printing hard-copy that the user can decipher via touch, their focus is on rendering graphical elements in an alternative medium and they have serious limitations. For example, it would be extremely difficult for a user to compare two related lines on a line graph via a soundscape. The underlying hypothesis of our work is that alternative access to what the graphic looks like is not enough --- the user should be provided with the message and knowledge that one would gain from viewing the graphic in order to enable effective and efficient use of this information resource.

We have developed an interactive natural language system, Interactive_SIGHT (Summarizing Information GrapHics Textually), whose long term goal is to enable visually impaired users to gain access to the content of graphics that appear in popular media. Once launched by a keystroke combination, Interactive_SIGHT first provides the user with a brief summary of a graphic with the inferred underlying message as the core content, and then responds to follow-up requests for further information from the graphic. The current system, which is implemented as a browser extension, works on simple bar charts but the work to extend the system to other kinds of information graphics (e.g., line graphs and grouped bar charts) is continuing.