Exploiting Information Graphics in a Digital Library

Overview:

The aim of this project is to investigate a method for retrieving information graphics (such as bar charts, line graphs, and grouped bar charts) from a digital library in response to user queries. Information retrieval research has focused on the text of documents, and their information graphics have largely been ignored despite the fact that they contain significant knowledge that is often not repeated in the document's text.

Our focus is on information graphics from popular media, which generally have a high-level message that they are intended to convey. We hypothesize that this intended message should play a significant role in a graph retrieval system, since it captures the high-level content of the graphic. Thus in ranking graphics for retrieval, we will investigate a methodology that takes into account the graphic's intended message, the graphic's caption, and relevant portions of the textual article that accompanies the graphic.

This material is based upon work supported by the National Science Foundation under grant number III-1016916. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The project is a collaborative effort with Millersville University under award number III-1016900

Personnel

University of Delaware

PI Sandra Carberry (Professor, Dept. of Computer & Information Sciences)
Peng Wu (graduate student, Dept. of Computer & Information Sciences) PhD completed Dec. 2012
Chris Boston (graduate student, Dept. of Computer & Information Sciences) MS completed June 2012
Zhuo Li (graduate student, Dept. of Computer & Information Sciences) current; 2012-2015, PhD thesis in preparation
Matt Stagitis (undergraduate student, Dept. of Computer & Information Sciences) BS (Honors thesis) June 2013
Kelly Peterson (undergraduate student, Dept. of Computer & Information Sciences) June 2013-May 2014
Natalie Lane (undergraduate student, Dept. of Computer & Information Sciences), 2013-2014
Mingzhi Yu (undergraduate student, Dept. of Computer & Information Sciences) current, 2014-2015
Natalie Corrado (undergraduate student, Dept. of Computer & Information Sciences), 2015

Millersville University

PI Stephanie Elzer (Associate Professor, Dept. of Computer Science)
Logan Peck (undergraduate student, Dept. of Computer Science) current; began 2010
Alyssa Zevallos (undergraduate student, Dept. of Computer Science) current; began 2011
Kenneth Acevedo (undergraduate student, Dept. of Computer Science) current; began 2012
Ricardo Myers (undergraduate student, Dept. of Computer Science) current; began 2012

Project Duration: 9/1/2010 to 8/31/2015

Broader Impacts: Besides its research contributions to digital libraries, in conjunction with our development of the SIGHT system for conveying information graphics via speech, the project will expand the knowledge resources available to individuals with sight impairments. In addition, the project is contributing to the education of both graduate and undergraduate students, one of whom is female and two of whom are first-generation college students.

Point of Contact: Sandra Carberry (carberry@cis.udel.edu)

Last updated: July, 2014