RobustFigureSegmentation
Robust Segmentation of Biomedical Figures Toward an Image-based Document Retrieval
Abstract
Figures play an important role in illustrating concepts, methodology and results in biomedical literature. However, figures in biomedical literature are often composed of multiple subfigures (panels), which may illustrate diverse methodologies or results. Robust and accurate panel partitioning is crucial to support article categorization based on methods or experimental results and to provide the evidence source for derived assertions. But, it is a challenging task. In this paper, we present a comprehensive framework for harvesting multimodal panels in biomedical literature, and demonstrate its application to protein-protein interaction (PPI)-related literature as a use case. A unique feature of our solution is that we combine pixel-level representations of images with figure captions. Our approach first analyzes figure captions to identify the label style used to mark panels. We then use pixel-level representations to partition a figure into a set of bounding boxes of connected components. We also perform a lexical analysis on the text within the figure to locate panel labels that match the caption analysis results. Finally, we estimate the optimal panel layout and use the layout to partition the figure. We tested our system on a dataset provided by the Molecular INTeraction database (MINT), and show that our approach surpasses pure captionbased and pure image-based approaches, achieving a 96.64% precision.
Pipeline
Download
 |
 | |
|
Paper |
Talk |
Bibtex |
Software: The software is available for testing at our image mining webpage
Source Code: Source code is available upon request for research