Automatically Capturing Source Code Context
Companion Site

Emily Hill, Lori Pollock, K. Vijay-Shanker
Natural Language Program Analysis (NLPA) Group
Department of Computer & Information Sciences
University of Delaware

Overview

As software systems continue to grow and evolve, locating code for maintenance and reuse tasks becomes increasingly difficult. We have developed a novel approach that provides automated support to the developer both in formulating queries and discriminating between relevant and irrelevant search results. Our contextual search approach automatically captures the context of query words in source code by extracting and generating natural language phrases from method and field signatures. These phrases naturally form a hierarchy that allows the developer to quickly identify relevant program elements by reducing the number of relevance judgments, while the natural language phrases help the developer to formulate effective queries.

We conducted an empirical evaluation of 22 developers that compares our contextual search approach to verb-direct object (V-DO), the most closely related search technique. Our results indicate that contextual search significantly outperforms verb-direct object in terms of effort and effectiveness.

 

Publications

"Automatically Capturing Source Code Context of NL-Queries for Software Maintenance and Reuse." Emily Hill, Lori Pollock, and K. Vijay-Shanker. International Conference on Software Engineering (ICSE '09), May 2009. [pdf]   [Presentation: Keynote]

 

Contextual Search Tool

You can test out our contextual search technique, or our implementation of V-DO.

So far the search tool has been used on 5 Java programs. If you would like to try out our contextual search technique on another, please e-mail a link to the source code and she will add it to the site.

 

Experiment Materials

In the experiment, we were interested in two comparisons: contextH with V-DO and contextH with contextL.

Experimental Unit &
Link to Instructions		First Tool,
Concern Block		 Second Tool,
Concern Block		Number of
Replications
1 V-DO, A contextH, B 3
2 V-DO, C contextH, D 2
3 contextL, B contextH, A 2
4 contextL, D contextH, C 4
5 contextH, A V-DO, D 2
6 contextH, C V-DO, B 2
7 contextH, D contextL, A 3
8 contextH, B contextL, C 3

At the end of the experiment, each subject was asked to fill out an exit survey.

Subject Concerns

A tarball of the concerns used in this study is available: concerns.tar.gz