CISC 879 Software Testing
Research Proposal Experience
Spring 2004
Learning
Objectives: Graduate students completing this project
should be able to independently and successfully:
(1) Perform a thorough bibliographical search
of a particular topic in computer science.
(2) Compare and contrast and summarize a set of
papers/research contributions on a particular topic.
(3) Write an informative, but concise and
critical survey of the state-of-the-art in a particular topic in computer
science.
(4) Brainstorm new ideas on a particular topic,
based on previous work in the area.
(5) Develop an evaluation plan which would
successfully determine the effectiveness of new ideas.
(6) Write a well-developed, concise research
proposal.
(7) Learn to work together as a research team
in gathering information, discussing existing approaches to gain understanding,
critical thinking, and formulating new ideas.
The
Divided Project: The following chart lays out the various
subtasks and the timeline of deliverables to ensure that each student makes the
best use of the first part of the semester to hit every aspect of the first stages
of research up through a research proposal.
Some deliverables are done as a group effort while others are to be done
individually.
Potential
Topics: Based on current
proceedings and lectures in various conferences over the past few years, I have
developed the section of the course’s reading list, called Testing in Different
Paradigms. While you may choose a topic
not on this list, I prefer that you choose one of these topics, as they were
included due to their “hot topic” status and the relevance to this course. Only one group will focus on a given topic
from the list. If more than one group
selects the same topic, we will flip a coin.
If you are currently doing research in one
of these areas, and are already past the stage of creating a literature survey,
then you should choose another topic. If
you have just started research in one of these topics, but have not done a
literature survey, you may choose a topic somewhat closely related to your
topic or a completely unrelated topic.
If you are not sure which category you belong in here, you should talk
to me.
The rest of this document explains each
deliverable in detail. For each
deliverable, submit a printed copy or email a pdf
file as an attachment.
2/12: Topic and Group identification
2/26: Deadline 1: ref list
3/4: Deadline 2: lit review outline
3/16: Deadline 3: background and lit review
4/6: Deadline 4: proposed idea writeup
4/29: Deadline 5: first complete draft
5/13: Deadline 6: final complete draft
Finals week: Deadline 7: Oral presentation
Deadline
1 (output from a first draft bibtex file and
paragraph on topic statement): The
goal for this deadline is to identify the relevant papers for your topic; that
is, search for all papers relevant to your topic; not reading the papers, but
determining relevance based on title and abstract. You should search in digital libraries (ACM,
IEEE,...), recent conferences and workshops that cover
your topic, and then use the bibliographies of recent papers to identify
earlier relevant papers. The deliverable
is a nicely formatted reference list, created by using bibtex and latex, and a
sentence describing the overall topic you are investigating. (i.e.,
testing of web applications). This is a group effort, with one deliverable
for the whole group.
Deadline
2 (outline of literature review): The
goal for this deliverable is to understand the timeline, overall contributions,
relative merits and limitations of the work embodied in the state-of-the-art in
your topic. You need to read only the
abstract, introduction, related work, and conclusions sections of each
paper. Do this reading in chronological
order (or reverse chronological order) of paper publication dates to obtain
some sense of how the research has evolved over the years. Then, develop an outline where you have
grouped the papers focusing on very similar problems, and then have a section
of the outline for each paper. For each
paper, be sure to include a subpart for problem addressed, contribution,
findings of any evaluation of the contribution, and limitations. Thus, your outline should look like:
This
outline should be in plain text so it is easy to insert into a latex file to
start writing. This is a group effort and has only one deliverable per group.
Deadline 3 (First Draft of background
and literature survey):
The goal for this deadline is what would typically be found in the Background
and Related Work sections of a paper or proposal. You should read some Background and Related
Work sections of papers to see how they are written. That is, a good literature survey does not
just write a separate paragraph on every paper written in the field in any
order you want.
*
A good literature survey starts with a background section that familiarizes a
computer science reader to the basic topic area, such as testing web
applications – what constitutes a web application, how are they characterized,
examples from real life,...
*A
good literature survey tries to group papers addressing similar problems and
discuss and compare them together.
*A
good literature survey also presents the papers typically in some chronological
order within each problem identified.
*The
most relevant papers to what you want to focus on are presented first, and then
other papers that deal with problems related, but not so relevant, are
discussed very briefly and sometimes only cited as a group with a single
sentence. So, paragraphs are ordered
from most relevant work to least relevant work to your chosen problem of
interest.
*
A good literature survey will do the following for the most relevant papers:
describe the overall goals/contributions of the
paper, general approach and unique characteristics of their approach, then end
with restrictions/limitations of that research.
What didn’t they address? Did they implement it and evaluate it?
A
related work section should be no more than 2 pages in the double column format
you are given in the latex file. Most
are more like 2 pages maximum. Each person in the group should write their own
background and related work sections.
Again, you can discuss what you think are the most relevant and least
relevant papers and agree on them as a group, but each member needs to learn
how to write these surveys by writing them on their own. It takes time and sometimes rewriting to put
it together as above. The deliverable is
a nicely formatted document with your Background section and your Related Work
section (done individually), and reference list. You should use latex for
formatting.
Deadline 4 (Writeup of brainstorming session): The goal for this deadline is to get the
group to:
* review and discuss the key
restrictions and limitations of existing research in the topic area as a whole
*
write a few paragraphs summarizing those limitations
*
discuss the key open issues/problems left unaddressed by existing work
*
write a few paragraphs summarizing those key open issues
*
brainstorm about possible approaches to attempt to
address some subset of those key issues: applying some technique used in
solving other problems in other domains, trying to develop new algorithms or
alternate program representations,...
*
write a few paragraphs outlining the possible approaches you could propose for
addressing these problems
*
think about how would you would evaluate the success of your proposed approach:
what are the appropriate questions to ask for evaluating the approach, what
could be implemented, what metrics could be measured and experiments could be
performed to judge whether your approach indeed addresses the problems or
improves on previous approaches in some way (space, execution time, program
analysis time,...)
The deliverable for this
deadline is a nicely formatted document that includes:
* your
proposal title
* a section
called Current Limitations and Key Open Issues
* a section
called Proposed Research, which consists of two subsections:
* Proposed Novel Approach(es)
* Evaluation Plans
This is a group deliverable. You
need only hand in one deliverable per group. These will be
incorporated into each member’s own individual proposal file for the next
deadline.
Deadline 5 (Draft of
proposal): The goal for
this deadline is a complete first draft of a research proposal, created by
merging, integrating, and smoothing out your previous writings. This proposal should include the following
components:
* Proposal Title
* abstract
(probably rewritten now to reflect what you are addressing and your overall
approach) (300 words or less)
* introduction
(motivate the general topic and why it is worth studying, present overview of
what goals your research has) (1 page)
* background and state-of-the-art (give
the computer science reader some background on your general topic area, as if
they knew nothing about embedded systems or security,... and include your
related work section as part of this background now (getting rid of the related
work section title); end with a subsection called Limitations/unaddressed
problems if appropriate.) (2-3 pages). So, this section includes your literature
review (background and related work) and your current limitations writeup from your brainstorming session. Be sure to
incorporate suggestions from the instructor on the background and related work
first draft.
* challenges
and goals (enumerate the challenges and goals that you are focusing on (1 page).
this can include the open issues that you wrote up for the brainstorming
session.
* proposed
research (present your proposed approach(es), subsectioning as appropriate.,
including some steps you would follow to do your research (1 page)
* evaluation
plan (enumerate the steps you will follow to do your research)(no more than 1/2 page)
* summary
of foreseen contributions - how will
your work help society, summarize the contributions it would make if you indeed
followed through with this research (1 paragraph)
* reference
list
This draft is written
individually by each member of the group, separately. The
deliverable is a nicely formatted first draft of a research proposal, one per
group member.
Deadline 6 (Final version of
proposal): The goal for
this deadline is a rewritten, complete version of your research proposal,
focusing on the suggestions for improvement by me and your own observed
potential improvements. It has the same
format as the previous deadline, and is also done individually.