Whole
Test Suite Generation
ABSTRACT:
Not all bugs lead to program crashes, and not always
is there a formal specification to check the correctness of a software test’s
outcome. A common scenario in software testing is therefore that test data are generated,
and a tester manually adds test oracles. As this is a difficult task, it is
important to produce small yet representative test sets, and this
representativeness is typically measured using code coverage. There is,
however, a fundamental problem with the common approach of targeting one
coverage goal at a time: Coverage goals are not independent, not equally
difficult, and sometimes infeasible—the result of test generation is therefore
dependent on the order of coverage goals and how many of them are feasible. To
overcome this problem, we propose a novel paradigm in which whole test suites
are evolved with the aim of covering all coverage goals at the same time while
keeping the total size as small as possible. This approach has several
advantages, as for example, its effectiveness is not affected by the number of
infeasible targets in the code. We have implemented this novel approach in the
EVOSUITE tool, and compared it to the common approach of addressing one goal at
a time. Evaluated on open source libraries and an industrial case study for a
total of 1,741 classes, we show that EVOSUITE achieved up to 188 times the
branch coverage of a traditional approach targeting single branches, with up
to62 percent smaller test suites.
EXISTING SYSTEM:
Software testing is an essential
component of any successful software development process. A software test
consists of an input that executes the program and a definition of the expected
outcome. Many techniques to automatically produce inputs have been proposed
over the years, and today are able to produce test suites with high code
coverage. We produce test inputs, and then a human tester needs to specify the
oracle in terms of the expected outcome. To make this feasible, test generation
needs to aim not only at high code coverage, but also at small test suites that
make oracle generation as easy as possible. A common approach in the literature
is to generate a test case for each coverage goal (e.g., branches in branch
coverage), and then to combine them in a single test suite. However, the size
of a resulting test suite is difficult to predict as a test case generated for
one goal may implicitly also cover any number of further coverage goals.
DISADVANTAGES
OF EXISTING SYSTEM:
·
The problem of the expected outcome
persists and has become known as the oracle problem. Sometimes, essential
properties of programs are formally specified or have to hold universally such
that no explicit oracles need to be defined (e.g., programs should normally not
crash).
·
Some targets are more difficult to cover
than others.
·
Coverage goals can be infeasible such
that there exists no input that would cover them Even if this particular
infeasible branch may be easy to detect, this is not true in general and thus
targeting infeasible goals will, per definition, fail and the effort would be
wasted.
PROPOSED SYSTEM:
In this paper, we evaluate a novel
approach for test data generation, which we call whole test suite generation,
which improves upon the current approach of targeting one goal at a time. We
use an evolutionary technique in which, instead of evolving each test case
individually, we evolve all the test cases in a test suite at the same time,
and the fitness function considers all the testing goals simultaneously. The
technique starts with an initial population of randomly generated test suites,
and then uses a Genetic Algorithm to optimize toward satisfying a chosen
coverage criterion, while using the test suite size as a secondary objective.
At the end, the best resulting test suite is minimized, giving us a test suite
as shown in fig 2. Stack example from fig
1. With such an approach, most of the complications and downsides of the one
target at a time approach either disappear or become significantly reduced. The
technique is implemented as part of our testing tool EVOSUITE which is freely
available online.
ADVANTAGES
OF PROPOSED SYSTEM:
v Because
to effectively address the problem of test suite generation we had to develop
specialized search operators, there would be no guarantee on the convergence
property of the resulting search algorithm. To cope with this problem, we
formally prove the convergence of our proposed technique.
v The
EVOSUITE approach yields significantly better results (i.e., either higher
coverage or, if the same coverage, then smaller test suites) compared to the
traditional approach of targeting each testing goal independently.
v In
some cases, EVOSUITE achieved up to 188 times higher coverage on average, and
test suites that were 62 percent smaller while maintaining the same structural
coverage.
v Furthermore,
running EVOSUITE with a constrained budget (1 million statement executions
during the search, up to a maximum 10 minutes timeout) resulted in an
impressive83 percent of coverage on average on our case study.
SYSTEM CONFIGURATION:-
HARDWARE CONFIGURATION:-
ü Processor - Pentium –IV
ü Speed - 1.1
Ghz
ü RAM - 256
MB(min)
ü Hard Disk -
20 GB
ü Key Board -
Standard Windows Keyboard
ü Mouse - Two
or Three Button Mouse
ü Monitor - SVGA
SOFTWARE CONFIGURATION:-
ü Operating System :
Windows XP
ü Programming Language :
JAVA
ü Java Version :
JDK 1.6 & above.
REFERENCE:
Gordon Fraser, Member, IEEE, and Andrea
Arcuri “Whole Test Suite Generation” IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,
VOL. 39, NO. 2, FEBRUARY 2013.
