Software product line test suite reduction with constraint optimization
2017 (English)Conference paper, Published paper (Refereed)
Abstract [en]
In many cases, Software Product Line Testing (SPLT) targets only the selection of test cases which cover product features or feature interactions. However, higher testing efficiency can be achieved through the selection of test cases with improved fault-revealing capabilities. By associating each test case a priority-value representing (or aggregating) different criteria, such as importance (in terms of fault discovered in previous test campaigns), duration or cost, it becomes possible to select a feature-covering test suite with improved capabilities. A crucial objective in SPLT then becomes to identify a test suite that optimizes reaching a specific goal (lower test duration or cost), while preserving full feature coverage. In this article, we revisit this problem with a new approach based on constraint optimization with the NValue and GlobalCardinality constraints and a sophisticated search heuristic. These constraints enforce the coverage of all features through the computation of max flows in a network flow representing the coverage relation. The computed max flows represent possible solutions which are further processed in order to determine the solution that optimizes the given objective function, e.g., the smallest test execution costs. Our approach is implemented in a tool called Flower/C and experimentally evaluated on both randomly generated instances and standard benchmarks. Comparing Flower/C with MiniSAT+ and Cplex, stateof-the-art tools for constraint optimization, we show that our approach is competitive with both tools on random instances and benchmarks. Our results show that MiniSAT+ is not competitive at all, whereas when the priority-value of each test case is uniformly set to 1, that Flower/C approaches Cplex in performance. We compare four different models of Flower/C, using different global constraints, and the one mixing different constraints shows the best performance with high reduction rates. These results open the door to an industrial adoption of the proposed technology.
Place, publisher, year, edition, pages
2017. p. 68-87
Keywords [en]
Feature coverage, Software product line testing, Test suite optimization, Test suite reduction, Computer software, Constrained optimization, Costs, Feature extraction, Heuristic algorithms, Optimization, Testing, Constraint optimizations, Feature interactions, Objective functions, Software Product Line, Test-execution cost, Software testing
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:ri:diva-31109DOI: 10.1007/978-3-319-62569-0_4Scopus ID: 2-s2.0-85026636393ISBN: 9783319625683 (print)OAI: oai:DiVA.org:ri-31109DiVA, id: diva2:1136543
Conference
11th International Joint Conference on Software Technologies, ICSOFT 2016. 24 July 2016 through 26 July 2016
Note
Funding details: Cisco Systems; Funding details: Norges Forskningsråd; Funding text: We are grateful to Marius Liaeen from Cisco Systems, Norway and Alexandre Petillon for their participation to the discussion and initial work related to the approach described in the article. This work is partly supported by the Research Council of Norway (RCN) through the research-based innovation center Certus, under the SFI program.
2017-08-282017-08-282023-05-05Bibliographically approved