Machine learning testing in an ADAS case study using simulation-integrated bio-inspired search-based testingShow others and affiliations
2024 (English)In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, no 5, article id e2591Article in journal (Refereed) Published
Abstract [en]
This paper presents an extended version of Deeper, a search-based simulation-integrated test solution that generates failure-revealing test scenarios for testing a deep neural network-based lane-keeping system. In the newly proposed version, we utilize a new set of bio-inspired search algorithms, genetic algorithm (GA), (Formula presented.) and (Formula presented.) evolution strategies (ES), and particle swarm optimization (PSO), that leverage a quality population seed and domain-specific crossover and mutation operations tailored for the presentation model used for modeling the test scenarios. In order to demonstrate the capabilities of the new test generators within Deeper, we carry out an empirical evaluation and comparison with regard to the results of five participating tools in the cyber-physical systems testing competition at SBST 2021. Our evaluation shows the newly proposed test generators in Deeper not only represent a considerable improvement on the previous version but also prove to be effective and efficient in provoking a considerable number of diverse failure-revealing test scenarios for testing an ML-driven lane-keeping system. They can trigger several failures while promoting test scenario diversity, under a limited test time budget, high target failure severity, and strict speed limit constraints.
Place, publisher, year, edition, pages
John Wiley and Sons Ltd , 2024. no 5, article id e2591
Keywords [en]
advanced driver assistance systems, deep learning, evolutionary computation, lane-keeping system, machine learning testing, search-based testing, Automobile drivers, Biomimetics, Budget control, Deep neural networks, Embedded systems, Genetic algorithms, Learning systems, Particle swarm optimization (PSO), Software testing, Case-studies, Lane keeping, Machine-learning, Software Evolution, Software process, Test scenario
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:ri:diva-65687DOI: 10.1002/smr.2591Scopus ID: 2-s2.0-85163167144OAI: oai:DiVA.org:ri-65687DiVA, id: diva2:1786788
Note
Correspondence Address: M.H. Moghadam; Smart Industrial Automation, RISE Research Institutes of Sweden, Västerås, Stora Gatan 36, 722 12, Sweden;
This work has been funded by Vinnova through the ITEA3 European IVVES ( https://itea3.org/project/ivves.html ) and H2020‐ECSEL European AIDOaRT ( https://www.aidoart.eu/ ) and InSecTT ( https://www.insectt.eu/ ) projects. Furthermore, the project received partially financial support from the SMILE III project financed by Vinnova, FFI, Fordonsstrategisk forskning och innovation under the grant number: 2019‐05871.
2023-08-102023-08-102024-06-07Bibliographically approved