Industrial products and devices (e.g., vehicles, drones) are broadly accessed and managed using web applications. These applications have been a major concern for enterprises, as they are the preferred targets of attackers due to persistent vulnerabilities in their code. To address vulnerabilities, Static analysis tools (SASTs) have been widely used for detection, alongside the growing trend of employing prompt-engineered large language models (LLMs). Although they have proven useful for detection, both techniques tend to generate false positives (FPs), thereby unnecessarily increasing manual effort in the search for non-existent vulnerabilities; moreover, SASTs tend to miss vulnerabilities. In contrast, fine-tuned LLMs have proven effective at reasoning and classification tasks, but often require expensive training with balanced corpora. In this paper, we study SASTs, both types of LLMs, and their combination to improve overall vulnerability detection in web applications. We tested two modern SAST tools and two LLM models, across seven datasets for SQL injection (SQLi) vulnerability detection. Our findings reveal that combining the results of multiple solutions can improve vulnerability detection. The best combination integrates both LLMs and a SAST, where i) the fine-tuned LLM, together with the SAST, reduces FPs, mainly produced by the prompt-engineering LLM, and ii) both LLMs overcome SAST's limitation of missing vulnerabilities. On average, the F1-Score increases by 17-60% when SASTS and LLMs are combined. In particular, it can improve from 6% (with a standalone solution) to ≈100% when LLMs are combined with SASTs

The evolving landscape of software development demands that software testers continuously adapt to new tools, practices, and acquire new skills. This study investigates software testing competency needs in industry, identifies knowledge gaps in current testing education, and highlights competencies and gaps not addressed in academic literature. This is done by conducting two focus group sessions and interviews with professionals across diverse domains, including railway industry, healthcare, and software consulting and performing a curated small-scale scoping review. The study instrument, co-designed by members of the ENACTEST project consortium, was developed collaboratively and refined through multiple iterations to ensure comprehensive coverage of industry needs and educational gaps. In particular, by performing a thematic qualitative analysis, we report our findings and observations regarding: professional training methods, challenges in offering training in industry, different ways of evaluating the quality of training, identified knowledge gaps with respect to academic education and industry needs, future needs and trends in testing education, and knowledge transfer methods within companies. Finally, the scoping review results confirm knowledge gaps in areas such as AI testing, security testing and soft skills

[Context and Motivation] Engineers often need to refer back to release notes, manuals, and system architecture documents to understand, modify, or upgrade functionalities in alignment with new software releases. This is crucial to ensure that new stakeholder requirements align with the existing system, maintaining compatibility and preventing integration issues. [Problem] In practice, the manual process of retrieving the relevant information from technical documentation is time-intensive and frequently results in inefficient software release management. [Principal ideas/results] In this paper, we propose a question-answering chatbot, ReqRAG, leveraging Retrieval Augmented Generation (RAG) with Large Language Models (LLMs) to deliver accurate and up-to-date information from technical documents in response to given queries. We employ various context retrieval techniques paired with state-of-the-art LLMs to evaluate the ReqRAG approach in industrial settings. Furthermore, we conduct human evaluations of the results in collaboration with experts from Alstom to gain practical insights. Our results indicate that, on average, 70% of the generated responses are adequate, useful, and relevant to the practitioners. [Contribution] Fewer studies have comprehensively evaluated RAG-based approaches in industrial settings. Therefore, this work provides technical considerations for domain-specific chatbots, guiding researchers and practitioners facing similar challenges. 

Developing large-scale industrial systems requires high-quality requirements to avoid costly rework and project delays. However, linguistic ambiguities in natural language (NL) requirements have been a long-standing challenge, often introducing misinterpretations and inconsistencies that propagate throughout the development lifecycle. Such ambiguous NL requirements necessitate early detection and well-reasoned explanations to clarify and prevent further misunderstandings among stakeholders. While solutions have been developed to detect ambiguities in NL requirements, the advent of generative large language models (LLMs) offers new avenues for explanation-augmented requirements ambiguity detection. This paper empirically investigates LLMs for ambiguity detection and explanation in real-world industrial requirements by adopting an in-context learning paradigm. Our results from three industrial datasets show that LLMs achieve a 20.2% average performance increase in classifying ambiguous requirements when prompted with ten relevant in-context demonstrations (10 -shot), compared to no demonstrations (0 -shot). Additionally, we conducted human evaluations of the LLM-generated outputs with eight industry experts along four dimensions-naturalness, adequacy, usefulness and relevance-to gain practical insights. The results show an average rating of 3.84 out of 5 across evaluation criteria, indicating that the approach is effective in providing supporting explanations for requirement ambiguities

Requirement Engineering (RE) is crucial for identifying, analysing and documenting stakeholders’ needs and constraints for developing software systems. In most safety-critical domains, maintaining requirements and their links to other artifacts is also often required by regulatory bodies. Furthermore, in such contexts, requirements for new products often share similarities with previous existing projects performed by the company. Therefore, similar requirements can be retrieved to facilitate the feasibility analysis of new projects. In addition, when a new customer requests a new product, retrieval of similar requirements can enable requirements-driven software reuse and avoid redundant development efforts. Manually retrieving similar requirements for reuse is typically dependent on the engineer’s experience and is not scalable, as the set could be quite large. In this regard, applying natural language processing (NLP) techniques for automated similarity computation and retrieval ensures the independence of the process from the human experience and makes the process scalable. This chapter introduces linguistic similarity and several NLP-based similarity computation techniques that leverage linguistic features for similarity computation. Specifically, we cover techniques for computing similarity ranging from lexical to state-of-the-art deep neural network-based methods. We demonstrate their application in two example cases: (a) requirements reuse and (b) requirements-driven software retrieval. The practical guidance and example cases presented in the chapter can help practitioners apply the concepts to improve their processes where similarity computation is relevant.

Context. Software testing is a critical aspect of the software development lifecycle, yet it remains underrepresented in academic curricula. Despite advances in pedagogical practices and increased attention from the academic community, challenges persist in effectively teaching software testing. Understanding these challenges from the teachers' perspective is crucial to aligning education with industry needs.Objective. To analyze the characteristics, practices, tools, and challenges of software testing courses in higher education, from the perspective of educators, and to assess the integration of recent pedagogical approaches in software testing education.Method. A structured survey consisting of 52 questions was distributed to 143 software testing educators across Western European universities, resulting in 49 valid responses. The survey explored topics taught, course organization, teaching practices, tools and materials used, gamification approaches, and teacher satisfaction.Results. The survey revealed significant variability in course content, structure, and teaching methods. Most dedicated software testing courses are offered at the master's level and are elective, whereas testing is introduced earlier in less specialized (NST) courses. There is low adoption of formal guidelines (e.g., ACM, SWEBOK), limited integration of non-functional testing types, and a high diversity in textbooks and tools used. While modern practices like Test-Driven Development and automated assessment are increasingly adopted, gamification and active learning approaches remain underutilized. Teachers expressed a need for improved and more consistent teaching materials.Conclusion. The study highlights a mismatch between academic practices and industry expectations in software testing education. Greater integration of standardized curricula, broader adoption of modern teaching tools, and increased support for teachers through high-quality, adaptable teaching materials are needed to enhance the effectiveness of software testing education. © 2025

This paper presents an extended version of Deeper, a search-based simulation-integrated test solution that generates failure-revealing test scenarios for testing a deep neural network-based lane-keeping system. In the newly proposed version, we utilize a new set of bio-inspired search algorithms, genetic algorithm (GA), (Formula presented.) and (Formula presented.) evolution strategies (ES), and particle swarm optimization (PSO), that leverage a quality population seed and domain-specific crossover and mutation operations tailored for the presentation model used for modeling the test scenarios. In order to demonstrate the capabilities of the new test generators within Deeper, we carry out an empirical evaluation and comparison with regard to the results of five participating tools in the cyber-physical systems testing competition at SBST 2021. Our evaluation shows the newly proposed test generators in Deeper not only represent a considerable improvement on the previous version but also prove to be effective and efficient in provoking a considerable number of diverse failure-revealing test scenarios for testing an ML-driven lane-keeping system. They can trigger several failures while promoting test scenario diversity, under a limited test time budget, high target failure severity, and strict speed limit constraints. 

The Workshop on Automating Test Case Design, Selection and Evaluation (A-TEST) has provided a venue for researchers and industry members alike to exchange and discuss trending views, ideas, state of the art, work in progress, and scientific results on automated testing. Up until now it has run 13 editions since 2009. The 13th edition of the A-TEST workshop has been performed as an in-person workshop in Singapore during 17 to 18 of November, 2022. This edition of the A-TEST workshop was co-located with ESEC/FSE 2022 conference.

Processing and reviewing nightly test execution failure logs for large industrial systems is a tedious activity. Furthermore, multiple failures might share one root/common cause during test execution sessions, and the review might therefore require redundant efforts. This paper presents the LogGrouper approach for automated grouping of failure logs to aid root/common cause analysis and for enabling the processing of each log group as a batch. LogGrouper uses state-of-art natural language processing and clustering approaches to achieve meaningful log grouping. The approach is evaluated in an industrial setting in both a qualitative and quantitative manner. Results show that LogGrouper produces good quality groupings in terms of our two evaluation metrics (Silhouette Coefficient and Calinski-Harabasz Index) for clustering quality. The qualitative evaluation shows that experts perceive the groups as useful, and the groups are seen as an initial pointer for root cause analysis and failure assignment.