Context When developing software, it is vitally important to keep the level of technical debt down since, based on several studies, it has been well established that technical debt can lower the development productivity, decrease the developers' morale and compromise the overall quality of the software, among others. However, even if researchers and practitioners working in today's software development industry are quite familiar with the concept of technical debt and its related negative consequences, there has been no empirical research focusing specifically on how software managers actively communicate and manage the need to keep the level of technical debt as low as possible. Objective This study aims to understand how software companies give incentives to manage technical debt. This is carried out by exploring how companies encourage and reward practitioners for actively keeping the level of technical debt down add whether the companies use any forcing or penalising initiatives when managing technical debt. Method As a first step, this paper reports the results of both an online survey providing quantitative data from 258 participants and interviews with 32 software practitioners. As a second step, this study sets out to specifically provide a detailed assessment of additional and in-depth analysis of technical debt management strategies based on an encouraging mindset and attitude from both managers and technical roles to understand how, when and by whom such strategies are adopted in practice. Results Our findings show that having a technical debt management strategy (specially based on encouragement) can significantly impact the amount of technical debt related to the software. Conclusion The result indicates that there is considerable unfulfilled potential to influence how software practitioners can further limit and reduce technical debt by adopting a strategy based explicitly on an encouraging mindset from managers where they also specifically dedicate time and resources for technical debt remediation activities.

[Context] It is an enigma that agile projects can succeed "without requirements" when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. [Objective] We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. [Method] We performed an iterative case study at three companies and collected data through 14 interviews and 2 focus groups. [Results] The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. [Conclusions] The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.

Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components. Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.

Context

In software project management, the distribution of resources to various project activities is one of the most challenging problems since it affects team productivity, product quality and project constraints related to budget and scheduling.

Objective

The study aims to (a) reveal the high complexity of modelling the effort usage proportion in different phases as well as the divergence from various rules-of-thumb in related literature, and (b) present a systematic data analysis framework, able to offer better interpretations and visualisation of the effort distributed in specific phases.

Method

The basis for the proposed multivariate statistical framework is Compositional Data Analysis, a methodology appropriate for proportions, along with other methods like the deviation from rules-of-thumb, the cluster analysis and the analysis of variance. The effort allocations to phases, as reported in around 1500 software projects of the ISBSG R11 repository, were transformed to vectors of proportions of the total effort and were analysed with respect to prime project attributes.

Results

The proposed statistical framework was able to detect high dispersion among data, distribution inequality and various interesting correlations and trends, groupings and outliers, especially with respect to other categorical and continuous project attributes. Only a very small number of projects were found close to the rules-of-thumb from the related literature. Significant differences in the proportion of effort spent in different phrases for different types of projects were found.

Conclusion

There is no simple model for the effort allocated to phases of software projects. The data from previous projects can provide valuable information regarding the distribution of the effort for various types of projects, through analysis with multivariate statistical methodologies. The proposed statistical framework is generic and can be easily applied in a similar sense to any dataset containing effort allocation to phases.

Context: Deep Neural Networks (DNN) have shown great promise in various domains, for example to support pattern recognition in medical imagery. However, DNNs need to be tested for robustness before being deployed in safety critical applications. One common challenge occurs when the model is exposed to data samples outside of the training data domain, which can yield to outputs with high confidence despite no prior knowledge of the given input. Objective: The aim of this paper is to investigate how the performance of detecting out-of-distribution (OOD) samples changes for outlier detection methods (e.g., supervisors) when DNNs become better on training samples. Method: Supervisors are components aiming at detecting out-of-distribution samples for a DNN. The experimental setup in this work compares the performance of supervisors using metrics and datasets that reflect the most common setups in related works. Four different DNNs with three different supervisors are compared during different stages of training, to detect at what point during training the performance of the supervisors begins to deteriorate. Results: Found that the outlier detection performance of the supervisors increased as the accuracy of the underlying DNN improved. However, all supervisors showed a large variation in performance, even for variations of network parameters that marginally changed the model accuracy. The results showed that understanding the relationship between training results and supervisor performance is crucial to improve a model's robustness. Conclusion: Analyzing DNNs for robustness is a challenging task. Results showed that variations in model parameters that have small variations on model predictions can have a large impact on the out-of-distribution detection performance. This kind of behavior needs to be addressed when DNNs are part of a safety critical application and hence, the necessary safety argumentation for such systems need be structured accordingly.

Context: Testing for properties such as robustness or security is complicated because their concerns are often repeated in many locations and muddled with the normal code. Such “cross-cutting concerns” include things like interrupt events, exception handling, and security protocols. Aspect-oriented (AO) modeling allows developers to model the cross-cutting behavior independently of the normal behavior, thus supporting model-based testing of cross-cutting concerns. However, mutation operators defined for AO programs (source code) are usually not applicable to AO models (AOMs) and operators defined for models do not target the AO features. Objective: We present a method to design abstract tests at the aspect-oriented model level. We define mutation operators for aspect-oriented models and evaluate the generated mutants for an example system. Method: AOMs are mutated with novel operators that specifically target the AO modeling features. Test traces killing these mutant models are then generated. The generated and selected traces are abstract tests that can be transformed to concrete black-box tests and run on the implementation level, to evaluate the behavior of the woven cross-cutting concerns (combined aspect and base models). Results: This paper is a significant extension of our paper at Mutation 2015. We present a complete fault model, additional mutation operators, and a thorough analysis of the mutants generated for an example system. Conclusions: The analysis shows that some mutants are stillborn (syntactically illegal) but none is equivalent (exhibiting the same behavior as the original model). Additionally, our AOM-specific mutation operators can be combined with pre-existing operators to mutate code or models without any overlap.

Context: The development of software-intensive systems includes many decisions involving various stakeholders with often conflicting interests and viewpoints. Objective: Decisions are rarely systematically documented and sporadically explored. This limits the opportunity for learning and improving on important decisions made in the development of software-intensive systems. Method: In this work, we enable support for the systematic documentation of decisions, improve their traceability and contribute to potentially improved decision-making in strategic, tactical and operational contexts. Results: We constructed a taxonomy for documentation supporting decision-making, called GRADE. GRADE was developed in a research project that required composition of a common dedicated language to make feasible the identification of new opportunities for better decision support and evaluation of multiple decision alternatives. The use of the taxonomy has been validated through thirty three decision cases from industry. Conclusion: This paper occupies this important yet greatly unexplored research gap by developing the GRADE taxonomy that serves as a common vocabulary to describe and classify decision-making with respect to architectural assets.

Context: Since 1998, the ACM/IEEE 25th International Conference on Model Driven Engineering Languages and Systems (MODELS) has been studying all aspects surrounding modeling in software engineering, from languages and methods to tools and applications. In order to enable empirical studies, the MODELS community developed a need for having examples of models, especially of models used in real software development projects. Such models may be used for a range of purposes, but mostly related to domain analysis and software design (at various levels of abstraction). However, finding such models was very difficult. The most used ones had their origin in academic books or student projects, which addressed “artificial” applications, i.e., were not base on real-case scenarios. To address this issue, the authors of this reflection paper, members of the modeling and of the mining software repositories fields, came together with the aim of creating a dataset with an abundance of modeling projects by mining GitHub. As a scoping of our effort we targeted models represented using the UML notation because this is the lingua franca in practice for software modeling. As a result, almost 100k models from 22k projects were made publicly available, known as the Lindholmen dataset. Objective: In this paper, we analyze the impact of our research, and compare this to what we envisioned in 2016. We draw practical lessons gained from this effort, reflect on the perils and pitfalls of the dataset, and point out promising avenues of research. Method: We base our reflection on the systematic analysis of recent research literature, and especially those papers citing our dataset and its associated publications. Results: What we envisioned in the original research when making the dataset available has to a major extent not come true; however, fellow researchers have found alternative uses of the dataset. Conclusions: By understanding the possibilities and shortcomings of the current dataset, we aim to offer the research community i) future research avenues of how the data can be used; and ii) raise awareness of the limitations, not only to point out threats to validity of research, but also to encourage fellow researchers to find ideas to overcome them. Our reflections can also be helpful to researchers who want to perform similar mining efforts.

Context: Software engineering is becoming more and more distributed. Developers and other stakeholders are often located in different locations, departments, and countries and operating within different time zones. Most online software design and modeling tools are not adequate for distributed collaboration since they do not support awareness and lack features for effective communication. Objective: The aim of our research is to support distributed software design activities in Virtual Reality (VR). Method: Using design science research methodology, we design and evaluate a tool for collaborative design in VR. We evaluate the collaboration efficiency and recall of design information when using the VR software design environment compared to a non-VR software design environment. Moreover, we collect the perceptions and preferences of users to explore the opportunities and challenges that were incurred by using the VR software design environment. Results: We find that there is no significant difference in the efficiency and recall of design information when using the VR compared to the non-VR environment. Furthermore, we find that developers are more satisfied with collaboration in VR. Conclusion: The results of our research and similar studies show that working in VR is not yet faster or more efficient than working on standard desktops. It is very important to improve the interface in VR (gestures with haptics, keyboard and voice input), as confirmed by the difference in results between the first and second evaluation. 

Context: In the digital age, there is a notable increase in fraudulent activities perpetrated by social engineers who exploit individuals’ limited knowledge of digital devices. These actors strategically manipulate human psychology, targeting IT devices to gain unauthorized access to sensitive data. Objectives: Our study is centered around two distinct objectives to be accomplished through the utilization of a serious game: (i) The primary objective entails delivering training and educational content to participants with a focus on phishing attacks; (ii) The secondary objective aims to heighten participants’ awareness regarding the perils associated with divulging excessive information online. Methodology: To address these objectives, we have employed the following techniques and methods: (i) A comprehensive literature review was conducted to establish foundational knowledge in areas such as social engineering, game design, learning principles, human interaction, and game-based learning; (ii) We meticulously aligned the game design with the philosophical concept of social engineering attacks; (iii) We devised and crafted an advanced hybrid version of the game, incorporating the use of QR codes to generate game card data; (iv) We conducted an empirical evaluation encompassing surveys, observations, discussions, and URL assessments to assess the effectiveness of the proposed hybrid game version. Results: Quantitative data and qualitative observations suggest the “PhishDefend Quest” game successfully improved players’ comprehension of phishing threats and how to detect them through an interactive learning experience. The results highlight the potential of serious games to educate people about social engineering risks. Conclusion: Through the evaluation, we can readily arrive at the following conclusions: (i) Game-based learning proves to be a viable approach for educating participants about phishing awareness and the associated risks tied to the unnecessary disclosure of sensitive information online; (ii) Furthermore, game-based learning serves as an effective means of disseminating awareness among participants and players concerning prevalent phishing attacks. © 2024 The Authors