Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 70) Show all publications
Tahvili, S., Hatvani, L., Felderer, M., Afzal, W. & Bohlin, M. (2019). Automated functional dependency detection between test cases using Doc2Vec and Clustering. In: Proceedings - 2019 IEEE International Conference on Artificial Intelligence Testing, AITest 2019: . Paper presented at 1st IEEE International Conference on Artificial Intelligence Testing, AITest 2019, 4 April 2019 through 9 April 2019 (pp. 19-26). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Automated functional dependency detection between test cases using Doc2Vec and Clustering
Show others...
2019 (English)In: Proceedings - 2019 IEEE International Conference on Artificial Intelligence Testing, AITest 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 19-26Conference paper, Published paper (Refereed)
Abstract [en]

Knowing about dependencies and similarities between test cases is beneficial for prioritizing them for cost-effective test execution. This holds especially true for the time consuming, manual execution of integration test cases written in natural language. Test case dependencies are typically derived from requirements and design artifacts. However, such artifacts are not always available, and the derivation process can be very time-consuming. In this paper, we propose, apply and evaluate a novel approach that derives test cases' similarities and functional dependencies directly from the test specification documents written in natural language, without requiring any other data source. Our approach uses an implementation of Doc2Vec algorithm to detect text-semantic similarities between test cases and then groups them using two clustering algorithms HDBSCAN and FCM. The correlation between test case text-semantic similarities and their functional dependencies is evaluated in the context of an on-board train control system from Bombardier Transportation AB in Sweden. For this system, the dependencies between the test cases were previously derived and are compared to the results our approach. The results show that of the two evaluated clustering algorithms, HDBSCAN has better performance than FCM or a dummy classifier. The classification methods' results are of reasonable quality and especially useful from an industrial point of view. Finally, performing a random undersampling approach to correct the imbalanced data distribution results in an F1 Score of up to 75% when applying the HDBSCAN clustering algorithm.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2019
Keywords
Clustering Doc2Vec, FCM, HDBSCAN, Paragraph Vectors, Software Testing, Test Case Dependency, Artificial intelligence, Cost effectiveness, Semantics, Testing, Bombardier Transportation, Classification methods, Functional dependency, Random under samplings, Test case, Train control systems, Clustering algorithms
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-39269 (URN)10.1109/AITest.2019.00-13 (DOI)2-s2.0-85067096441 (Scopus ID)9781728104928 (ISBN)
Conference
1st IEEE International Conference on Artificial Intelligence Testing, AITest 2019, 4 April 2019 through 9 April 2019
Note

Funding details: 20130085, 20160139; Funding details: VINNOVA, MegaM@RT2; Funding text 1: ECSEL & VINNOVA (through projects MegaM@RT2 & TESTOMAT) and the Swedish Knowledge Foundation (through the projects TOCSYC (20130085) and TestMine (20160139)) have supported this work.

Available from: 2019-07-03 Created: 2019-07-03 Last updated: 2019-07-03Bibliographically approved
Helali Moghadam, M., Saadatmand, M., Borg, M., Bohlin, M. & Lisper, B. (2019). Machine Learning to Guide Performance Testing: An Autonomous Test Framework. In: ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'19, 2019: . Paper presented at ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'19, 22 Apr 2019, Xi’an, China.
Open this publication in new window or tab >>Machine Learning to Guide Performance Testing: An Autonomous Test Framework
Show others...
2019 (English)In: ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'19, 2019, 2019Conference paper, Published paper (Refereed)
Abstract [en]

Satisfying performance requirements is of great importance for performance-critical software systems. Performance analysis to provide an estimation of performance indices and ascertain whether the requirements are met is essential for achieving this target. Model-based analysis as a common approach might provide useful information but inferring a precise performance model is challenging, especially for complex systems. Performance testing is considered as a dynamic approach for doing performance analysis. In this work-in-progress paper, we propose a self-adaptive learning-based test framework which learns how to apply stress testing as one aspect of performance testing on various software systems to find the performance breaking point. It learns the optimal policy of generating stress test cases for different types of software systems, then replays the learned policy to generate the test cases with less required effort. Our study indicates that the proposed learning-based framework could be applied to different types of software systems and guides towards autonomous performance testing.

Keywords
performance requirements, performance testing, test case generation, reinforcement learning, autonomous testing, Engineering and Technology, Teknik och teknologier, Computer Systems, Datorsystem
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-39327 (URN)10.1109/ICSTW.2019.00046 (DOI)2-s2.0-85068406208 (Scopus ID)
Conference
ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'19, 22 Apr 2019, Xi’an, China
Available from: 2019-07-04 Created: 2019-07-04 Last updated: 2019-08-08Bibliographically approved
Tahvili, S., Pimentel, R., Afzal, W., Ahlberg, M., Fornander, E. & Bohlin, M. (2019). SOrTES: A Supportive Tool for Stochastic Scheduling of Manual Integration Test Cases. IEEE Access, 7, 12928-12946, Article ID 8616828.
Open this publication in new window or tab >>SOrTES: A Supportive Tool for Stochastic Scheduling of Manual Integration Test Cases
Show others...
2019 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 7, p. 12928-12946, article id 8616828Article in journal (Refereed) Published
Abstract [en]

The main goal of software testing is to detect as many hidden bugs as possible in the final software product before release. Generally, a software product is tested by executing a set of test cases, which can be performed manually or automatically. The number of test cases which are required to test a software product depends on several parameters such as the product type, size, and complexity. Executing all test cases with no particular order can lead to waste of time and resources. Test optimization can provide a partial solution for saving time and resources which can lead to the final software product being released earlier. In this regard, test case selection, prioritization, and scheduling can be considered as possible solutions for test optimization. Most of the companies do not provide direct support for ranking test cases on their own servers. In this paper, we introduce, apply, and evaluate sOrTES as our decision support system for manual integration of test scheduling. sOrTES is a Python-based supportive tool which schedules manual integration test cases which are written in a natural language text. The feasibility of sOrTES is studied by an empirical evaluation which has been performed on a railway use-case at Bombardier Transportation, Sweden. The empirical evaluation indicates that around 40 % of testing failure can be avoided by using the proposed execution schedules by sOrTES, which leads to an increase in the requirements coverage of up to 9.6%. 

Keywords
decision support systems, dependency, integration testing, manual testing, scheduler algorithm, Software testing, stochastic test scheduling, test optimization, Artificial intelligence, Integration, Program debugging, Scheduling, Stochastic systems, Testing, Bombardier Transportation, Empirical evaluations, Natural language text, Stochastic scheduling, Test scheduling
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-37921 (URN)10.1109/ACCESS.2019.2893209 (DOI)2-s2.0-85061302207 (Scopus ID)
Note

Funding details: European Research Consortium for Informatics and Mathematics, ERCIM; Funding text 1: This work was supported in part by ECSEL & VINNOVA through projects XIVT, TESTOMAT, and MegaM@RT2, in part by the Swedish Knowledge Foundation through projects TOCSYC and TESTMINE, and in part by ERCIM ‘‘Alain Bensoussan’’ Fellowship Programme.

Available from: 2019-03-05 Created: 2019-03-05 Last updated: 2019-03-05Bibliographically approved
Helali Moghadam, M., Saadatmand, M., Borg, M., Bohlin, M. & Lisper, B. (2018). Adaptive Run-time Response Time Control in PLC-based Real-Time Systems using Reinforcement Learning. In: : . Paper presented at 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (pp. 217-223).
Open this publication in new window or tab >>Adaptive Run-time Response Time Control in PLC-based Real-Time Systems using Reinforcement Learning
Show others...
2018 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Timing requirements such as constraints on response time are key characteristics of real-time systems and violations of these requirements might cause a total failure, particularly in hard real-time systems. Runtime monitoring of the system properties is of great importance to check the system status and mitigate such failures. Thus, a runtime control to preserve the system properties could improve the robustness of the system with respect to timing violations. Common control approaches may require a precise analytical model of the system which is difficult to be provided at design time. Reinforcement learning is a promising technique to provide adaptive model-free control when the environment is stochastic, and the control problem could be formulated as a Markov Decision Process. In this paper, we propose an adaptive runtime control using reinforcement learning for real-time programs based on Programmable Logic Controllers (PLCs), to meet the response time requirements. We demonstrate through multiple experiments that our approach could control the response time efficiently to satisfy the timing requirements.

Keywords
adaptive response time contro, lPLC-based real-time programs, reinforcement learning, runtime monitoring
National Category
Software Engineering
Identifiers
urn:nbn:se:ri:diva-34197 (URN)10.1145/3194133.3194153 (DOI)2-s2.0-85051555083 (Scopus ID)9781450357159 (ISBN)
Conference
13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems
Available from: 2018-07-13 Created: 2018-07-13 Last updated: 2019-04-05Bibliographically approved
Tahvili, S., Hatvani, L., Felderer, M., Afzal, W., Saadatmand, M. & Bohlin, M. (2018). Cluster-based test scheduling strategies using semantic relationships between test specifications. In: : . Paper presented at Proceedings of the 5th International Workshop on Requirements Engineering and Testing. Gothenburg, Sweden (pp. 1-4).
Open this publication in new window or tab >>Cluster-based test scheduling strategies using semantic relationships between test specifications
Show others...
2018 (English)Conference paper, Published paper (Other academic)
Abstract [en]

One of the challenging issues in improving the test efficiency isthat of achieving a balance between testing goals and testing resources.Test execution scheduling is one way of saving time andbudget, where a set of test cases are grouped and tested at thesame time. To have an optimal test execution schedule, all relatedinformation of a test case (e.g. execution time, functionality to betested, dependency and similarity with other test cases) need tobe analyzed. Test scheduling problem becomes more complicatedat high-level testing, such as integration testing and especially inmanual testing procedure. Test specifications are generally writtenin natural text by humans and usually contain ambiguity anduncertainty. Therefore, analyzing a test specification demands astrong learning algorithm. In this position paper, we propose anatural language processing-based approach that, given test specificationsat the integration level, allows automatic detection oftest cases semantic dependencies. The proposed approach utilizesthe Doc2Vec algorithm and converts each test case into a vectorin n-dimensional space. These vectors are then grouped using theHDBSCAN clustering algorithm into semantic clusters. Finally, aset of cluster-based test scheduling strategies are proposed for execution.The proposed approach has been applied in a sub-systemfrom the railway domain by analyzing an ongoing testing projectat Bombardier Transportation AB, Sweden.

Keywords
clustering, dependency, Doc2Vec, HDBSCAN, NLP, software testing, test optimization
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-34878 (URN)10.1145/3195538.3195540 (DOI)2-s2.0-85051238162 (Scopus ID)978-1-4503-5749-4 (ISBN)
Conference
Proceedings of the 5th International Workshop on Requirements Engineering and Testing. Gothenburg, Sweden
Available from: 2018-08-21 Created: 2018-08-21 Last updated: 2019-03-28Bibliographically approved
Tahvili, S., Ahlberg, M., Fornander, E., Afzal, W., Saadatmand, M., Bohlin, M. & Sarabi, M. (2018). Functional Dependency Detection for Integration Test Cases. In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security Companion, QRS-C 2018: . Paper presented at 18th IEEE International Conference on Software Quality, Reliability, and Security Companion, QRS-C 2018, 16 July 2018 through 20 July 2018 (pp. 207-214).
Open this publication in new window or tab >>Functional Dependency Detection for Integration Test Cases
Show others...
2018 (English)In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security Companion, QRS-C 2018, 2018, p. 207-214Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a natural language processing (NLP) based approach that, given software requirements specification, allows the functional dependency detection between integration test cases. We analyze a set of internal signals to the implemented modules for detecting dependencies between requirements and thereby identifying dependencies between test cases such that: module 2 depends on module 1 if an output internal signal from module 1 enters as an input internal signal to the module 2. Consequently, all requirements (and thereby test cases) for module 2 are dependent on all the designed requirements (and test cases) for module 1. The dependency information between requirements (and thus corresponding test cases) can be utilized for test case prioritization and scheduling. We have implemented our approach as a tool and the feasibility is evaluated through an industrial use case in the railway domain at Bombardier Transportation (BT), Sweden. © 2018 IEEE.

Keywords
Dependency, Internal Signals, NLP, Optimization, Software Requirement, Software Testing, C (programming language), Computer software selection and evaluation, Integral equations, Natural language processing systems, Requirements engineering, Software reliability, Testing, Bombardier Transportation, Dependency informations, Functional dependency, Software requirements, Software requirements specifications, Test case prioritization
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-35160 (URN)10.1109/QRS-C.2018.00047 (DOI)2-s2.0-85052305334 (Scopus ID)9781538678398 (ISBN)
Conference
18th IEEE International Conference on Software Quality, Reliability, and Security Companion, QRS-C 2018, 16 July 2018 through 20 July 2018
Available from: 2018-09-11 Created: 2018-09-11 Last updated: 2018-09-11Bibliographically approved
Helali Moghadam, M., Saadatmand, M., Borg, M., Bohlin, M. & Lisper, B. (2018). Learning-based Response Time Analysis in Real-Time Embedded Systems: A Simulation-based Approach. In: : . Paper presented at 1st International Workshop on Software Qualities and their Dependencies (pp. 21-24).
Open this publication in new window or tab >>Learning-based Response Time Analysis in Real-Time Embedded Systems: A Simulation-based Approach
Show others...
2018 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Response time analysis is an essential task to verify the behavior of real-time systems. Several response time analysis methods have been proposed to address this challenge, particularly for real-time systems with different levels of complexity. Static analysis is a popular approach in this context, but its practical applicability is limited due to the high complexity of the industrial real-time systems, as well as many unpredictable run-time events in these systems. In this work-in-progress paper, we propose a simulation-based response time analysis approach using reinforcement learning to find the execution scenarios leading to the worst-case response time. The approach learns how to provide a practical estimation of the worst-case response time through simulating the program without performing static analysis. Our initial study suggests that the proposed approach could be applicable in the simulation environments of the industrial real-time control systems to provide a practical estimation of the execution scenarios leading to the worst-case response time.

National Category
Software Engineering
Identifiers
urn:nbn:se:ri:diva-34196 (URN)10.1145/3194095.3194097 (DOI)2-s2.0-85051494988 (Scopus ID)
Conference
1st International Workshop on Software Qualities and their Dependencies
Available from: 2018-07-13 Created: 2018-07-13 Last updated: 2019-01-09Bibliographically approved
Helali Moghadam, M., Saadatmand, M., Borg, M., Bohlin, M. & Lisper, B. (2018). Learning-Based Self-Adaptive Assurance of Timing Properties in a Real-Time Embedded System. In: : . Paper presented at 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Systems (pp. 77-80).
Open this publication in new window or tab >>Learning-Based Self-Adaptive Assurance of Timing Properties in a Real-Time Embedded System
Show others...
2018 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Providing an adaptive runtime assurance technique to meet the performance requirements of a real-time system without the need for a precise model could be a challenge. Adaptive performance assurance based on monitoring the status of timing properties can bring more robustness to the underlying platform. At the same time, the results or the achieved policy of this adaptive procedure could be used as feedback to update the initial model, and consequently for producing proper test cases. Reinforcement-learning has been considered as a promising adaptive technique for assuring the satisfaction of the performance properties of software-intensive systems in recent years. In this work-in-progress paper, we propose an adaptive runtime timing assurance procedure based on reinforcement learning to satisfy the performance requirements in terms of response time. The timing control problem is formulated as a Markov Decision Process and the details of applying the proposed learning-based timing assurance technique are described.

Keywords
Real-time embedded systems, Reinforcement learning, Self-adaptive performance assurance, Timing properties
National Category
Software Engineering
Identifiers
urn:nbn:se:ri:diva-34194 (URN)10.1109/ICSTW.2018.00031 (DOI)2-s2.0-85050958526 (Scopus ID)9781538663523 (ISBN)
Conference
2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Systems
Available from: 2018-07-13 Created: 2018-07-13 Last updated: 2019-03-28Bibliographically approved
Flemström, D., Potena, P., Sundmark, D., Afzal, W. & Bohlin, M. (2018). Similarity-based prioritization of test case automation. Software quality journal, 26(4), 1421-1449
Open this publication in new window or tab >>Similarity-based prioritization of test case automation
Show others...
2018 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 26, no 4, p. 1421-1449Article in journal (Refereed) Published
Abstract [en]

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated. 

Keywords
Effort, Prioritization, Reuse, Software-testing, Test-case automation, Automation, Computer software reusability, Heuristic methods, Computational time, Global competition, Meta-heuristic approach, System integration, Test case, Software testing
National Category
Natural Sciences
Identifiers
urn:nbn:se:ri:diva-33511 (URN)10.1007/s11219-017-9401-7 (DOI)2-s2.0-85043389019 (Scopus ID)
Available from: 2018-03-23 Created: 2018-03-23 Last updated: 2018-12-20
Ghaviha, N., Bohlin, M., Holmberg, C., Dahlquist, E., Skoglund, R. & Jonasson, D. (2017). A driver advisory system with dynamic losses for passenger electric multiple units. Transportation Research Part C: Emerging Technologies, 85, 111-130
Open this publication in new window or tab >>A driver advisory system with dynamic losses for passenger electric multiple units
Show others...
2017 (English)In: Transportation Research Part C: Emerging Technologies, ISSN 0968-090X, E-ISSN 1879-2359, Vol. 85, p. 111-130Article in journal (Refereed) Published
Abstract [en]

Driver advisory systems, instructing the driver how to control the train in an energy efficient manner, is one the main tools for minimizing energy consumption in the railway sector. There are many driver advisory systems already available in the market, together with significant literature on the mathematical formulation of the problem. However, much less is published on the development of such mathematical formulations, their implementation in real systems, and on the empirical data from their deployment. Moreover, nearly all the designed driver advisory systems are designed as an additional hardware to be added in drivers’ cabin. This paper discusses the design of a mathematical formulation and optimization approach for such a system, together with its implementation into an Android-based prototype, the results from on-board practical experiments, and experiences from the implementation. The system is based on a more realistic train model where energy calculations take into account dynamic losses in different components of the propulsion system, contrary to previous approaches. The experimental evaluation shows a significant increase in accuracy, as compared to a previous approach. Tests on a double-track section of the Mälaren line in Sweden demonstrates a significant potential for energy saving.

Keywords
Driver advisory system, Electric multiple unit, Energy efficiency, Driver training, Electric losses, Electric railroads, Electric traction, Energy conservation, Energy utilization, Propulsion, Railroad transportation, Rapid transit, Advisory systems, Double-track section, Energy calculation, Experimental evaluation, Mathematical formulation, Minimizing energy, Optimization approach, control system, electric vehicle, energy use, optimization, railway transport, train, Sweden
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:ri:diva-33060 (URN)10.1016/j.trc.2017.09.010 (DOI)2-s2.0-85034098198 (Scopus ID)
Note

 Funding details: 2012-01277, VINNOVA; Funding details: Bombardier

Available from: 2018-01-11 Created: 2018-01-11 Last updated: 2019-07-02Bibliographically approved
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-1597-6738

Search in DiVA

Show all publications
v. 2.35.7