Change search
Link to record
Permanent link

Direct link
Publications (2 of 2) Show all publications
Bashir, S., Abbas, M., Saadatmand, M., Enoiu, E., Bohlin, M. & Lindberg, P. (2023). Requirement or Not, That is the Question: A Case from the Railway Industry. In: Lecture Notes in Computer Science. Volume 13975. Pages 105 - 121 2023: . Paper presented at 29th International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ 2023. Barcelona, Spain. 17 April 2023 through 20 April 2023 (pp. 105-121). Springer Science and Business Media Deutschland GmbH
Open this publication in new window or tab >>Requirement or Not, That is the Question: A Case from the Railway Industry
Show others...
2023 (English)In: Lecture Notes in Computer Science. Volume 13975. Pages 105 - 121 2023, Springer Science and Business Media Deutschland GmbH , 2023, p. 105-121Conference paper, Published paper (Refereed)
Abstract [en]

Requirements in tender documents are often mixed with other supporting information. Identifying requirements in large tender documents could aid the bidding process and help estimate the risk associated with the project.  Manual identification of requirements in large documents is a resource-intensive activity that is prone to human error and limits scalability. This study compares various state-of-the-art approaches for requirements identification in an industrial context. For generalizability, we also present an evaluation on a real-world public dataset. We formulate the requirement identification problem as a binary text classification problem. Various state-of-the-art classifiers based on traditional machine learning, deep learning, and few-shot learning are evaluated for requirements identification based on accuracy, precision, recall, and F1 score. Results from the evaluation show that the transformer-based BERT classifier performs the best, with an average F1 score of 0.82 and 0.87 on industrial and public datasets, respectively. Our results also confirm that few-shot classifiers can achieve comparable results with an average F1 score of 0.76 on significantly lower samples, i.e., only 20% of the data.  There is little empirical evidence on the use of large language models and few-shots classifiers for requirements identification. This paper fills this gap by presenting an industrial empirical evaluation of the state-of-the-art approaches for requirements identification in large tender documents. We also provide a running tool and a replication package for further experimentation to support future research in this area. © 2023, The Author(s)

Place, publisher, year, edition, pages
Springer Science and Business Media Deutschland GmbH, 2023
Keywords
NLP, Requirements classification, Requirements identification, tender documents, Deep learning, Information retrieval systems, Natural language processing systems, Requirements engineering, Risk perception, Text processing, Bidding process, F1 scores, Human errors, Manual identification, Public dataset, Railway industry, Requirement identification, Requirements classifications, State-of-the-art approach, Classification (of information)
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:ri:diva-64397 (URN)10.1007/978-3-031-29786-1_8 (DOI)2-s2.0-85152587069 (Scopus ID)9783031297854 (ISBN)
Conference
29th International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ 2023. Barcelona, Spain. 17 April 2023 through 20 April 2023
Note

Correspondence Address: Abbas, M. RISE Research Institutes of Sweden, Sweden; email: muhammad.abbas@ri.se; Funding details: ITEA; Funding text 1: Acknowledgement. This work is partially funded by the AIDOaRt (KDT) and SmartDelta [27] (ITEA) projects.

Available from: 2023-05-08 Created: 2023-05-08 Last updated: 2023-11-03Bibliographically approved
Bashir, S., Abbas, M., Ferrari, A., Saadatmand, M. & Lindberg, P. (2023). Requirements Classification for Smart Allocation: A Case Study in the Railway Industry. In: 31st IEEE International Requirements Engineering Conference: . Paper presented at 2023 IEEE 31st International Requirements Engineering Conference (RE). Hannover, Germany: IEEE
Open this publication in new window or tab >>Requirements Classification for Smart Allocation: A Case Study in the Railway Industry
Show others...
2023 (English)In: 31st IEEE International Requirements Engineering Conference, Hannover, Germany: IEEE , 2023Conference paper, Published paper (Refereed)
Abstract [en]

Allocation of requirements to different teams is a typical preliminary task in large-scale system development projects. This critical activity is often performed manually and can benefit from automated requirements classification techniques. To date, limited evidence is available about the effectiveness of existing machine learning (ML) approaches for requirements classification in industrial cases. This paper aims to fill this gap by evaluating state-of-the-art language models and ML algorithms for classification in the railway industry. Since the interpretation of the results of ML systems is particularly relevant in the studied context, we also provide an information augmentation approach to complement the output of the ML-based classification. Our results show that the BERT uncased language model with the softmax classifier can allocate the requirements to different teams with a 76% F1 score when considering requirements allocation to the most frequent teams. Information augmentation provides potentially useful indications in 76% of the cases. The results confirm that currently available techniques can be applied to real-world cases, thus enabling the first step for technology transfer of automated requirements classification. The study can be useful to practitioners operating in requirements-centered contexts such as railways, where accurate requirements classification becomes crucial for better allocation of requirements to various teams.

Place, publisher, year, edition, pages
Hannover, Germany: IEEE, 2023
National Category
Computer Sciences
Identifiers
urn:nbn:se:ri:diva-67433 (URN)10.1109/RE57278.2023.00028 (DOI)
Conference
2023 IEEE 31st International Requirements Engineering Conference (RE)
Available from: 2023-09-28 Created: 2023-09-28 Last updated: 2023-11-03Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0009-0006-8512-6412

Search in DiVA

Show all publications