Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Requirement or Not, That is the Question: A Case from the Railway Industry
RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mälardalen University, Sweden.ORCID iD: 0009-0006-8512-6412
RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mälardalen University, Sweden.ORCID iD: 0000-0001-6418-9971
RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.ORCID iD: 0000-0002-1512-0844
Mälardalen University, Sweden.
Show others and affiliations
2023 (English)In: Lecture Notes in Computer Science. Volume 13975. Pages 105 - 121 2023, Springer Science and Business Media Deutschland GmbH , 2023, p. 105-121Conference paper, Published paper (Refereed)
Abstract [en]

Requirements in tender documents are often mixed with other supporting information. Identifying requirements in large tender documents could aid the bidding process and help estimate the risk associated with the project.  Manual identification of requirements in large documents is a resource-intensive activity that is prone to human error and limits scalability. This study compares various state-of-the-art approaches for requirements identification in an industrial context. For generalizability, we also present an evaluation on a real-world public dataset. We formulate the requirement identification problem as a binary text classification problem. Various state-of-the-art classifiers based on traditional machine learning, deep learning, and few-shot learning are evaluated for requirements identification based on accuracy, precision, recall, and F1 score. Results from the evaluation show that the transformer-based BERT classifier performs the best, with an average F1 score of 0.82 and 0.87 on industrial and public datasets, respectively. Our results also confirm that few-shot classifiers can achieve comparable results with an average F1 score of 0.76 on significantly lower samples, i.e., only 20% of the data.  There is little empirical evidence on the use of large language models and few-shots classifiers for requirements identification. This paper fills this gap by presenting an industrial empirical evaluation of the state-of-the-art approaches for requirements identification in large tender documents. We also provide a running tool and a replication package for further experimentation to support future research in this area. © 2023, The Author(s)

Place, publisher, year, edition, pages
Springer Science and Business Media Deutschland GmbH , 2023. p. 105-121
Keywords [en]
NLP, Requirements classification, Requirements identification, tender documents, Deep learning, Information retrieval systems, Natural language processing systems, Requirements engineering, Risk perception, Text processing, Bidding process, F1 scores, Human errors, Manual identification, Public dataset, Railway industry, Requirement identification, Requirements classifications, State-of-the-art approach, Classification (of information)
National Category
Language Technology (Computational Linguistics)
Identifiers
URN: urn:nbn:se:ri:diva-64397DOI: 10.1007/978-3-031-29786-1_8Scopus ID: 2-s2.0-85152587069ISBN: 9783031297854 (print)OAI: oai:DiVA.org:ri-64397DiVA, id: diva2:1755564
Conference
29th International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ 2023. Barcelona, Spain. 17 April 2023 through 20 April 2023
Note

Correspondence Address: Abbas, M. RISE Research Institutes of Sweden, Sweden; email: muhammad.abbas@ri.se; Funding details: ITEA; Funding text 1: Acknowledgement. This work is partially funded by the AIDOaRt (KDT) and SmartDelta [27] (ITEA) projects.

Available from: 2023-05-08 Created: 2023-05-08 Last updated: 2023-11-03Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Bashir, SarmadAbbas, MuhammadSaadatmand, Mehrdad

Search in DiVA

By author/editor
Bashir, SarmadAbbas, MuhammadSaadatmand, Mehrdad
By organisation
Industrial Systems
Language Technology (Computational Linguistics)

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 47 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf