Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice
RISE Research Institutes of Sweden, Digital Systems, Mobility and Systems.ORCID iD: 0000-0001-7879-4371
Lund University, Sweden.
Lund University, Sweden.
NordAxon AB, Sweden.
Show others and affiliations
2022 (English)In: Proceedings - 1st International Conference on AI Engineering - Software Engineering for AI, CAIN 2022, Institute of Electrical and Electronics Engineers Inc. , 2022, p. 22-32Conference paper, Published paper (Refereed)
Abstract [en]

Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2022. p. 22-32
Keywords [en]
action research, AI quality, conversational agent, generative dialog model, requirements engineering, software testing, Learning algorithms, Natural language processing systems, Quality assurance, Software agents, Conversational agents, Dialogue models, Megatrends, Model Selection, Requirement engineering, Software testings, Swedishs
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:ri:diva-59869DOI: 10.1145/3522664.3528592Scopus ID: 2-s2.0-85133467455ISBN: 9781450392754 (print)OAI: oai:DiVA.org:ri-59869DiVA, id: diva2:1685105
Conference
1st International Conference on AI Engineering - Software Engineering for AI, CAIN 2022, 16 May 2022 through 17 May 2022
Available from: 2022-08-01 Created: 2022-08-01 Last updated: 2024-01-10Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Borg, MarkusTomaszewski, Piotr

Search in DiVA

By author/editor
Borg, MarkusTomaszewski, Piotr
By organisation
Mobility and Systems
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 82 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf