Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A tale of four parsers: methodological reflections on diagnostic evaluation and in-depth error analysis for meaning representation parsing
University of Oslo, Norway.ORCID iD: 0000-0003-2033-1408
RISE Research Institutes of Sweden. Uppsala University, Sweden.ORCID iD: 0000-0002-7873-3971
University of Oslo, Norway.
University of Oslo, Norway.
2022 (English)In: Language resources and evaluation, ISSN 1574-020X, E-ISSN 1574-0218, Vol. 56, p. 1075-1102Article in journal (Refereed) Published
Abstract [en]

We discuss methodological choices in diagnostic evaluation and error analysis in meaning representation parsing (MRP), i.e. mapping from natural language utterances to graph-based encodings of semantic structure. We expand on a pilot quantitative study in contrastive diagnostic evaluation, inspired by earlier work in syntactic dependency parsing, and propose a novel methodology for qualitative error analysis. This two-pronged study is performed using a selection of submissions, data, and evaluation tools featured in the 2019 shared task on MRP. Our aim is to devise methods for identifying strengths and weaknesses in different broad families of parsing techniques, as well as investigating the relations between specific parsing approaches, different meaning representation frameworks, and individual linguistic phenomena—by identifying and comparing common error patterns. Our preliminary empirical results suggest that the proposed methodologies can be meaningfully applied to parsing into graph-structured target representations, as a side-effect uncovering hitherto unknown properties of the different systems that can inform future development and cross-fertilization across approaches.

Place, publisher, year, edition, pages
2022. Vol. 56, p. 1075-1102
National Category
Language Technology (Computational Linguistics)
Identifiers
URN: urn:nbn:se:ri:diva-59294DOI: 10.1007/s10579-022-09591-7OAI: oai:DiVA.org:ri-59294DiVA, id: diva2:1662036
Available from: 2022-05-30 Created: 2022-05-30 Last updated: 2023-07-06Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records

Nirve, Joakim

Search in DiVA

By author/editor
Buljan, MajaNirve, Joakim
By organisation
RISE Research Institutes of Sweden
In the same journal
Language resources and evaluation
Language Technology (Computational Linguistics)

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 17 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf