Automatic Error Analysis for Document-level Information Extraction Article Swipe

PDF

Related Concepts

Computer science Relationship extraction Precision and recall Information extraction Sentence Relation (database) Transformation (genetics) Data mining Artificial neural network Range (aeronautics) Event (particle physics) Information retrieval Recall Artificial intelligence Natural language processing Engineering Chemistry Biochemistry Philosophy Aerospace engineering Physics Linguistics Quantum mechanics Gene

Aliva Das , Xinya Du , Barry Wang , Kejian Shi , Jiayuan Gu , Thomas C. Porter , Claire Cardie ·

YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.18653/v1/2022.acl-long.274 · OA: W4285272511

Document-level information extraction (IE) tasks have recently begun to be revisited in earnest using the end-to-end neural network techniques that have been successful on their sentence-level IE counterparts. Evaluation of the approaches, however, has been limited in a number of dimensions. In particular, the precision/recall/F1 scores typically reported provide few insights on the range of errors the models make. We build on the work of Kummerfeld and Klein (2013) to propose a transformation-based framework for automating error analysis in document-level event and (N-ary) relation extraction. We employ our framework to compare two state-of-the-art document-level template-filling approaches on datasets from three domains; and then, to gauge progress in IE since its inception 30 years ago, vs. four systems from the MUC-4 (1992) evaluation.