← Back to papers
2022 ACL ACL 2022

Clozer”:” Adaptable Data Augmentation for Cloze-style Reading Comprehension

Abstract

AbstractTask-adaptive pre-training (TAPT) alleviates the lack of labelled data and provides performance lift by adapting unlabelled data to downstream task. Unfortunately, existing adaptations mainly involve deterministic rules that cannot generalize well. Here, we propose Clozer, a sequence-tagging based cloze answer extraction method used in TAPT that is extendable for adaptation on any cloze-style machine reading comprehension (MRC) downstream tasks. We experiment on multiple-choice cloze-style MRC tasks, and show that Clozer performs significantly better compared to the oracle and state-of-the-art in escalating TAPT effectiveness in lifting model performance, and prove that Clozer is able to recognize the gold answers independently of any heuristics.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Holy Lovenia , Bryan Wilie , Willy Chung , Zeng Min , Samuel Cahyawijaya , Dan Su , Pascale Fung

Topics

Machine Learning > Learning Types > Self-Supervised Learning Machine Learning > Application Areas > Data Augmentation Natural Language Processing > Applications > Machine Reading Comprehension Machine Learning > Learning Paradigms > Self-Supervised Learning Deep Learning > Learning Types > Data Augmentation

Keywords

data augmentation reading comprehension sequence tagging machine reading comprehension masked language modeling cloze task task-adaptive pre-training
Download PDF

Related papers

HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing 2022
KG-CRuSE: Recurrent Walks over Knowledge Graph for Explainable Conversation Reasoning using Semantic Embeddings 2022
Toward Knowledge-Enriched Conversational Recommendation Systems 2022
Investigating the Medical Coverage of a Translation System into Pictographs for Patients with an Intellectual Disability 2022
TableFormer: Robust Transformer Modeling for Table-Text Encoding 2022