← Back to papers
2020 EMNLP EMNLP 2020

Sentence Boundary Detection on Line Breaks in Japanese

Abstract

AbstractFor NLP, sentence boundary detection (SBD) is an essential task to decompose a text into sentences. Most of the previous studies have used a simple rule that uses only typical characters as sentence boundaries. However, some characters may or may not be sentence boundaries depending on the context. We focused on line breaks in them. We newly constructed annotated corpora, implemented sentence boundary detectors, and analyzed performance of SBD in several settings.

🌉 Interdisciplinary Bridge — Machine Learning and Natural Language Processing
🧭 Keyword Pioneer — natural language preprocessing
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Security & Privacy, Speech & Audio

Authors

Yuta Hayashibe , Kensuke Mitsuzawa

Topics

Machine Learning > Core Methods > Classification Machine Learning > Learning Types > Semi-Supervised Learning Natural Language Processing > Understanding > Syntax Machine Learning > Core Methods > Sequence Labeling Natural Language Processing > Applications > Text Processing

Keywords

text classification sentence boundary detection text segmentation annotated corpus text processing japanese text rule-based method natural language preprocessing corpora annotation line break
Download PDF

Related papers

Fast semantic parsing with well-typedness guarantees 2020
Detecting Objectifying Language in Online Professor Reviews 2020
Analogous Process Structure Induction for Sub-event Sequence Prediction 2020
Aspect Sentiment Classification with Aspect-Specific Opinion Spans 2020
Robust and Interpretable Grounding of Spatial References with Relation Networks 2020