← Back to papers
2017 EMNLP EMNLP 2017

Syllable-aware Neural Language Models: A Failure to Beat Character-aware Ones

Abstract

AbstractSyllabification does not seem to improve word-level RNN language modeling quality when compared to character-based segmentation. However, our best syllable-aware language model, achieving performance comparable to the competitive character-aware model, has 18%-33% fewer parameters and is trained 1.2-2.2 times faster.

🌉 Interdisciplinary Bridge — Deep Learning and Natural Language Processing
📈 Trend Setter — Language Models
🧭 Keyword Pioneer — rnn language model
🐣 Hot Topic Early Bird — neural language model
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Zhenisbek Assylbekov , Rustem Takhanov , Bagdat Myrzakhmetov , Jonathan N. Washington

Topics

Deep Learning > Architectures > Neural Networks Natural Language Processing > Generation > Language Modeling Deep Learning > Models > Language Models

Keywords

recurrent neural network language model neural language model rnn language model syllable segmentation character segmentation word-level language model
Download PDF

Related papers

Reinforced Video Captioning with Entailment Rewards 2017
Cross-lingual Character-Level Neural Morphological Tagging 2017
Inter-Weighted Alignment Network for Sentence Pair Modeling 2017
Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings 2017
An Empirical Analysis of Edit Importance between Document Versions 2017