← Back to papers
2019 ACL ACL 2019

Better Exploiting Latent Variables in Text Modeling

Abstract

AbstractWe show that sampling latent variables multiple times at a gradient step helps in improving a variational autoencoder and propose a simple and effective method to better exploit these latent variables through hidden state averaging. Consistent gains in performance on two different datasets, Penn Treebank and Yahoo, indicate the generalizability of our method.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning
🧭 Keyword Pioneer — latent variable sampling
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Canasai Kruengkrai

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Optimization & Theory > Bayesian Inference Deep Learning > Models > Variational Inference Deep Learning > Learning Types > Generative Models

Keywords

text modeling latent variable variational autoencoder latent variable sampling hidden state averaging
Download PDF

Related papers

What do phone embeddings learn about Phonology? 2019
Unsupervised Morphological Segmentation for Low-Resource Polysynthetic Languages 2019
Understanding Undesirable Word Embedding Associations 2019
Inferential Machine Comprehension: Answering Questions by Recursively Deducing the Evidence Chain from Text 2019
Domain Adaptation of Neural Machine Translation by Lexicon Induction 2019