← Back to papers
2019 ACL ACL 2019

Detecting and Extracting of Adverse Drug Reaction Mentioning Tweets with Multi-Head Self Attention

Abstract

AbstractThis paper describes our system for the first and second shared tasks of the fourth Social Media Mining for Health Applications (SMM4H) workshop. We enhance tweet representation with a language model and distinguish the importance of different words with Multi-Head Self-Attention. In addition, transfer learning is exploited to make up for the data shortage. Our system achieved competitive results on both tasks with an F1-score of 0.5718 for task 1 and 0.653 (overlap) / 0.357 (strict) for task 2.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Healthcare & Medicine and Machine Learning and Natural Language Processing
🧭 Keyword Pioneer — multi-head self attention
🐣 Hot Topic Early Bird — multi-head attention
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Suyu Ge , Tao Qi , Chuhan Wu , Yongfeng Huang

Topics

Artificial Intelligence > Learning Paradigms > Transfer Learning Machine Learning > Learning Types > Self-Supervised Learning Natural Language Processing > Applications > Named Entity Recognition Deep Learning > Techniques > Attention Healthcare & Medicine > Clinical > Medical NLP

Keywords

transfer learning text classification named entity recognition language model multi-head attention adverse drug reaction multi-head self attention
Download PDF

Related papers

What do phone embeddings learn about Phonology? 2019
Unsupervised Morphological Segmentation for Low-Resource Polysynthetic Languages 2019
Understanding Undesirable Word Embedding Associations 2019
Inferential Machine Comprehension: Answering Questions by Recursively Deducing the Evidence Chain from Text 2019
Domain Adaptation of Neural Machine Translation by Lexicon Induction 2019