← Back to papers
2020 COLING COLING 2020

Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating

Abstract

AbstractThe use of pre-trained language models such as BERT and ULMFiT has become increasingly popular in shared tasks, due to their powerful language modelling capabilities. Our entry to SemEval uses ERNIE 2.0, a language model which is pre-trained on a large number of tasks to enrich the semantic and syntactic information learned. ERNIE’s knowledge masking pre-training task is a unique method for learning about named entities, and we hypothesise that it may be of use in a dataset which is built on news headlines and which contains many named entities. We optimize the hyperparameters in a regression and classification model and find that the hyperparameters we selected helped to make bigger gains in the classification model than the regression model.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning and Natural Language Processing
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

J. A. Meaney , Steven Wilson , Walid Magdy

Topics

Artificial Intelligence > Learning Paradigms > Transfer Learning Machine Learning > Core Methods > Classification Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Natural Language Processing > Understanding > Sentiment Analysis Natural Language Processing > Applications > Text Classification Natural Language Processing > Resources & Methods > Large Language Models Natural Language Processing > Applications > Sentiment Analysis

Keywords

text classification hyperparameter optimization named entity recognition humor detection text regression language model classification model pre-trained language model regression model named entity humor ranking knowledge masking
Download PDF

Related papers

Persuasiveness of News Editorials depending on Ideology and Personality 2020
A Graph Representation of Semi-structured Data for Web Question Answering 2020
Span-based Joint Entity and Relation Extraction with Attention-based Span-specific and Contextual Semantic Representations 2020
Hierarchical Chinese Legal event extraction via Pedal Attention Mechanism 2020
End-to-End Emotion-Cause Pair Extraction with Graph Convolutional Network 2020