2024
EACL
EACL 2024
Using ChatGPT for Annotation of Attitude within the Appraisal Theory: Lessons Learned
Abstract
AbstractWe investigate the potential of using ChatGPT to annotate complex linguistic phenomena, such as language of evaluation, attitude and emotion. For this, we automatically annotate 11 texts in English, which represent spoken popular science, and evaluate the annotations manually. Our results show that ChatGPT has good precision in itemisation, i.e. detecting linguistic items in the text that carry evaluative meaning. However, we also find that the recall is very low. Besides that, we state that the tool fails in labeling the detected items with the correct categories on a more fine-grained level of granularity. We analyse the errors to find systematic errors related to specific categories in the annotation scheme.
🌉
Interdisciplinary Bridge
— Artificial Intelligence and Interdisciplinary and Natural Language Processing
🧭
Keyword Pioneer
— attitude detection
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Topics
Natural Language Processing > Understanding > Semantic Analysis
Natural Language Processing > Understanding > Sentiment Analysis
Interdisciplinary > Linguistics > Computational Linguistics
Natural Language Processing > Applications > Sentiment Analysis
Artificial Intelligence > Core AI > Large Language Models