← Back to papers
2025 ACL ACL 2025

ClimateEval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change

Abstract

AbstractClimateEval is a comprehensive benchmark designed to evaluate natural language processing models across a broad range of tasks related to climate change. ClimateEval aggregates existing datasets along with a newly developed news classification dataset, created specifically for this release. This results in a benchmark of 25 tasks based on 13 datasets, covering key aspects of climate discourse, including text classification, question answering, and information extraction. Our benchmark provides a standardized evaluation suite for systematically assessing the performance of large language models (LLMs) on these tasks. Additionally, we conduct an extensive evaluation of open-source LLMs (ranging from 2B to 70B parameters) in both zero-shot and few-shot settings, analyzing their strengths and limitations in the domain of climate change.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Murathan Kurfali , Shorouq Zahra , Joakim Nivre , Gabriele Messori

Topics

Machine Learning > Learning Types > Zero-Shot Learning Natural Language Processing > Applications > Question Answering Natural Language Processing > Applications > Text Classification Natural Language Processing > Resources & Methods > Large Language Models Machine Learning > Learning Types > Few-Shot Learning Deep Learning > Models > Large Language Models Machine Learning > Optimization & Theory > Evaluation Deep Learning > Learning Types > Zero-Shot Learning Deep Learning > Learning Types > Few-Shot Learning

Keywords

zero-shot learning benchmark evaluation few-shot learning natural language processing text classification information extraction question answering climate change large language model
Download PDF

Related papers

From Real to Synthetic: Synthesizing Millions of Diversified and Complicated User Instructions with Attributed Grounding 2025
Graphically Speaking: Unmasking Abuse in Social Media with Conversation Insights 2025
CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision 2025
Structural Deep Encoding for Table Question Answering 2025
Vision-aided Unsupervised Constituency Parsing with Multi-MLLM Debating 2025