← Learning Types
Deep Learning › Learning Types ›

Pretraining

26 directly classified papers

Papers per year

Papers

Scalable Vision Language Model Training via High Quality Data Curation ACL 2025
Pre-trained Models Perform the Best When Token Distributions Follow Zipf’s Law EMNLP 2025
ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws EMNLP 2024
Bootstrapped Pre-training with Dynamic Identifier Prediction for Generative Retrieval ACL 2024
BanglaTLit: A Benchmark Dataset for Back-Transliteration of Romanized Bangla EMNLP 2024
Code-Switching with Word Senses for Pretraining in Neural Machine Translation EMNLP 2023
Pretraining Language Models with Text-Attributed Heterogeneous Graphs EMNLP 2023
Train Once and Explain Everywhere: Pre-training Interpretable Graph Neural Networks NIPS 2023
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models EMNLP 2022
Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training EMNLP 2022
Generative Approach for Gender-Rewriting Task with ArabicT5 EMNLP 2022
Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis ACL 2022
Data Contamination: From Memorization to Exploitation ACL 2022
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models ACL 2022
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples EMNLP 2022
Pre-training Language Models with Deterministic Factual Knowledge EMNLP 2022
Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment EMNLP 2021
When Do You Need Billions of Words of Pretraining Data? ACL 2021
PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction ACL 2021
Are Pretrained Convolutions Better than Pretrained Transformers? ACL 2021
A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations ACL 2021
GAIA: A Transfer Learning System of Object Detection That Fits Your Needs CVPR 2021
Exploiting Unsupervised Data for Emotion Recognition in Conversations EMNLP 2020
Span Selection Pre-training for Question Answering ACL 2020
Cross-lingual Language Model Pretraining NIPS 2019