2022
ACL
ACL 2022
Modular and Parameter-Efficient Multimodal Fusion with Prompting
Abstract
AbstractRecent research has made impressive progress in large-scale multimodal pre-training. In the context of the rapid growth of model size, it is necessary to seek efficient and flexible methods other than finetuning. In this paper, we propose to use prompt vectors to align the modalities. Our method achieves comparable performance to several other multimodal fusion methods in low-resource settings. We further show that our method is modular and parameter-efficient for processing tasks involving two or more data modalities.
🌉
Interdisciplinary Bridge
— Artificial Intelligence and Deep Learning and Machine Learning
🧭
Keyword Pioneer
— prompt vector
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Artificial Intelligence > Core AI > Multimodal Learning
Machine Learning > Learning Types > Self-Supervised Learning
Machine Learning > Application Areas > Efficient Computing
Deep Learning > Techniques > Pretraining
Machine Learning > Learning Types > Multi-Modal Learning
Deep Learning > Learning Types > Multi-Modal Learning
Deep Learning > Techniques > Prompt Engineering