2019
ACL
ACL 2019
Identifying Visible Actions in Lifestyle Vlogs
Abstract
AbstractWe consider the task of identifying human actions visible in online videos. We focus on the widely spread genre of lifestyle vlogs, which consist of videos of people performing actions while verbally describing them. Our goal is to identify if actions mentioned in the speech description of a video are visually present. We construct a dataset with crowdsourced manual annotations of visible actions, and introduce a multimodal algorithm that leverages information derived from visual and linguistic clues to automatically infer which actions are visible in a video.
🌉
Interdisciplinary Bridge
— Computer Vision and Deep Learning and Machine Learning and Natural Language Processing
🧭
Keyword Pioneer
— lifestyle vlog
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Core Methods > Classification
Machine Learning > Learning Types > Self-Supervised Learning
Computer Vision > Analysis > Action Recognition
Machine Learning > Learning Types > Multi-Modal Learning
Computer Vision > Analysis > Video Understanding
Natural Language Processing > Understanding > Natural Language Inference
Deep Learning > Learning Types > Multi-Modal Learning