← Back to papers
2020 CVPR CVPR 2020

4D Visualization of Dynamic Events From Unconstrained Multi-View Videos

Abstract

We present a data-driven approach for 4D space-time visualization of dynamic events from videos captured by hand-held multiple cameras. Key to our approach is the use of self-supervised neural networks specific to the scene to compose static and dynamic aspects of an event. Though captured from discrete viewpoints, this model enables us to move around the space-time of the event continuously. This model allows us to create virtual cameras that facilitate: (1) freezing the time and exploring views; (2) freezing a view and moving through time; and (3) simultaneously changing both time and view. We can also edit the videos and reveal occluded objects for a given view if it is visible in any of the other views. We validate our approach on challenging in-the-wild events captured using up to 15 mobile cameras.

🌉 Interdisciplinary Bridge — Computer Vision and Deep Learning and Machine Learning
🧭 Keyword Pioneer — virtual camera
🐣 Hot Topic Early Bird — neural rendering
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Aayush Bansal , Minh Vo , Yaser Sheikh , Deva Ramanan , Srinivasa Narasimhan

Topics

Machine Learning > Learning Types > Self-Supervised Learning Deep Learning > Architectures > Neural Networks Computer Vision > Analysis > 3D Vision Deep Learning > Learning Types > Self-Supervised Learning

Keywords

3d reconstruction neural rendering view synthesis multi-view video temporal interpolation virtual camera self-supervised neural network 4d visualization
Download PDF

Related papers

Deep Polarization Cues for Transparent Object Segmentation 2020
HRank: Filter Pruning Using High-Rank Feature Map 2020
Panoptic-Based Image Synthesis 2020
Select, Supplement and Focus for RGB-D Saliency Detection 2020
ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings 2020