Robust Multi-Body Feature Tracker: A Segmentation-Free Approach

Feature tracking is a fundamental problem in computer vision with applications in various tasks including 3D reconstruction and visual SLAM. While many methods have been devoted to making these tasks robust to noise and outliers, less attention has been attracted to improving the feature tracking itself. This paper introduces a novel multi-body feature tracker that takes advantage of the multi-body rigidity assumption to improve tracking robustness. A conventional approach to addressing this problem would consist of alternating between solving two subtasks: motion segmentation and feature tracking under rigidity constraints for each segment. This approach, however, requires knowing the number of motions, as well as assigning points to motion groups, which is typically sensitive to the motion estimates. By contrast, here, we introduce a segmentation-free solution to multi-body feature tracking that bypasses the motion assignment step and reduces to solving a series of subproblems with closed-form solutions. Our experiments demonstrate the benefits of our approach in terms of tracking accuracy and robustness to noise.