Seeing Glassware: from Edge Detection to Pose Estimation and Shape Recovery

Perception of transparent objects has been an open challenge in robotics despite advances in sensors and data- driven learning approaches. In this paper, we introduce a new approach that combines recent advances in learnt object detectors with perceptual grouping in 2D, and projective geometry of apparent contours in 3D. We train a state of the art structured edge detector on an annotated set of foreground glassware. We assume that we deal with surfaces of revolution (SOR) and apply perceptual symmetry grouping in a 2D spherical transformation of the image to obtain a 2D detection of the glassware object and a hypothesis about its 2D axis. Rather than stopping at a single view detection, we ultimately want to reconstruct the 3D shape of the object and its 3D pose to allow for a robot to grasp it. Using two views allows us to decouple the 3D axis localization from the shape estimation. We develop a parametrization that uniquely relates the shape reconstruction of SOR to given a set of contour points and tangents. Finally, we provide the first annotated dataset for 2D detection, 3D pose and 3D shape of glassware and we show results comparable to category-based detection and localization of opaque objects without any training on the object shape.