Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study

Over the past several years there has been a considerable research investment into learning-based approaches to industrial assembly; but despite significant progress these techniques have yet to be adopted by industry. We argue that it is the prohibitively large design space for Deep Reinforcement Learning (DRL); rather than algorithmic limitations per se; that are truly responsible for this lack of adoption. Pushing these techniques into the industrial mainstream requires an industry-oriented paradigm which differs significantly from the academic mindset. In this paper we define criteria for industry-oriented DRL; and perform a thorough comparison according to these criteria of one family of learning approaches; DRL from demonstration; against a professional industrial integrator on the recently established NIST assembly benchmark. We explain the design choices; representing several years of investigation; which enabled our DRL system to consistently outperform the integrator baseline in terms of both speed and reliability. Finally; we conclude with a competition between our DRL system and a human on a challenge task of insertion into a randomly moving target. This study suggests that DRL is capable of outperforming not only established engineered approaches; but the human motor system as well; and that there remains significant room for improvement. Videos can be found on our project website: https://sites.google.com/view/shield-nist.