2nd Workshop on Closing the Reality Gap in Sim2Real Transfer for Robotics
Full Day Workshop at R:SS 2020 (Corvallis, Oregon, USA), 12th July
< Back to Main Page
2020-07-14: Post-Workshop Update 2. We have uploaded all recordings of both the debates and panel discussion: Videos
2020-07-13: Post-Workshop Update 1. We’d love to send a thanks out to all debaters, authors, and participants that helped make this workshop so great! Thanks folks! :) Also we’ve uploaded the poster teasers that the authors provided (see Accepted Papers).
| Debate 1:
| Debate 2:
What Is Sim2Real?
| Debate 3:
How to Sim2Real?
Physical simulation is an important tool for robotics. While simulation has been well-established for robotics education and integrated robot software testing for a long time, only recently the robotics community has made significant progress in transferring capabilities learned in simulation to reality, a concept termed Sim2Real transfer.
Sim2Real refers to a concept of transferring robot skills acquired in simulation to the real robotic system. Sim2Real draws its appeal from the fact that it is cheaper, safer and more informative to perform experiments in simulation than in the real world – yet, Sim2Real faces significant challenges.
In this workshop we invite well-known researchers to debate the state of the art and the impact of Sim2Real on robotics. The proposed workshop is the second edition of a workshop held at R:SS 2019. The R:SS 2019 workshop had the goal of surveying the state of the art in Sim2Real for robotics, with a focus on robotic manipulation, and featured invited talks from top researchers in the field of Sim2Real. In this year’s edition, we aim to review the actual progress in Sim2Real more critically by changing the workshop format: inspired by the ICRA 2019 robotic debates workshop, for this workshop we will invite top-researchers to participate in debates focussed on controversial key topics with a few spotlight presentations.
Invited Debaters [Top]
|Chris Atkeson (Robotics Institute, Carnegie Mellon University)|
|Anca Dragan (UC Berkeley)|
|Gregory Dudek (McGill University)|
|Dieter Fox (University of Washington and Nvidia Research)|
|Ken Goldberg (UC Berkeley and UCSF)|
|Abhinav Gupta (Robotics Institute, Carnegie Mellon University)|
|John Leonard (MIT)|
|Karen Liu (Georgia Tech)|
|Jan Peters (Technische Universität Darmstadt, MPI for Intelligent Systems)|
|Shuran Song (Columbia University)|
|Peter Welinder (OpenAI)|
|Martha White (University of Alberta)|
Debate Topics [Top]
In contrast to the R:SS 2019 edition, we changed the format from invited talks to debates. Each debate will focus on a key - rather controversial - statement regarding Sim2Real, and will be discussed by two proponents and two opponents of that statement. Each debate consists of short introductory pitches by the two sides, providing an opportunity to presenters to also highlight their work, followed by a moderated discussion. An example list of statements includes:
Why sim2real? - “Investing into Sim2Real is a waste of time and money“
What is sim2real? - “Sim2Real is old news. It’s just X (X=model-based RL, X=domain randomization, X=system identification, …)”
How to sim2real? - “For successful sim2real transfer, there is no alternative to accurate simulation.“
Debate Structure [Top]
We will adhere to the following debate structure (Total time per debate: 50 min)
- Introductions (5 min = 4 debaters x ~1 min each)
- Opening arguments (20 min = 4 debaters x 5 min each)
- Moderated discussion between debaters (15 min)
- Closing remarks (10 min = 4 debaters x 2.5 min each)
Debate Results [Top]
Through rigorous application of the scientific method, we’ve established absolutely waterproof statistics on how the debate changed people’s minds:
(Disclaimer: The results may not be statistically significant, the results may have been obtained from different populations, the results may contains nuts)
|Debate 1 - “Investing into Sim2Real is a waste of time and money”|
|Debate 2 - “Sim2Real is old news. It’s just X…”|
|Debate 3 - “Successful sim2real transfer necessitates accurate simulation”|
Call for Contributions [Top]
Participants are invited to submit extended abstracts (maximum 2 pages in length, excluding references). We encourage justified position papers (wrt. to the aforementioned debate points) but we allow other sim2real-related entries as long as they mention in the submission notes how they relate to one of the subjects (e.g. a work about a new visual grasping sim2real method might mention that it’s ONLY reporting sim2sim performance since their simulation is sufficiently detailed).
Accepted abstracts will be given the opportunity to record a short pitch video, and receive an online Q&A slot during the live session of the virtual workshop.
- Submissions must not exceed 2 pages in length, excluding references.
- Submissions must be in PDF following the IEEE style available from
- Submissions must include a short abstract. The extended abstract should go in the main section of the template and the short abstract in the abstract section.
- Anonymous submissions are not required.
Submission website: https://cmt3.research.microsoft.com/SIMREALRM2020
Important dates [Top]
- Submission Deadline: June 26th, 2020 AoE time (UTC-12)
- Notification of Acceptance: July 6th, 2020
July 1st, 2020
- Camera ready and Video submission: July 8th, 2020
- Workshop: July 12, 2020 (full day)
Accepted Papers [Top]
(ordered by title)
- “Accurate High Fidelity Simulations for Training Robot Navigation Policies for Dense Crowds using Deep Reinforcement Learning”
Jing Liang, Utsav Patel, Adarsh Jagan Sathyamoorthy, Dinesh Manocha
- “An Imitation from Observation Approach to Sim-to-Real Transfer”
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Garrett Warnell, Josiah Hanna, Peter Stone
- “Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap”
Eric Heiden, David Millard, Erwin Coumans, Gaurav Sukhatme
- “BlenderProc: Reducing the Reality Gap with Photorealistic Rendering”
Maximilian Denninger, Martin Sundermeyer, Dominik Winkelbauer, Dmitry Olefir, Tomas Hodan, Youssef Zidan, Mohamad Elbadrawy, Markus Knauer, Harinandan Katam, Ahsan Lodhi
- “CuNAS - CUriosity-driven Neural-Augmented Simulator”
Sharath Chandra Raparthy, Melissa Mozifian, Liam Paull, Florian Golemo
- “Continual Learning on Incremental Simulations for Real-World Robotic Manipulation Tasks”
Josip Josifovski, Mohammadhossein Malmir, Noah Klarmann, Alois Knoll
- “How to Sim2Real with Gaussian Processes: Prior Mean versus Kernels as Priors”
Rika Antonova, Akshara Rai, Danica Kragic
- “Learning Predictive Representations for Deformable Objects with Contrastive Estimation”
Wilson Yan, Ashwin Vangipuram, Pieter Abbeel, Lerrel Pinto
- “Learning to Manipulate Deformable Objects without Demonstrations”
Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, Pieter Abbeel
- “Learning to Walk without Dynamics Randomization”
Jeremy Dao, Helei Duan, Kevin Green, Jonathan Hurst, Alan Fern
- “Modular Latent Space Transfer with Analytic Manifold Learning”
Rika Antonova, Maksim Maydanskiy, Danica Kragic, Sam Devlin, Katja Hofmann
- “Necessity for More Realistic Contact Simulation”
- “On Assessing the Value of Simulation for Robotics”
Liam Paull, Anthony Courchesne
- “Online BayesSim for Combined Simulator Parameter Inference and Policy Improvement”
Rafael Possas, Fabio Ramos, Dieter Fox, Lucas Barcelos, Rafael Oliveira
- “Robust Sim2Real Transfer by Learning Inverse Dynamics of Simulated Systems”
Mohammadhossein Malmir, Josip Josifovski, Noah Klarmann, Alois Knoll
- “Sim2Real Learning of Vision-based Obstacle Avoidance for Robotic Manipulators”
Kefang Zhang, Tan Zhang, Jiatao Lin, Lv Bi
- “Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?”
Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lee, Manolis Savva, Sonia Chernova, Dhruv Batra
- “The Importance and the Limitations of Sim2Real for Robotic Manipulation in Precision Agriculture”
Carlo Rizzardo, SUNNY KATYARA, Miguel Fernandes, Fei Chen
Taking inspiration from the ICRA 2019 Debates on the Future of Robotics Research, the workshop will consist of three debates. Every debates focuses on a single key question and is structured as follows: Poster teasers related to debate topic (3-5 minutes each depending on number) Pitch by proponents (2 researchers), 15 minutes Pitch by opponents (2 researchers), 15 minutes Debate, 45 minutes
We will be soliciting poster submissions for the workshop. We will review the contributions by putting together a program committee with experts in the field. We will have two poster sessions during the coffee breaks along with teaser presentations.
We will conclude the workshop with a panel discussion focusing on the question on how Sim2Real and real-world experimentation can be combined in the best way to achieve the best of both worlds.
|Sebastian Höfer is an applied scientist at Amazon Robotics AI headed by Siddhartha Srinivasa. Before joining Amazon, he received his Ph.D. with Oliver Brock at Technische Universität Berlin.|
|Kostas Bekris is an Associate Professor at the Computer Science department of Rutgers University and an Amazon Scholar at the Amazon Robotics AI team headed by Siddhartha Srinivasa. He received his PhD with Lydia Kavraki at Rice University.|
|Ankur Handa is a senior research scientist at NVIDIA robotics lab run by Dieter Fox. Prior to that he was a research scientist at OpenAI. He received his Ph.D. with Dr. Andrew Davison and spent two years at University of Cambridge in Prof. Roberto Cipolla’s lab as a post-doctoral researcher.|
|Juan Camilo Gamboa is a PhD Candidate at the School of Computer Science at McGill University. His research focuses on model-based RL for control and navigation of underwater vehicles.|
|Florian Golemo is a postdoctoral fellow at Mila and ElementAI, working with Liam Paull, Aaron Courville, and Chris Pal on Sim2Real and 3D perception problems. He received his PhD from INRIA Bordeaux under supervision of Pierre-Yves Oudeyer.|
|Melissa Mozifian is a PhD Candidate at the School of Computer Science at McGill University. Her research focuses on deep reinforcement learning and transfer for control of robot manipulation tasks.|