Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Robotic Simulation in Real World

almo
October 08, 2022

Robotic Simulation in Real World

Introduction to simularion in robotics, presenting not only MuJoCo, a general purpose physics engine, aiming to facilitate research and development in robotics, biomechanics, graphics and animation, machine learning, but also its applications.

In particular, the presentation introduces:
- OpenAI Gym, a standard API for reinforcement learning, and a diverse collection of reference environments
- DeepMind dm_control, the software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.
- MoCapAct, which uses motion capture (MoCap) clips to learn low-level motor skills for the "CMU Humanoid" from the dm_control package

almo

October 08, 2022
Tweet

More Decks by almo

Other Decks in Technology

Transcript

  1. About me… Andrés-Leonardo Martínez-Ortiz a.k.a almo, holds a PhD on

    Software, Systems and Computing and a Master on Computer Science. Based on Zurich, almo is a member of the Google Machine Learning Site Reliability Engineering team, leading several programs aiming for reliability, efficiency & convergence. He is also a member of IEEE, ACM, Linux Foundation and Computer Society. @davilagrau almo.dev almo
  2. About this talk… Feynman Effect • What is Simulation? •

    Why do robotics researchers do simulation? • How do robotics researchers do simulations?
  3. A simulation is the imitation of the operation of a

    real-world process or system over time. Simulations require the use of models; the model represents the key characteristics or behaviors of the selected system or process, whereas the simulation represents the evolution of the model over time. Wikipedia, accessed Oct 2022.
  4. MuJoCo • MuJoCo stands for Multi-Joint dynamics with Contact. •

    It is a general purpose physics engine (Robotics, biomechanics, graphics and animation, machine learning) • It is a C/C++ library with a C API, including an interactive visualization with a native GUI, rendered in OpenGL. It also offer a wrapper in Python. • Scene descriptions (models) are describe in the native MJCF language (an XML file format). • 3D rendering helps the user gain insight into the physics simulation • Integrated/Supported by OpenAI Gym, PyBullet and others
  5. Modelling Elements • Inspired by the idea of Cascading Style

    Sheets (CSS) inlined in HTML. • Native 3D visualizer provides rendering, textures, etc… • Modelling ropes, cloth, and deformable 3D objects. • The positions, orientations and inertial properties are inferred. • Modelling elements: ◦ Options ◦ Assets ◦ Kinematic Tree ◦ Stand-alone elements.
  6. Modelling Elements: Options Established by default, but customizable using the

    model file. Can be updated dynamically. • mjOption: physics simulation • mjVisual: visualization options (non OpenGL options) • mjStatistic: average body mass, spatial extent of the model etc. Photo by Victoriano Izquierdo on Unsplash
  7. Modelling Elements: Assets Asset changes the properties of t model

    elements. • Mesh: OBJ files and binary STL • Skin: meshes whose shape can deform at runtime. • Height fields: model terrain maps. • Textures: only be referenced from another asset (namely material) • Material: control the appearance of geoms, sites and tendons. Photo by Nick Nice on Unsplash
  8. Modelling Elements: Kinematic Tree System state represented in joint coordinates

    and the bodies are explicitly organized into kinematic trees • Body: Bodies have mass and inertial properties but do not have any geometric properties. • Joints: are defined within bodies, creating motion degrees of freedom (DOFs) between the body and its parent Photo by Nick Nice on Unsplash
  9. Modelling Elements: Kinematic Tree System state represented in joint coordinates

    and the bodies are explicitly organized into kinematic trees • Geom: 3D shapes attached to the bodies. • Camera: There is always a default camera, but multiple cameras can be defined in a model. • Lights: full lighting model in OpenGL, including Photo by Nick Nice on Unsplash
  10. Example (C++) Loading file (MJCF or URDF) mjModel contains the

    model description (constant) Getting preallocated and internally managed stack, mjData contains all dynamic variables and intermediate results. Simulation: input(m,d) - output(d)
  11. mjModel MuJoCo Simulation Workflow XML File User defined Mujoco App

    (C++) Mujoco Library OpenGL mjData Compilation time Execution time Function 1 Function 2 Function 2 Function N
  12. Generalized Coordinates • Suitable for rigid body in 3D space

    • Joins add degrees of freedom • Forward Kinematic Images from Wikipedia
  13. OpenAI Gym • Python environment for developing and testing learning

    agenda • Suitable for Reinforcement Learning (RL) but… • It can also be used for training Deep Learning algorithms • Implementation of a simple RL loop
  14. DeepMind dm_control • Collection of Python libraries and task suites

    for reinforcement learning agents in an articulated-body simulation. • A MuJoCo wrapper provides convenient bindings to functions and data structures. • PyMJCF and Composer libraries enable procedural model manipulation. • Control Suite offers a set of tasks with standardised structure, intended to serve as performance benchmarks. • Locomotion framework provides high-level abstractions and examples of locomotion tasks.
  15. Application III What would it take to get humanoid, bipedal

    robots to dance like Mick Jagger? bit.ly/3MgkG3u
  16. MoCapAct - Motion Capture with Actions • MoCapAct is a

    large library of pre-trained humanoid control models along with enriched data for training new ones. • Based on using motion capture (MoCap) data. • Aiming to train AI agents with humanoid morphology to match human performance across the entire diversity of human motion. • Repositories such as CMU Motion Capture Dataset contain hours of clips for just about any common motion of a human body. • Based on dm_control* (MuJoCo) *DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.
  17. Use case II Inspired by sentence completion from NLP, we

    train a GPT policy to perform motion completion
  18. Next steps: Robot Benchmark • Program simulated robots online. •

    Open source • From middle school to PhD • Python and other languages coming soon! https://robotbenchmark.net
  19. Others Simulation Environment Simulator Pros Gazebo Opensource, ROS, sponsored by

    the Open Source Robotic Foundations, Integrated w/ 4 physics engines PyBullet Opensource, Python, Blender. MuJoCo Models accepted Webots Opensouce, C++, Python, ROS, Matlab, OpenStreetMap