Simulation#
Simulation serves as the digital proving ground for the Robotics AI Suite. By accurately modeling kinematics, multi-body rigid physics, environmental collisions, and synthetic sensor streams, simulation allows developers to develop, benchmark, and validate complex autonomous systems in a risk-free virtual environment before deploying to physical hardware.
The suite standardizes on Gazebo Harmonic as its primary 3D robotics simulation platform for ROS 2 Jazzy on Canonical Ubuntu 24.04 LTS, while also integrating MuJoCo for high-frequency Embodied AI and manipulation dynamics.
Supported Simulation Frameworks#
The standard 3D simulation platform for the Robotics AI Suite. Provides realistic rigid-body dynamics (DART physics engine), sensor plugins (RGB-D depth cameras, LiDAR, IMU), material properties, and rich 3D warehouse and industrial work cell environments.
ros_gz)Bidirectional communication bridge connecting Gazebo transport with the ROS 2 message bus. Transparently translates sensor streams, clock signals, joint states, and actuator commands between the simulator and ROS 2 nodes.
High-performance contact-dynamics simulator utilized for Embodied AI, Humanoid robot control, and imitation learning workflows such as Model Predictive Control (OCS2/MPC), Action Chunking with Transformers (ACT), and Diffusion Policies.
Physics-free execution mode available in driver and control packages (such as Universal Robots and MoveIt 2 Servo). Allows instantaneous testing of state machines, communication logic, and trajectory generation without physics or rendering overhead.
Architecture & Core Concepts#
flowchart TD
subgraph Simulation_Engine["Simulation Engine (Gazebo / MuJoCo)"]
Physics["Rigid Body & Contact Physics\n(DART / ODE / MuJoCo)"]
World["World Models & Environments\n(Warehouse, Worktable, Obstacles)"]
Sensors["Sensor Plugins\n(RealSense RGB-D, 2D/3D LiDAR, IMU)"]
Clock["Simulation Clock\n(/clock)"]
end
subgraph Bridge["Communication Bridge"]
RosGz["ROS-Gazebo Bridge (ros_gz)\nMuJoCo ROS Bindings"]
end
subgraph ROS2_Stack["ROS 2 Application Stack (Jazzy)"]
Perception["Perception & OpenVINO™\n(Object Detection, SLAM, Point Clouds)"]
Navigation["Navigation & Planning\n(Nav2, ITS Planner, FastMapping)"]
Manipulation["Manipulation & Control\n(MoveIt 2, MoveIt 2 Servo, MPC)"]
Viz["Visualization & Monitoring\n(RViz2, Foxglove)"]
end
Physics --> Sensors
Clock --> RosGz
Sensors --> RosGz
RosGz --> Perception
RosGz --> Navigation
Navigation --> RosGz
Manipulation --> RosGz
ROS2_Stack --> Viz
Simulation Time Synchronization (/clock)#
In simulation, wall-clock time differs from virtual physics time. Gazebo publishes simulation time on the /clock topic.
All ROS 2 nodes in the application stack must run with simulation time enabled:
ros2 launch <package_name> <launch_file>.py use_sim_time:=true
When use_sim_time:=true is configured:
Message timestamps, TF2 coordinate transforms, and trajectory interpolation align with physics simulation steps.
ROS 2 performance profiling tools (such as
ros-kpiand the Robotics System Profiler) evaluate latency and throughput against simulation clock timestamps rather than host wall-clock time.
Headless vs. Graphical Execution#
Robotics AI Suite simulation packages support decoupling backend physics execution from graphical rendering:
Headless Mode (
gui:=false): Runs only the simulation server (gz-server/gz sim -s). This minimizes CPU and GPU usage, making it ideal for automated CI/CD testing, headless edge servers, and high-throughput benchmarking.Interactive Graphical Mode (
gui:=true): Launches the Gazebo 3D rendering client alongside RViz2, enabling real-time visual inspection of robot behavior, collision bounding boxes, and camera perspectives.
Synthetic Sensor Streams#
Simulated robots in the suite publish synthetic sensor feeds matching real hardware interfaces:
RealSense Depth Cameras: Color video (
/camera/color/image_raw), depth images (/camera/depth/image_rect_raw), camera intrinsics, and aligned 3D point clouds (/camera/depth/color/points).2D/3D LiDAR: Planar scans (
sensor_msgs/LaserScan) and full volumetric point clouds (sensor_msgs/PointCloud2) for mapping and obstacle detection.Odometry and Joint Feedback: Wheel encoders (
nav_msgs/Odometry) and manipulator joint positions (sensor_msgs/JointState).
Installing Gazebo Harmonic#
If you installed the Robotics AI Suite using the Express Setup or Image Composer Tool, Gazebo Harmonic and simulation bridges are pre-installed.
For manual installation on Canonical Ubuntu 24.04 LTS:
sudo apt-get update
sudo apt-get install -y curl lsb-release gnupg
# Add the official Open Source Robotics Foundation (OSRF) repository
sudo -E curl https://packages.osrfoundation.org/gazebo.gpg --output /usr/share/keyrings/pkgs-osrf-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/pkgs-osrf-archive-keyring.gpg] https://packages.osrfoundation.org/gazebo/ubuntu-stable $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/gazebo-stable.list > /dev/null
# Install Gazebo Harmonic and the ROS 2 Jazzy bridge
sudo apt-get update
sudo apt-get install -y gz-harmonic ros-jazzy-ros-gz
Verify your Gazebo installation:
gz sim --version
Simulation Tutorials#
The Robotics AI Suite provides pre-built simulation workflows across mobile robots, industrial manipulators, and humanoid platforms:
Introduction to simulating mobile robots and material-handling cells in Gazebo Classic and Gazebo Harmonic.
Simulate a full autonomous exploration pipeline with TurtleBot3 Waffle RGB-D, RTAB-Map SLAM, and Nav2 in Gazebo.
Simulate a Universal Robots UR5e arm, Robotiq gripper, and RealSense camera with MoveIt 2 Servo in Gazebo.
Coordinate two UR5 arms and a mobile robot on a conveyor line with MoveIt 2 and Nav2.
Simulate dual-arm manipulation and ACT imitation learning with OCS2 MPC in the MuJoCo physics engine.
Validate adaptive DBSCAN person detection and target tracking using simulated LiDAR and depth data.