Offload or Overload: A Platform Measurement Study of Mobile Robotic Manipulation Workloads

Technical Deep Dive: Offload or Overload: A Platform Measurement Study of Mobile Robotic Manipulation Workloads

Overview

This technical deep dive summarizes a recent study on the challenges of mobile robotic manipulation, a core capability for physical AI systems. The key contributions are:

1. Characterization of the computational demands of modern robotics workloads, including semantic mapping, navigation, and manipulation, across different onboard and offloaded GPU platforms.
2. Quantification of the tradeoffs between latency, accuracy, and power consumption for offloading these workloads to edge and cloud compute.
3. Exploration of the opportunities and limitations of sharing compute and network resources across a fleet of robots.

The study provides important insights to guide the design of effective inference systems for mobile robots.

Problem & Context

Mobile Robotic Manipulation

Mobile robotic manipulation combines key capabilities like perception, planning, navigation, and physical interaction. Recent breakthroughs in foundation models have enabled robots to operate in less structured, open-world environments beyond highly controlled factory settings.

However, these advanced robotics models come with significant computational demands. Robots equipped with onboard GPUs face tradeoffs between memory usage, task performance, and power consumption. Offloading workloads to edge or cloud platforms introduces new challenges around network latency and bandwidth.

Objectives

The study aimed to answer three key questions:

1. How severe are the tradeoffs between memory usage, task performance, and power consumption for onboard GPUs?
2. What are the effects of network latency and bandwidth on the execution time and accuracy of offloaded workloads?
3. What are the implications of sharing compute resources across a fleet of robots?

Answering these questions is critical for designing effective mobile robotic manipulation systems.

Methodology

Robotic Platforms

The researchers used three open-source robotic platforms to evaluate the workloads:

1. Stretch 3: A mobile manipulator with 7-DOF gripper, RGB-D cameras, LiDAR, and an Intel NUC 12 compute.
2. TurtleBot 4: A mobile robot with RGB-D camera and LiDAR, built on an iRobot Create 3 base.
3. Bimanual SO-101: A pair of 6-DOF robotic arms with wrist cameras, integrated with the LeRobot SDK.

Workloads

The study considered four key mobile robotic manipulation workloads:

1. Semantic Mapping (VLMaps): Continuously updating a 3D semantic map of the environment.
2. Embodied Question Answering (GraphEQA): Navigating to answer questions about the environment.
3. Multi-Step Manipulation ($\pi_{0.5}$): Executing a sequence of pick-and-place tasks.
4. Collision-Free Navigation (RTAB-Map & nvblox): Real-time obstacle detection and mapping.

These workloads capture the essential perception, planning, navigation, and manipulation capabilities required for mobile manipulation.

Compute & Network Infrastructure

The researchers evaluated these workloads on a range of onboard GPU options (Jetson Orin, Thor, Nano), edge GPU servers (DGX Spark, L4), and a cloud A100 VM. Connectivity included both Wi-Fi 6 and 5G networks.

Results

Onboard Compute Analysis

1. Memory Constraints: The full stack of workloads requires over 50GB of memory, exceeding the capabilities of smaller onboard GPUs like the Jetson Nano and 32GB Orin.
2. Performance Tradeoffs: Smaller onboard GPUs see significant slowdowns compared to cloud/edge GPUs:
   • VLMaps mapping time increased by up to 383% on the Orin.
   • GraphEQA answer time increased by 63% on the Orin.
   • $\pi_{0.5}$ manipulation accuracy dropped from 80% to 30% on the Orin.
3. Power Consumption: Larger onboard GPUs like the Thor can drain robot batteries 160% faster than smaller options.

These results indicate the limitations of onboard compute and motivate the need for offloading.

Offloading Challenges

1. Latency Effects: Even modest network latencies of tens of milliseconds can degrade task accuracy by over 10% for manipulation workloads.
2. Bandwidth Constraints: Naively offloading high-bandwidth video streams is infeasible, requiring aggressive compression that further reduces accuracy by up to 20%.

Offloading is not a panacea and requires carefully balancing GPU, connectivity, and compression tradeoffs.

Sharing Compute Resources

1. Batching Opportunities: Workloads involving large language models (like $\pi_{0.5}$ and GraphEQA) can benefit significantly from batching, with up to 3.55x speedup and 74.8% memory savings.
2. Statistical Multiplexing: Robot workloads exhibit periodic activity patterns, presenting opportunities for sharing edge/cloud GPUs.
3. Contention Challenges: Naive sharing can increase latency by 75-230% due to GPU scheduling. Careful throttling and QoS management is required.

Multiplexing compute and network resources across a fleet of robots introduces both opportunities and challenges that must be addressed.

Interpretation

The study highlights the growing computational demands of mobile robotic manipulation and the limitations of current onboard GPU options. While offloading can alleviate some of these constraints, it introduces new challenges around network latency and bandwidth that must be carefully managed.

The researchers advocate for a distributed robotics inference platform that can intelligently distribute workloads across robots, edge devices, and the cloud. Key desired properties of such a platform include:

1. Multiplexing compute resources across robots to leverage statistical multiplexing and batching opportunities.
2. Adaptive model selection to balance task accuracy and resource usage based on the current load.
3. Transparent cloud overflow to handle demand spikes and reduce over-provisioning of edge hardware.

Addressing these challenges is crucial for realizing the potential of physical AI systems powered by mobile robotic manipulation.

Limitations & Uncertainties

• The study focused on a limited set of robotic platforms and workloads. The generalization of the findings to a broader range of robots and tasks remains to be explored.
• The network characteristics, such as latency and bandwidth, were simulated and may not fully capture the complexity of real-world deployments.
• The study did not consider other system-level factors, such as thermal constraints, that may impact the performance and power consumption of onboard GPUs.

What Comes Next

The researchers plan to build a prototype of the proposed distributed robotics inference platform and further investigate the design challenges around workload scheduling, model adaptation, and cloud integration. Evaluating the platform in real-world deployment scenarios will be crucial to validating the study's findings and identifying additional practical considerations.