Step-by-Step Teaching Guide

Step 1: Understanding the Core Problem — Why Onboard AI Has Limits

The Current Standard Approach

Today, most robots carry their AI "brain" physically onboard — meaning a GPU (Graphics Processing Unit) is wired directly to the robot.

Think of it like this:

[Robot Body] + [Onboard GPU] = Robot thinks and acts locally

The GPU processes all AI decisions right there on the robot itself.

Why This Becomes a Problem

As AI models grow larger and more sophisticated, keeping everything onboard creates serious constraints:

🔑 Key Insight: The robot's physical body and its AI brain are currently treated as one unit — but they don't have to be.

---

Step 2: What Does a Robot Actually Need to Do?

Breaking Down a Real Robot Task

The researchers studied mobile robotic manipulation — robots that move around and interact with objects. A representative task:

"Check for rubbish in the kitchen and put it in the trash."

This single task requires multiple AI capabilities working together:

TASK: Clean up rubbish

├── 1. SEMANTIC MAPPING & PLANNING
│       → Understand the environment
│       → Plan the route to the kitchen
│
├── 2. NAVIGATION
│       → Move safely through space
│       → Detect and avoid obstacles in real time
│
└── 3. MANIPULATION
        → Identify the rubbish
        → Pick it up
        → Place it in the trash

Each of these capabilities requires its own AI model, and each model demands significant computing power.

🔑 Key Insight: Even a simple-sounding task requires multiple heavy AI models running simultaneously. This multiplies the compute burden on the robot.

---

Step 3: What Happens When Onboard GPUs Are Tested?

The Research Findings — Measured Performance Gaps

The researchers tested these AI models across different hardware configurations. Here is what they found:

Finding 1: Mapping and Planning Slowed Dramatically

• Smaller onboard GPUs caused mapping and planning to slow by up to 383% compared to a powerful cloud GPU (A100)
• A robot that thinks slowly cannot react to a changing environment — imagine a robot that takes 4x longer to decide where to go while someone walks into its path

Finding 2: Navigation Became Dangerous

• Lighter onboard GPUs caused a 30% drop in timely obstacle detection
• Missing obstacles by 30% is not a minor inconvenience — it is a safety risk

Finding 3: Manipulation Accuracy Dropped by Half

• Even though manipulation models did not slow down as dramatically, the slowdown was still enough to drop task success rates by 50%
• A robot that fails half the time is not useful in real-world deployment

Finding 4: Some GPUs Simply Could Not Run the Stack

• Certain smaller onboard GPUs did not have enough memory to even load the full mobile manipulation AI stack
• The robot could not function at all

🔑 Key Insight: Onboard GPU limitations do not just reduce performance slightly — they can make robots unsafe, unreliable, or completely non-functional.

---

Step 4: The Solution — Inference Offloading

What Is Inference Offloading?

Inference = the moment an AI model processes input and produces an output (a decision or action)

Offloading = moving that computation away from the robot to a more powerful external computer

WITHOUT OFFLOADING:
[Robot] → [Onboard GPU processes AI] → [Robot acts]

WITH OFFLOADING:
[Robot] → [Sends data to Edge/Cloud GPU] → [Receives decision back] → [Robot acts]

Where Can Inference Be Offloaded?

There are two main destinations:

---

Step 5: The Benefits of Offloading — Measured Results

Benefit 1: Better Task Performance

When inference was offloaded to powerful GPUs like the A100:

• Object handover success rates increased substantially
• Navigation success rates improved significantly
• Robots could run larger, more capable AI models that simply would not fit onboard

Benefit 2: Dramatically Extended Battery Life

This is one of the most striking findings:

Onboard Jetson Thor GPU → Drains battery up to 160% faster
                                    ↓
Replace with Raspberry Pi 5 + Offloaded Inference → Battery life more than doubles

Practical meaning: A robot that previously needed recharging every few hours could now operate for twice as long between charges — simply by removing the onboard GPU and offloading the AI work.

Benefit 3: Scalability and Cost

• Cheaper, lighter hardware onboard means lower cost per robot
• Multiple robots can share powerful edge or cloud GPUs
• Easier to upgrade AI models without physically modifying each robot

---

Step 6: Understanding the Tradeoffs

Offloading is not without complexity. There is a tradeoff triangle to consider:

        PERFORMANCE
             △
             │
             │
NETWORK ─────┼───── GPU AVAILABILITY
LATENCY      │

The Key Tradeoffs

Network Latency

• Sending data to a remote GPU and receiving a response takes time
• If the network is slow or unreliable, the robot may receive decisions too late
• This is why the article shows that high inference latency causes task failure (the tape handover example)

Network Bandwidth

• Robots generate large amounts of sensor data (cameras, lidar, etc.)
• Transmitting all of this continuously requires significant bandwidth

GPU Availability

• Edge and cloud GPUs must be available when the robot needs them
• Multiple robots sharing resources requires careful scheduling

🔑 Key Insight: Offloading is not simply "always better." It requires thoughtful system design that accounts for network conditions and resource availability.

---

Step 7: The Toolset — How Offloading Is Made Practical

The Challenge of Implementation

Offloading sounds logical in theory, but implementing it requires:

• Packaging AI models so they can run remotely
• Routing data between robot and GPU
• Managing multiple robots and GPUs simultaneously
• Handling failures gracefully

The Solution: Kubernetes-Based Tooling

The researchers built a toolset that automates this complexity. Here is how it works conceptually:

STEP 1: Developer specifies WHAT to offload
        (using simple declarative instructions)
                    ↓
STEP 2: Toolset automatically CONTAINERIZES the AI workload
        (packages it so it can run anywhere)
                    ↓
STEP 3: Kubernetes DEPLOYS the container to the right GPU
        (edge or cloud, based on availability and policy)
                    ↓
STEP 4: Robot communicates with the remote container
        for all AI inference

What Is Kubernetes?

Think of Kubernetes as an intelligent traffic controller for computing workloads:

• It knows which computers have available resources
• It automatically routes work to the right place
• It handles failures by rerouting to backup resources
• It scales up or down based on demand

Integration With Existing Robot Tools

The toolset connects with:

• ROS2 — the standard robot operating system
• LeRobot — a popular robotics AI framework
• Robotic simulators — for testing before real-world deployment

---

Step 8: Putting It All Together — The Big Picture

Conceptual Summary

OLD MODEL:
┌─────────────────────────────┐
│  ROBOT                      │
│  ┌──────────┐ ┌──────────┐  │
│  │ Body &   │ │ Onboard  │  │
│  │ Sensors  │ │   GPU    │  │
│  └──────────┘ └──────────┘  │
│  (heavy, expensive, limited) │
└─────────────────────────────┘

NEW MODEL:
┌──────────────┐     Network      ┌─────────────────┐
│  ROBOT       │ ←────────────→  │  EDGE / CLOUD   │
│  ┌─────────┐ │                 │  ┌───────────┐   │
│  │ Body &  │ │                 │  │ Powerful  │   │
│  │ Sensors │ │                 │  │    GPU    │   │
│  │ + RPi5  │ │                 │  └───────────┘   │
│  └─────────┘ │                 │  (shared, scalable│
│  (light,     │                 │   upgradeable)   │
│  efficient)  │                 └─────────────────┘
└──────────────┘

Why This Matters for the Future

---

Summary: Key Takeaways

✅ 1. Running AI inference exclusively onboard robots creates real, measurable limitations in performance, battery life, and scalability

✅ 2. Offloading inference to edge or cloud GPUs can improve task success rates, enable larger models, and more than double battery life

✅ 3. Offloading involves tradeoffs — particularly around network latency and bandwidth — that must be carefully managed

✅ 4. Kubernetes-based tooling can automate the complexity of distributed inference, making offloading practical for real-world robot deployments

✅ 5. This architectural shift — separating the robot's physical body from its AI compute — represents a fundamental rethinking of how physical AI systems should be designed