Tell the Robot What You Want

What if you could command a robot just by talking to it?

This session introduces Strands Agents, an open-source framework that lets developers control physical sensors and actuators using natural language by exposing hardware capabilities as programmable agent tools through a unified interface. The agent interprets the request, selects the appropriate tools, and orchestrates execution. We explore a hybrid model where low-latency perception and actuation run locally on edge hardware, and higher-level reasoning and multi-step planning are delegated to cloud-based agents when needed. This preserves real-time responsiveness while enabling richer reasoning.

A live robot demonstration anchors the session. Using the EarthRover Mini, attendees will see how an instruction such as “drive toward the doorway and describe what you see” moves from conversation to perception, tool selection, bounded motion commands and real-world physical action.