NVIDIA Isaac ROS 5.0 Adds AI Agent Workflows and Jetson Support for Robotics

NVIDIA Isaac ROS 5.0 adds AI agent workflows to a GPU accelerated robotics toolkit, extending support from development tasks to model tuning, perception and robot deployment. Released at ROSCon in Toronto, the update supports ROS Lyrical and Ubuntu 24.04 and runs across NVIDIA Jetson hardware, from Orin Nano to Thor. The practical pitch is a shared path for developers and AI agents to build robotics applications with familiar open source tools, then run them on robots.

ROS is an open framework from Open Robotics used to build and connect robot software. NVIDIA says nearly 1.3 million ROS users can use Isaac ROS packages to add accelerated computing, physical AI models and production ready libraries to their applications. Isaac ROS itself is a collection of GPU accelerated packages built on ROS; the release remains free and open source.

Agent workflows reach into perception

Isaac ROS 5.0 adds reusable skills for setup and manipulation, along with documentation structured to help AI agents work with Isaac ROS tools. The aim is to let an agent take a developer’s intent and help turn it into a working application. That could reduce repetitive setup and coding work, though the announcement doesn’t quantify time saved or describe how much supervision each workflow needs.

One skill targets FoundationStereo, a stereo perception model. It helps an agent adapt the model to a developer’s cameras, environment and application, with the goal of improving perception for a particular sensor setup. Another makes pick and place available as a standalone skill, combining detection, depth estimation and pose output. That packaging gives teams a way to use the workflow beyond Isaac ROS.

FoundationPose, NVIDIA’s model for object pose estimation and tracking, now has an agent ready inference library. NVIDIA says robots can use it to perceive and track object position and orientation up to 5.5 times faster. The release provides no further benchmark conditions, so the figure should be read as NVIDIA’s reported result rather than a universal speedup across hardware and workloads.

Platform changes underpin the new workflows. NVIDIA says it worked with the Open Source Robotics Alliance on a standard data handling interface for ROS Lyrical. The interface is intended to help robotics software use different computing hardware efficiently, including GPUs. NVIDIA describes CUDA as a working example of acceleration through this shared interface.

Projects apply the tools to working systems

AgenticROS, an open source project sponsored by 3D perception company RealSense, connects Isaac ROS with NVIDIA Nemotron models and NemoClaw blueprints so agents can interact with ROS based robots. RealSense is also adapting its AI focused stereo depth cameras, including RealSense D585 Pro, and its open source software development kit for Isaac ROS and Jetson Thor. The intended applications include perception, navigation and manipulation.

Intrinsic’s Open Machine Tending Solution is a reference application for tending CNC machines, included in the open source Intrinsic Core suite. It includes NVIDIA FoundationPose for object registration, tracking and pose estimation. Its perception pipeline is designed to let robots detect and handle changing parts, reducing dependence on fixed physical fixtures and specialized integration.

Other ecosystem examples span development, testing and robot hardware. Seeed Studio combines Isaac ROS with its reBot Arm for perception, spatial understanding and motion planning on Jetson Thor. Magna uses Isaac ROS for GPU accelerated perception and synchronized data collection, alongside Isaac GR00T model deployment and Isaac Sim hardware in the loop testing.

Prefix.dev’s Pixi package manager helps create repeatable development environments that combine ROS and CUDA. Foxglove tools support visualization and debugging of live ROS applications, including 3D topics, nvblox meshes and rosbags. Flexiv is connecting Isaac ROS with its Rizon 4 adaptive robot, giving developers a route from Isaac Sim testing to physical deployment.

Ekumen uses Isaac ROS packages with existing ROS and Nav2 software for precision docking, 3D obstacle detection, visual localization and motion planning, validating applications in Isaac Sim. Its reported GPU path planning time for a warehouse arm using isaac_ros_cumotion is roughly 2 to 5 milliseconds. Ouster integrates Stereolabs ZED stereo cameras with Isaac ROS for accelerated perception, including object detection, mapping and navigation.

From development to on robot compute

Isaac ROS supports deployment on Jetson devices, which bring ROS, perception, navigation, models and application logic onto the robot. Support across Jetson Orin Nano and Jetson Thor gives developers a hardware range for different workloads, though the announcement does not specify performance, cost or power comparisons between the devices.

Mentee Robotics uses Isaac ROS for perception and AI in its MenteeBot humanoid, with a shared software foundation across Jetson Orin and Jetson Thor. Universal Robots has incorporated Isaac ROS into its AI Accelerator software development kit, aiming to help integrators add perception and motion capabilities without building complex robotics software from scratch. The company says the system can handle parts that aren’t precisely positioned, reducing reliance on fixtures.

ROBOTIS is integrating Isaac ROS into its AI Worker robot for vision guided picking, placing and alignment. FieldAI is bringing its robot foundation models, which can run without cloud connectivity, to Isaac ROS on Jetson to make greater use of GPU acceleration. Noble Machines is using Isaac ROS on Jetson to build industrial robots with ready to use perception and AI capabilities.

These examples show the release’s two main bets: agents can help assemble and adapt robotics software, and a common accelerated stack can carry that software onto devices at the edge. How much the agent skills simplify real projects will depend on the sensors, robot hardware and integration work each application requires. Isaac ROS 5.0 is available now through GitHub.