Uli Kaya is a foosball-playing robot with voice conversation capabilities. This robot is connected to the fine tuned LLM, enabling it to conduct presentations and answer questions.
The capabilities of Uli Kaya are implemented with Uli SDK applications, which communicate between the two on board Nvidia Jetson devices. The Uli SDK applications form a subsystem with two nodes in the infrastructure.
The underline navigation, sensor perceptions, and drives are performed through the ROS modules, integrated with Uli SDK applications through auto generated python bindings.
The ROS topics are routed through the Data Topic Services of the Uli SDK for viewing and logging. This eliminates the need to set up environments for ROS RVIZ.
Here are the software blocks of the Uli Kaya:
Core Functional Capabilities:
- Maneuverability: omnidirectional movement capability including 360-degree zero-radius turning.
- Teleoperation: remote control via gamepad joysticks.
- Autonomous Navigation: independent movement capability.
- Obstacle and Cliff Avoidance: detection and evasion of obstacles and drop-offs
- Proximity Sensing: range detection across frontal, lateral, and rear aspects.
- Object Tracking and Acquisition: ability to track and acquire specific objects, such as a foosball
- Mechanical eyes produce expressions: directed gaze (right, left, forward), aperture adjustments (open, wide open, closed), and blinking.
- Motorized pusher: kick the foosball.
- Voice Conversation: fine tuned LLM with private information.
Specifications:
- Dual onboard computing systems: one Nvidia Jetson Orin and one Jetson Nano.
- One Arduino Uno microcontroller with a Pulse Width Modulation (PWM) Shield.
- Wireless connectivity.
- Capability to accommodate up to three DeWALT 20V 6AH batteries.
- Four range sensors.
- Two HD cameras.
- One Intel RealSense 3D camera.
- ReSpeaker V2 microphone array designed for far-field voice capture and Acoustic Source Localization.
- Two audio speakers.
- Operational speed of 0.3 meters per second.
- Unit weight of 8 pounds without battery.
- Overall dimensions: 200mm x 220mm x 360mm (height).