Credit: arXiv (2024). DOI: 10.48550/arxiv.2401.12202
A team of roboticists from New York University, in collaboration with AI colleague Meta, developed a robot capable of picking up designated objects in an unfamiliar room and placing them in a new designated location. In their article published on the arXiv preprint server, the team describes how the robot was programmed and how it performed when tested in several real-world environments.
Researchers have noted that visual language models (VLMs) have progressed greatly in recent years and have become very effective at recognizing objects based on linguistic prompts. They also pointed out that the robots’ skills have also improved: they can grab objects without breaking them, transport them to the desired locations and drop them off. But so far, little has been done to combine VLMs with trained robots.
This is exactly what the researchers tried for this new study, with a robot sold by Hello Robot. It features casters, a pole and retractable arms with hand clasps. The research team gave him a pre-trained VLM and nicknamed him OK-Robot.
They then transported it to 10 volunteer homes where they created 3D videos using an iPhone and fed them to the robot to give it an overall idea of the layout of a given home. They then asked him to perform some simple moving tasks: “move the pink bottle on the shelf to the trash can,” for example.
In total, they asked the robot to perform 170 such tasks – it managed to complete them successfully in 58% of cases. Researchers found they could improve its success rate by up to 82% by decluttering the workspace.
The research team points out that their system uses a zero-shot algorithm, meaning the robot was not trained for the environment in which it was working. They also suggest that the success rate obtained proves that VLM-based robotic systems are viable.
They suspect the success rate could be improved through adjustments and perhaps using a more sophisticated robot. They conclude by suggesting that their work could be the first step toward advanced VLM-based robots.
More information:
Peiqi Liu et al, OK-Robot: what really matters in integrating open knowledge models for robotics, arXiv (2024). DOI: 10.48550/arxiv.2401.12202
OK-Robot: ok-robot.github.io/
arXiv
© 2024 Science X Network
Quote: A robot that can pick up objects and drop them at the desired location in an unknown house (February 5, 2024) retrieved on February 5, 2024 from
This document is subject to copyright. Apart from fair use for private study or research purposes, no part may be reproduced without written permission. The content is provided for information only.
