Algorithm allows robots to ‘operate on intuition’

Researchers at the University of Hertfordshire have developed a new algorithm that will allow robots to operate more intuitively, i.e. to make decisions based on their environment.

The principle is that, thanks to the algorithm, the robot agent creates its own objectives.

For the first time, the algorithm unifies different approaches to goal setting under a concept directly related to physics, and it further makes this computation transparent so that others can study and adopt it.

The principle of the algorithm is related to the famous chaos theory, because the method makes the agent “master of the chaos of the system dynamics”.

The study was published in the journal PRX LifeHerts researchers explored “motivation models” of robots that mimic the decision-making processes of humans and animals, even in the absence of clear reward signals.

The study presents artificial intelligence (AI) formulas that calculate a way for a robot to decide future actions without direct instructions or human intervention.

Daniel Polani, professor of computer science and lead author, explains: “From an applied perspective, this could mean, for example, getting a robot to play and manipulate objects on its own without being told to do so.

“This could improve the way robots learn to interact with humans and other robots by encouraging more ‘natural’ behaviors and interactions.”

“This has other applications, such as the survival behavior of semi-autonomous robots placed in situations where they are inaccessible to a human operator, such as in underground or interplanetary locations.”

In humans and animals, one theory assumes the existence of “intrinsic motivation,” where behaviors are solely determined by the interaction between the being and its environment rather than by specific learned rewards, such as food. This paper successfully translates this theory of “intrinsic motivation” into a theory that can be used by robotic agents.

Professor Polani adds: “This work is exciting because we can now implement a mechanism, similar to those that help humans and animals solve new problems without prior experience, in robots.

“We hope to build on this work to develop more humanoid robots with more intuitive processes in the future. This opens up a huge opportunity for more sophisticated robots with decision-making processes similar to ours.”

The theory on which this paper is based, called ’empowerment maximisation’, has been developed at Herts over many years. It suggests that by increasing the range of future outcomes, a robot will have better options also in the longer future. Importantly, this method replaces and therefore possibly overrides traditional reward systems (e.g. food cues).

Although empowerment maximization has shown promise, it is not yet fully understood or widely applied. Most studies have relied on simulations, while the careful calculation of the information needed for complex systems and theory remains a challenge.

However, this latest innovative research aims to explain why empowerment-based motivations can create behaviors similar to those of living organisms, potentially leading to more intrinsically motivated robots; and it further offers a significantly improved way to calculate these motivations.

According to Professor Polani, the next steps will be to use this revolutionary algorithm to enable robots to learn more about the world, by developing direct learning and identifying and refining new skills that would increase their value in real-world scenarios.