Inspired by the functioning of the human brain and based on a biological mechanism called neuromodulation, it allows intelligent agents to adapt to unknown situations

Artificial Intelligence (AI) has enabled the development of high-performance automatic learning techniques in recent years. However, these techniques are often applied task by task, which implies that an intelligent agent trained for one task will perform poorly on other tasks, even very similar ones. To overcome this problem, researchers at the University of Liège (ULiège) have developed a new algorithm based on a biological mechanism called neuromodulation. This algorithm makes it possible to create intelligent agents capable of performing tasks not encountered during training. This novel and exceptional result is presented this week in the magazine PLOS ONE.

Despite the immense progress in the field of AI in recent years, we are still very far from human intelligence. Indeed, if current AI techniques allow to train computer agents to perform certain tasks better than humans when they are trained specifically for them, the performance of these same agents is often very disappointing when they are put in conditions (even slightly) different from those experienced during training.

The human being is capable of adapting to new situations very effectively by using the skills he has acquired throughout his life. For example, a child who has learned to walk in a living room will quickly learn to walk in a garden as well. In such a context, learning to walk is associated with synaptic plasticity, which modifies the connections between neurons, while the rapid adaptation of walking skills learned in the living room to those needed to walk in the garden is associated with neuromodulation. Neuromodulation modifies the input-output properties of the neurons themselves via chemical neuromodulators.

Synaptic plasticity is the basis of all the latest advances in AI. However, no scientific work has so far proposed a way to introduce a neuromodulation mechanism into artificial neural networks. This quite exceptional result, described this week in the journal PLOS ONE, is the result of an extremely fruitful collaboration between neuroscientists and artificial intelligence researchers at the University of Liège developing intelligent algorithms: two Ph.D. students, Nicolas Vecoven and Antoine Wehenkel, as well as two professors, Damien Ernst (specialist in artificial intelligence) and Guillaume Drion (neuroscientist).

These ULiège researchers have developed a completely original artificial neural network architecture, introducing an interaction between two sub-networks. The first one takes into account all the contextual information concerning the task to be solved and, on the basis of this information, neuromodule the second subnetwork in the manner of the brain's chemical neuromodulators. Thanks to neuromodulation, this second sub-network, which determines the actions to be performed by the intelligent agent, can therefore be adapted extremely quickly to the current task. This allows the agent to efficiently solve new tasks.

This innovative architecture has been successfully tested on classes of navigation problems for which adaptation is necessary. In particular, agents trained to move towards a target, while avoiding obstacles, were able to adapt to situations in which their movement was disrupted by extremely variable wind directions.

Prof. Damien Ernst says, "The novelty of this research is that, for the first time, cognitive mechanisms identified in neuroscience are finding algorithmic applications in a multi-tasking context. This research opens perspectives in the exploitation in AI of neuromodulation, a key mechanism in the functioning of the human brain."

---

---