PhD Thesis
Epigenetic learning of autonomous behaviours in a society of agents
Humans and robots are autonomous agents acting within the constraints of the physical world. However, the intelligence and autonomy of humans is far superior to that of machines. Inspired by psychology and neurosciences, developmental robotics aims to give artificial agents the ability to adapt, learn and develop autonomously, in order to reach or even exceed the capabilities of humans. Many research fields are involved in the improvement of sensorimotor skill training, memory systems, emergent representations of symbols and languages, motivational systems, and the development of many learning strategies ranging from exploration to imitation and social learning.
However, most of these research projects are focused on a very specific and limited task. Few of them aim to bring together all aspects of embodied intelligence, from the initial development of behaviours to the interactions with other intelligent agents. There is therefore a real need to study which underlying structures can unify this heterogeneity of goals and methods in a perpetually evolving system.
Our goal is to provide such a structure, capable of learning sensorimotor skills as well as more complex skills that go beyond simple reactive behaviour. The main contribution of this thesis is a hierarchical architecture using modular properties to achieve cumulative skill learning, namely MIND. In MIND, sensory information and coordination commands between skills are both treated as signals, using a connectionist inspired approach.
Starting from preliminary work on social specialization in multi-agent systems, we conduct a series of experiments using a MIND hierarchy to accumulate behaviours, from simple sensorimotor behaviours to social behaviours. We first build complex behaviours based on simple reactive behaviours, then integrate simple memory systems with complex behaviours, and finally use these memory systems to learn social behaviours that replicate our initial model of social specialization.
We show that such an architecture is capable of managing the heterogeneity of the behaviours to be learned and the systems to be coordinated. The use of a connectionist approach, a signal-based system, as the underlying architecture made learning both motor control and decision behaviour possible, and also lead to the emergence of memory representations.
Beyond the benefits of MIND as a support for designing developmental agents, our work shows the feasibility of continuous development and the advantages of embodiment in grounding the emergent behaviour, which supports developmental robotics as an approach to general purpose AI.
MIND architecture
A hierarchical representation of behaviour supporting open ended development and progressive learning for artificial agents (Autonomous Robots 2021)
In the perspective of open ended or lifelong development, we place a crucial importance on the fact that the current desired final behaviour can be an element for the future creation of one, or even several, behaviours of greater complexity, whose purpose can't be anticipated.
In this paper, we present a supporting control architecture for an artificial agent designed to learn behaviours from a curriculum established by a human instructor. A curriculum is a set of independent subtasks of increasing complexity covering the different aspects of a desired behaviour.
We propose the MIND (Modular Influence Network Design) architecture which encapsulates sub behaviours into modules and combines them into a hierarchy reflecting the modular and hierarchical nature of complex tasks, and allows for the preservation and re-usability of acquired skills.
Une représentation hiérarchique de comportements agents pour l'apprentissage progressif et continu (JFSMA 2019)
Dans la perspective d'un développement ouvert et continu il est cruciale qu'un comportement final souhaité à un instant donné puisse être un élément pour la création future de comportements plus complexes, dont le but ne peut être anticipé.
Nous proposons MIND (Modular Influence Network Design), une architecture de contrôle pour agent artificiel conçue pour apprendre des comportements à partir d'un Curriculum établi par un instructeur humain. MIND encapsule les comportements dans des modules et les combine en une hiérarchie reflétant la nature modulaire et hiérarchique des tâches complexes, et permet la préservation et la réutilisation des compétences acquises.
Émergence de comportements collectifs basée sur l’apprentissage progressif individuel (JFSMA 2020)
Dans la perspective d'un développement ouvert et continu il est cruciale qu'un comportement final souhaité à un instant donné puisse être un élément pour la création future de comportements plus complexes, dont le but ne peut être anticipé.
Nous proposons MIND (Modular Influence Network Design), une architecture de contrôle pour agent artificiel conçue pour apprendre des comportements à partir d'un Curriculum établi par un instructeur humain. MIND encapsule les comportements dans des modules et les combine en une hiérarchie reflétant la nature modulaire et hiérarchique des tâches complexes, et permet la préservation et la réutilisation des compétences acquises.
Multi agent simulations
CogLogo: une implémentation de MetaCiv pour NetLogo (JFSMA 2019)
CogLogo est une extension pour NetLogo qui implémente les principes de MetaCiv. Le but est de proposer un framework pour la modélisation du système de décision d'un agent social intégrant un mécanisme de renforcement.
MetaCiv: A MASQ-based framework for simulating human societies (Unpublished)
In this paper we propose MetaCiv, a multi-agent based generic framework, that uses the MASQ meta-model to implement complex social systems. First we review the fundamental elements of MASQ and its organizational and social aspects that could be of interest for modelers in social sciences. Secondly, we present the main components of MetaCiv with a focus on its architecture that blends into a single model reactive, cognitive, organizational and reified cultural ingredients. We illustrate the use of MetaCiv through a simple example of goods exchange in the context of the shift from agriculture to craftwork with CogLogo, a NetLogo implementation of Metaciv.