Leadership in enabling and industrial technologies - ICT - CONBOTS

CONBOTS

Introduction

CONBOTS is a European project which aims to help humans to learn fine sensorimotor skills (handwriting and music learning) by assisting them with robots, artificial agents and augmented reality. CONBOTS stands for CONnected through roBOTS: physically coupling humans to boost handwriting and music learning.

From a parent coordinating movements to help a child learn to walk, to a violinist training a concerto, humans rely on physical interaction to learn from each other and from the environment. Building on a strongly multidisciplinary foundation with an integrated approach, CONBOTS proposes a paradigm shift that aims to augment handwriting and music learning through robotics, by creating a physically interacting robotic platform connecting humans in order to facilitate the learning of complex sensorimotor tasks.

CONBOTS has a duration of 42 months (3.5 years) and is coordinated by Professor Domenico Formica of the Universita Campus Bio Medico Di Roma (Italy). The project has seven partners spread over Italy, Serbia, the United Kingdom, Israel and Belgium.

1 UNIVERSITA CAMPUS BIO MEDICO DI ROMA (Italy)
2 IMPERIAL COLLEGE OF SCIENCE TECHNOLOGY AND MEDICINE (United Kingdom)
3 SCUOLA SUPERIORE DI STUDI UNIVERSITARI E DI PERFEZIONAMENTO SANT' ANNA (Italy)
4 ARVRTECH DOO NOVI SAD (Serbia)
5 UNIVERSITEIT GENT (Belgium)
6 IUVO SRL (Italy)
7 IBM ISRAEL - SCIENCE AND TECHNOLOGY LTD (Israel)


Project description

From a parent coordinating movements to help a child learn to walk, to a violinist training a concerto, humans rely on physical interaction to learn from each other and from the environment. Building on a strongly multidisciplinary foundation with an integrated approach, CONBOTS proposes a paradigm shift that aims to augment handwriting and music learning through robotics, by creating a physically interacting robotic platform connecting humans in order to facilitate the learning of complex sensorimotor tasks.

The newly designed platform will combine four enabling technologies: i) compact robotic haptic devices to gently interact with upper limbs; ii) an interactive controller yielding physical communication, integrating differential Game Theory (GT) and an algorithm to identify the partner’s control; iii) a bi-directional user interface encompassing AR-based application-driven serious games, and a set of wearable sensors and instrumented objects; iv) Machine learning algorithms for tailoring learning exercises to the user physical, emotional, and mental state.

CONBOTS is building on recent neuroscientific findings that showed the benefits of physical interaction to performing motor tasks together, where the human central nervous system understands a partner motor control and can use it to improve task performance and motor learning. This will be implemented on innovative robotic technology, wearable sensors and machine learning techniques to give rise to novel human-human and human-robot interaction paradigms applied in two different learning contexts: i) training graphomotor skills in children learning handwriting; ii) augmenting learning performance in beginner musicians.

Using its neuroscience-driven unifying approach to motor learning and physical communication CONBOTS will expand the impact and the application of robotics to the education industry.


Objectives

These are the objectives of CONBOTS

  • To define the user needs and platform requirements and to deploy measures to raise awareness and take-up of CONBOTS scientific and technological achievements
  • To design physically interacting robotic haptic devices in the form of an end-effector robotic workstation and a wearable exoskeleton
  • To exploit recent neuroscience insights on physical interaction and modern control theory to establish a Game Theory of physical communication for improving human-human and human- robot interactions ons (WP3).
  • To design a bi-directional user interface with Augmented Reality serious games, wearable sensors and instrumented objects for maximising the impact of physical interaction in learning contexts
  • To develop machine learning algorithms for describing the physical, emotional, and mental state of the users, in order to tailor serious games and optimise the training process to the specific user needs
  • To Integrate the different CONBOTS enabling technologies and optimise them for the two application scenarios
  • To test the efficacy of CONBOTS platforms for training sensorimotor skills of children during handwriting and music learning


Role of Ghent University

Ghent University will lead the validation of the CONBOTS platform in realistic environments. It will test the physically coupling CONBOTS platforms for training sensorimotor control of children during music learning.

More to the point, an intervention study will be done for children starting violin and drumming. Each group of instrument learners will be divided in two. One group will receive training with the CONBOTS platform and one group without. Before during and after the training progress will be tracked and evaluated. The results will make clear whether the physical coupling with CONBOTS helps when acquiring sensorimotor skills in those conditions.


Website

To be announced


Contact

Prof. Marc Leman
Department: IPEM, Department of Art, music and theatre sciences  - Faculty of Arts and Philosophy
Phone number: 09 264 41 25