Products - AAC4All

AAC Platform : Life Companion (WP1)

The AAC4ALL project will led to the development of LifeCompanion, an open software platform allowing the creation of specific modules for alternative and augmentative communication. The first version of LifeCompanion is already used in clinical practice.

The platform is accessible to different types of disabilities (motor, visual, cognitive, etc.) and is already used in many health structures (over 500 unique installations so far).

The platform will allow for :

the integration of new components in an open software architecture,
a real uses monitoring by a reporting module that will also enable the computation of evaluation metrics.

The specifications and technical documentation of the platform are available on the LifeCompanion repository .

Optimized adaptative prediction (WP3)

AAC systems incorporate ordinarily a word prediction module in order to speed up text input and thus communication. The benefit of word prediction depends largely on the system’s ability to adapt to the dialogue context and the user’s language. With this aim, we develop prediction engines adapted to children’s language, and more globally to the evolution of our language skills. These age-appropriate prediction engines have been developed and evaluated during the year 2023.

Software distribution – An adaptative word prediction engine is already integrated in the LifeCompanion platform. Now, the platform proposes also a prediction model developed specifically for young language.

Last results – Creation of a new French corpus for child language, enabling the traininf of specific child prediction model. With such an adapted model, the desired words are often proposed in a better position in the prediction list. This work was presented at the next HCI International conference (July 2023). We are currently working on the adaptation of the prediction to language register (formal vs. informal for instance). This work will be presented in june 2024 at the LREC-COLING’2024 conference. We are also investigating the integration of neural large language models (BERT) in word prediction techniques. For the moment being, these models do not behave better than our current simple N-gram prediction model.

Publications – Scientific presentation of our dedicated child prediction model : Ben Khelil C., Rayar F., Antoine J-Y., Hoiry M., Raynal M., Halftermeyer A. (2023) What you need is what you get: adapting word prediction of Augmentative and Alternative Communication aids to youth language. HCII International. Free version (pre-print) : https://hal.science/hal-04190258

Spell checking / correction (WP3)

Just like on a cell phone SMS editor, the prediction module of an AAC system displays, after each new letter input, the list of the most probable words being typed. The prediction is based on the preceding words but also on the letters already entered in the current word. Prediction may be seriously affected by the occurrence of spelling or grammatical errors. This problem quickly becomes insurmountable in case of DYS disorders (dyslexia in particular) or when the user has limited language skills. We will develop a spell correction module that adapts to the person’s language skills and/or disorders in order to improve text entry. This correction will work on the fly while typing. However, it must also be able to work in verification mode at the end of the word input (spell checking). We will particularly focus on the writings of people suffering from language disorders (DYS or other). Software distribution – A first spell correction model is already integrated in the LifeCompanion platform. Now speech therapists are investigating the ways to adapt correction rules to the needs of any specific user.

Publications – No publication yet ! Your current works are inspired by our critical analysis of the state of the art and some experiments we conducted. This work is presented here : Antoine Jean-Yves et al. (2018) Ma copie adore le vélo : analyse des besoins réels en correction orthographique sur un corpus de dictées d’enfants. Actes TALN’2019, Rennes, France. [HAL-02375246]

Ergonomic aspects of text input (WP2)

The main objective of this work is to study different forms of coupling and interaction between a keyboard and a prediction system. We will design, implement and evaluate (through controlled experiments) new interaction techniques focused on the coupling between a keyboard and a prediction system, according to two types of representations: (1) character-based (2) pictogram-based. These studies will consider the two main case studies identified in the literature: (a) keyboards used with a pointing device; (b) text entry using single-switch scanning. For all studies, we will follow an iterative design approach based on the 4 phases of the software design cycle (Analysis – Design – Prototyping – Evaluation). Publications – No publication yet !

Serious games (WP4)

AAC devices are often misused or abandoned by their users. The appropriation of these devices depends on the ability of users to learn to use them effectively. These learning skills are profoundly affected with children with cognitive disabilities. Games are often seen as a good way to learn. Moreover, if they are both serious and fun, these games allow to acquire certain skills quickly. GazePlay is a free and open-source software developed since 2016 that gathers serious games, usable in particular by eye-tracking, whose objective is to help children with cognitive disabilities to acquire a certain number of skills through play, including the ability to use the interactions of communication software. GazePlay is now in its 10th major version. The 9th version, after having been tested by a whole group of people in ecological situations (family, therapists, students in child neuropsychology), has been tested for more than 8 months and downloaded more than 1400 times. The AAC4All platform will integrate modules to support the use of the systems via serious games to promote training and appropriation of the system by the users. LIG’s experience with GazePlay will be essential in this WP where scientific questions still need to be addressed to know how games can be used to accelerate learning of AAC devices or make better use of them. Publications: Schwab et al (2018), The GazePlay Project: Open and Free Eye-trackers Games and a Community for People with Multiple Disabilities. In ICCHP 2018. URL Schwab et al (2020), The GazePlay Project: open, free eye-trackers games and a community for people with multiple disabilities. In 1024 – Bulletin de la Société informatique de France. April 2020. URL (in french).

Pictograms evaluation (WP5)

Design and implementation of experimental protocols in psycholinguistics and neurolinguistics (methodologies: EEG, Oculometry) on the processing of pictograms in patients with impaired communication (language disorders, cognitive disorders, motor disorders, etc)
Publication of results: articles and scientific communications.
Publication of protocols and data sets (stimuli, programs, data) following FAIR principles.

Translated with www.DeepL.com/Translator (free version)

Recommendation platform (WP6)

Le projet AAC4ALL va permettre la création d’une plateforme de recommandations sur les systèmes d’aide à la communication. En effet, il est aujourd’hui complexe de trouver des informations exhaustives et neutres sur le sujet : celles-ci existent mais sont éclatées et ne permettent pas un tri facile de l’information. Le but de la plateforme sera donc de regrouper et d’alimenter une veille sur les dispositifs d’aides à la communication permettant De sensibiliser différents publics au sujet de l’aide à la communication De rechercher des systèmes d’aides à la communication d’après des critères objectifs De trouver des ressources sur les systèmes d’aides à la communication De mettre en relation des ressources et des personnes sur la thématique, en ayant une vocation de HUB sur le sujet De créer du lien plus efficient entre les acteurs de la recherche et de la clinique Le CoWork’HIT sera responsable de créer le modèle économique de cette plateforme afin d’assurer sa pérennité post-projet. Ce modèle économique devra être basé sur une offre de service répondant aux lacunes aujourd’hui constatées : manque de formations, de compétences, besoin d’expertises « à disposition » …

Large panel evaluations (WP6)

The platform will be evaluated amongst a large panel of users in order to and The main objectives are: Design and evaluate the platform components by integrating the concepts and requirements in terms of functionality and usability – Two distinct user panels will be constituted. The first panel will consist of 10 healthcare professionals from two different sites (Garches, Kerpape), with a 50/50 ratio between occupational therapists and speech therapists who work with children and adults. The second panel will include 5 people with disabilities on both sites (i.e. 10 people) with various pathologies leading to oral or written communication disorders. Both panels will participate in focus groups for each component of the platform, focusing on the usability/utility criteria that will be constructed within the project. Focus groups will be conducted at the beginning (design) and at the end (evaluation). Evaluation of the completed platform – A clinical study will focus on 20 people with disabilities per site (Garches, Kerpape), accompanied by health professionals (occupational therapist and speech therapist) in the context of their current practice, consisting of recommending AAC tools. It will include all learning strategies commonly used by the clinicians and serious games. The pathologies of people with disabilities will be diverse in order to address as many possible situations, with a maximum 70/30 imbalance in terms of written/oral communication difficulties. A SCED (Single Case Experimental Designs) methodology will be used in practice. The evaluation will focus on the availability of functionalities, the ergonomics of the platform (usability, acceptability) and the satisfaction of users. The quantitative evaluation will use validated questionnaires and observation data provided by the platform. Semi-structured interviews will provide qualitative data. Based on the analysis of the quantitative and qualitative data extracted from the platform during the focus groups, recommendation guidelines will be produced.