Development and Testing of a System to Interpret Emotion for Children With Severe Motor and Communication Impairments (SMCI)

Abstract

This research aimed to advance the understanding of emotional expression in children with severe motor and communication impairments (SMCI) by developing and testing an algorithm to interpret psychophysiological signals of emotion. The study had three primary objectives: to create an emotion-detection algorithm, to validate its accuracy with typically developing populations, and to evaluate its applicability for children with SMCI. Despite the pandemicrelated restrictions that prevented direct collaboration with SMCI participants, the research leveraged data from typically developing individuals. Currently, models do not contain representative data or predict for individuals that are non-typical such as children with SMCI. The main limitation of this research was the reliance on data from typically developing populations in the development of the initial model, which limited the application of the model when processing data from children with SMCI. The complexities of emotion assessment in participants with SMCI were further compounded by challenges in communication and the limitations of the Self-Assessment Manikin (SAM) tool, which some children with SMCI struggled to comprehend. Additionally, there appeared to be a tendency to suppress or exaggerate emotions which posed challenges when corroborating scores from the caregiver. This research highlighted the need for an emotion detection device based on physiological signals especially in situations where it is important to know the child’s emotion for their overall wellbeing. The developed algorithm employed a two-phase model, integrating Signal to PAD (Pleasure, Arousal, Dominance) and PAD to Emotion, to decode emotions. Based on data from the typical population, the model confirmed the presence of six basic emotions, including happy, sadness, anger, disgust, and nuanced types of surprise (good and bad) while also highlighting the intricate nature of emotional experiences and the pivotal role of dominance in emotion perception. The model was tested using the DEAP dataset available online and the BDAT lab dataset that includes participants with SMCI. The model was accurate 21% of the time when selecting P, A, D scores on a nine point scale. However, the model was only successful in correctly predicting emotions in four cases of a potential 300 trials collected with the population of children with SMCI. The research contributes valuable insights into emotion detection in underrepresented populations, emphasizing the need for more nuanced models to capture the complexity of emotional experiences. Future research should focus on long-term studies involving participants with SMCI in natural settings and explore advanced technologies like ECG enabled smartwatches. Investigating alternative modeling techniques, such as neural networks, could also enhance the accuracy of emotion detection.