My research interests are related to Affective and Social Computing. In my work, I model complex dynamic phenomena such as nonverbal emotional communication and social relations, applying these models to create emotionally intelligent interactive systems capable of understanding human nonverbal behavior. The way I conduct research is characterized by strong interdisciplinarity: I apply state-of-the-art methods in Artificial Intelligence, such as soft computing and machine learning, and collaborate closely with psychologists. My work often originates from findings in the human sciences (e.g., annotations of expressive behaviors, theoretical models, emotion elicitation protocols) and models them using machine/deep learning methods. They have diverse application areas, including social skills training, the development of socially intelligent artificial assistants and companions (e.g., for elderly people), therapy and rehabilitation, social inclusion (e.g., for autistic and visually impaired individuals), entertainment (e.g., video games), and, more generally, the creation of interactive systems.

Emotion Recognition

Significant effort has been made over the last two decades in the field of automatic emotion recognition from nonverbal signals. The main focus has been on visual cues (e.g., full-body movements, facial expressions) and audio cues (e.g., prosody), while other modalities have been rarely investigated. Most works focus on the recognition of the six basic emotions, although the models are still often trained on datasets collected with limited ecological validity.
In this area, our interests are multiple. First, we aim to develop innovative methods and protocols for multimodal data collection. This includes a variety of sensors and modalities (e.g., affective touch [IJ20], emotion detection from audio of respriation [IJ13]) but especially novel procedures for inducing emotions in laboratory settings, as well as collecting data in the field [IJ16]. Examples of such approaches include using an artificial agent as a confederate [IJ22] and manipulating events in virtual reality [IC47]. Second, we are interested in the complexity of emotional expressions, including individual differences in affective reactions, the role of context, as well as emotion regulation. Our goal is to move away from the dominant Ekmanian approach, which has strongly influenced both research and commercial solutions in this field, toward a deeper and more nuanced perspective that accounts for the variety of meanings and subtle differences in affective reactions. This requires new techniques of data collection that allows for to collect reach contextual information.

Positive Computing

Positive psychology aims to improve well-being and strengthen positive attitudes. It proposes tools and methods that foster individual well-being by enhancing positive feelings, positive thoughts, and positive behaviors. At the same time, researchers have begun to investigate positive emotions more broadly, identifying at least 20 distinct positive emotional states, many of which are associated with specific expressive behaviors or physiological changes.
In this context, we are particularly interested in analyzing specific positive emotional states and classifying them from nonverbal cues [IJ27]. Among others, we studied smiles and laughter in various contexts and meanings—for example, distinguishing their positive versus negative social or emotional functions [IJ6][IC32]. We also proposed methods for detecting laughter from body movements[IJ10], we developed interactive laughter-aware agents [IC26], and created multiple relavant datasets for the research community [IJ3] [IC28]. Currently we shift toward recognizing a wider range of positive emotions [IJ20] and exploring applications of this technology for enhancing well-being [IC63], in line with the concept of positive computing—a rapidly emerging trend in HCI that aims to design and evaluate technologies supporting psychological well-being, human flourishing, and positive user experiences.

Multimodal Interactions with Artificial Agents

Artificial agents, such as virtual autonomous characters and social robots, are becoming increasingly present in daily life, entering various domains—from commerce and healthcare to education and entertainment. It is crucial that they are able to create natural, human-like interactions, including appropriate nonverbal communication. This encompasses the expression of affective and cognitive states, social attitudes, and interpersonal relations.
In this line of research, we have worked, among other topics, on perceived believability, warmth, and competence [IJ5], and more recently on social comfort [IJ22]. We have developed agents that nonverbally display empathy [IC8], deception [BC1], politeness [IJ2], and various other social attitudes [IJ26]. Our current focus is on the applicative dimension, particularly on developing artificial commensal companions [IJ18][IC62][IC70]. The commensality [IJ14] can be considered a valuable testbed for development of Social Artificial Agents, as it involves rich and complex interactions, including shared attention, unstructured verbal communication, interpersonal synchronization, and a wide variety of emotions.

Human Behavior Understanding

More details about selected works can be found here.