Russian Researchers Train AI to Recognize Complex Human Emotions
Scientists at Sberbank’s Center for Applied AI have developed algorithms that can detect subtle blends of emotions like “joyful surprise” or “anxious sadness” — a leap forward for emotional intelligence in machines.
At the European Conference on Computer Vision (ECCV), researchers from Sberbank’s Center for Applied Artificial Intelligence presented a breakthrough in emotion recognition technology. Their new algorithms can identify compound emotions — intricate blends of multiple feelings that humans often experience simultaneously — with significantly higher precision than before.
In everyday life, people rarely experience a single, isolated emotion. Most emotional states are complex combinations — excitement mixed with surprise, or sadness with anxiety. The new AI models capture these nuances using lightweight neural networks such as MT-EmotiMobileFaceNet, which extract emotional features from photos and videos.
On-device intelligence
The models go beyond emotion classification: they also detect facial expressions, emotional intensity, and 12 micro-movements based on psychologist Paul Ekman’s facial action coding system. Importantly, these systems run directly on mobile devices, eliminating the need to transmit data to the cloud. This design increases both speed and privacy.
A special post-processing method smooths predictions to increase stability, improving classification accuracy for mixed emotions by more than four percentage points. Even without retraining on specialized datasets, the system achieved a 7% boost in facial expression recognition and 1.25× higher precision in emotion-intensity detection, leading to an overall 4.5× increase in recognition quality.
The research team developed several lightweight neural architectures — including MT-EmotiDDAMFNet and MT-EmotiEffNet — and released their models and source code in the open library EmotiEffLib, supporting further global research in emotional AI.
Beyond science: marketing and mental health
The technology has potential applications in digital marketing, user-experience analytics, and mental-health diagnostics. Businesses could use it to interpret real-time customer reactions, while specialists in psychology and psychiatry may apply it for early detection of emotional or mood disorders.
By enabling machines to recognize emotions as complex, multi-layered experiences, Russian researchers are pushing AI closer to a more nuanced understanding of human behavior and emotional expression.
