Russia Develops Unique ‘Emotional’ Dictionary to Train Neural Networks

Researchers at the Higher School of Economics in St. Petersburg have built a multimodal emotional dataset designed to teach AI systems how to recognize subtle patterns of human feelings.

Russian linguists from the Higher School of Economics (HSE) in St. Petersburg have created a unique multimodal emotional dataset for training neural networks.

The so-called “emotional dictionary” enables large language models to detect hidden emotional cues that humans sense subconsciously but struggle to formalize. Unlike traditional dictionaries, which pair words with definitions, this dataset links text fragments to emotional labels.

During development, the researchers conducted experiments with volunteers to determine how people best perceive emotions in conversation. They then compared participants’ ratings of the same material to assess consistency.

The resulting dataset consists of 909 video fragments totaling 173 minutes. Each clip is annotated across six basic emotions in four different formats: full video, audio-only, text-only, and video without sound. The team plans to extend the dataset to cover mixed emotions, expanding its capabilities.

The researchers have also applied their findings in a practical setting: they developed an empathetic chatbot for the Hermitage Museum. This virtual assistant can detect user emotions and respond accordingly—sharing joy or offering sympathy when appropriate.