Gesture as Command: How Russian Scientists Are Redefining How Humans Communicate With Robots
Researchers at Don State Technical University, working with colleagues from MSTU “Stankin,” have developed domestic software that allows robots to be controlled through human gestures.
Russian-built industrial manipulators can now interpret operator input with the same level of precision as foreign systems, but without relying on imported technologies.
Modern collaborative robots, or cobots, work alongside humans in shared workspaces. To ensure safety and coordination, they rely on machine vision systems that allow them to track the operator’s position and actions. In most cases, these systems are embedded into hardware and supplied by foreign vendors. Amid import substitution pressures, Russian researchers have developed an alternative – a domestically developed AI algorithm.
The system combines input from a standard RGB camera and a depth sensor. The former captures image data, while the latter measures object depth. Together, they offset each other’s limitations, such as poor lighting or incomplete depth perception. This results in more accurate and robust action recognition. Based on these inputs, the team developed a descriptor – a feature vector that captures the operator’s movement with high fidelity.
A New Approach to Industrial Control
The new Russian system could reshape how industrial automation is controlled. Developed with support from a grant by the Russian Science Foundation, the project is already positioned for deployment in real-world production environments.
This is not a niche experiment but a practical step toward more natural human–machine interfaces. The approach is especially relevant for cobots, which are designed for safe, collaborative work with humans in shared environments. In this context, safety, responsiveness, and intuitive control are critical. According to the developers, the system has been trained to recognize ten gesture commands and can be used in loading operations, assembly, painting, laser processing, and marking.
Why This Matters for Russia
The development is not intended for the mass consumer market. It is an applied solution at the intersection of computer vision, artificial intelligence, and robotics. In the context of import substitution, it addresses a specific need for locally controlled interfaces that are independent of foreign hardware and software platforms.
For Russian citizens, the impact will take the form of safer and more automated industrial environments, lower reliance on foreign systems, and increased demand for engineers and AI specialists. For the average consumer, the impact will be indirect – through improved quality and lower costs of industrial goods, as well as the gradual adoption of domestic robotic systems in agriculture, logistics, healthcare, and services.
At the national level, the development contributes to strengthening technology sovereignty. The Ministry of Industry and Trade has previously set a target for Russia to enter the top 25 countries in robot density by 2030, which would require increasing domestic industrial robot production more than tenfold. Interfaces like this serve as an enabling layer within that strategy.
Global Context and Outlook
Globally, the development is not a breakthrough, but it aligns closely with current trends. Gesture-based interfaces and visual control systems are under active development in industrial robotics. According to the International Federation of Robotics, more than 542,000 industrial robots were installed worldwide in 2024, and annual deployments could exceed 700,000 units by 2028. Demand is expected to grow.
The export potential of the solution is likely to be limited to niche markets. Its value will depend not on the concept of gesture control itself but on recognition accuracy, robustness under interference, compatibility with industrial controllers, and deployment cost. If standardized as a modular component for cobots and mobile robots, the technology could gain traction in emerging markets where cost-effective alternatives to Western systems are needed.
From Lab to Production Line
The software is already ready for industrial deployment. It can be applied to automate loading, assembly operations, painting, laser processing, marking, and other collaborative human–robot tasks. The system prioritizes operator safety. At the same time, its gesture-based control model opens up new applications within DSTU’s strategic project – the development of the Dontech unmanned tractor – making interaction with the machine more intuitive.
The technology is relevant for large corporations, small and medium-sized businesses, as well as laboratories, research centers, hospitals, power plants, and agricultural enterprises that use autonomous systems and robotics.
From a market perspective, the development fits within a broader trend: global robot adoption is accelerating, while in Russia, the government is actively promoting industrial robotics and automation. The most likely path is integration into domestic robotic systems, which can later be brought to international markets as part of larger, integrated solutions.
