Sberbank Unveils Compact Pollux Model for Evaluating AI Responses
Researchers at Sberbank's Machine Learning Experimental Systems Division have introduced Pollux, an open-source compact AI model.
Pollux evaluates the quality of AI-generated responses against criteria set by the user. Such evaluations typically rely on large language models, while running them at scale requires substantial computing resources. Pollux runs locally, reducing those costs. The judge model has 4 billion parameters and supports both Russian and English. Users can define their own evaluation criteria and scoring scales. Pollux can analyze multi-turn conversations, outputs from retrieval-augmented generation (RAG) systems that draw on external sources, and sequences of tool calls made by AI agents.
The model is expected to help identify errors in AI-generated responses more quickly and improve the quality and safety of AI systems.
