Toloka
Toloka AI · toloka.ai · Last checked:
- Managed evaluation and red teaming
- Self-service platform
Company status
Active. May 2025 investment led by Bezos Expeditions; Nebius retains a significant majority economic stake but relinquished majority voting control.
Sources 1, 2, 3, 4 · checked 2026-11-08
Capabilities
- Expert evaluation of frontier LLMs and agents: pointwise, interactive, golden-answer, rubric-based, side-by-side, and red-teaming methods
- Custom domain-specific evaluation datasets; vetted experts with Masters or PhD degrees across 90+ domains
- Red teaming and safety: proprietary risk taxonomy, safety and fairness evaluations, attack prompt sets; first results in 2 weeks
- Agent evaluation and red-teaming; Toloka Arena leaderboard
Sources 1, 2, 3, 4 · checked 2026-11-08
Who it tends to fit
- Model builders wanting expert rubric-based evaluation and multilingual red teaming
- Teams that want red-team findings followed by mitigation data (SFT, guardrail tuning)
Sources 1, 2, 3, 4 · checked 2026-11-08
Pricing
Contact vendor (managed evaluation and red teaming)
Sources 1, 2, 3, 4 · checked 2026-11-08
Published certifications
Listed as stated by the vendor. Scope can differ by product or subsidiary; request current reports.
Sources 1, 2, 3, 4 · checked 2026-11-08
Limitations to check
- No public pricing for managed evaluation
- Review the 2025 ownership change (Nebius economic stake, Bezos Expeditions-led investment) in diligence
Sources 1, 2, 3, 4 · checked 2026-11-08
Sources
- Evaluation page: human or automated scoring methods incl. rubric-based, side-by-side, red-teaming; experts with Masters or PhD degrees, 90+ domainstoloka.ai/evaluationChecked 2026-11-08
- AI safety page: red teaming, risk taxonomy, safety and fairness evaluations, attack prompt sets, first results in 2 weeks, 40+ languagestoloka.ai/ai-safetyChecked 2026-11-08
- Security page: SOC 2 Type II, ISO 27001, ISO 27701, GDPRtoloka.ai/securityChecked 2026-11-08
- Nebius release May 7, 2025: Bezos Expeditions-led investment; Nebius keeps significant majority economic stake and relinquishes majority voting controlnebius.com/newsroom/nebius-welcomes-bezos-expeditions-as-lead-investor-in-ai-data-business-tolokaChecked 2026-11-08
Report a correction: [email protected]. Facts are re-checked at least every 90 days.