---
type: claim
domain: ai-alignment
description: "Samiksha represents unprecedented scale for multilingual, multi-domain AI evaluation in non-English contexts"
confidence: likely
source: "CIP Year in Review 2025, Samiksha program"
created: 2026-03-11
secondary_domains: [collective-intelligence]
---

# Samiksha conducted 25,000+ queries across 11 Indian languages with 100,000+ manual evaluations, representing the most comprehensive multilingual AI evaluation in non-English contexts

CIP's Samiksha program conducted 25,000+ queries across 11 Indian languages with 100,000+ manual evaluations, covering healthcare, agriculture, education, and legal domains. CIP describes this as "the most comprehensive evaluation of AI in Indian contexts," and the scale supports that claim — no comparable multilingual, multi-domain evaluation exists in public literature.

The significance is methodological and political. Methodologically, it demonstrates that rigorous AI evaluation can be conducted in non-English, non-Western contexts at scale. Politically, it provides evidence for Indian policymakers that AI systems trained primarily on English/Western data may not serve Indian populations adequately.

The 100,000+ manual evaluations indicate human-in-the-loop assessment at scale, not automated metrics. This matters because automated evaluation metrics (BLEU, ROUGE, perplexity) are known to correlate poorly with actual utility in domain-specific, multilingual contexts. Medical review was included for healthcare accuracy and safety assessment, indicating domain-expert validation.

## Evidence
- Samiksha: 25,000+ queries, 11 Indian languages, 100,000+ manual evaluations
- Domains: healthcare, agriculture, education, legal
- Medical review included for healthcare accuracy and safety assessment
- Indian government incorporated findings (specific policy changes not detailed in source)

## Limitations
The source does not provide specific findings from Samiksha — only scale metrics and domain coverage. We don't know what the evaluation revealed about model performance, what failure modes were identified, or how Indian government policy changed in response. The claim is about the evaluation's comprehensiveness and methodology, not its results. Confidence is 'likely' based on scale and institutional adoption, but the lack of detailed findings limits how much we can infer about impact.

---

Relevant Notes:
- [[community-centred norm elicitation surfaces alignment targets materially different from developer-specified rules]] — Samiksha as community-centered evaluation at scale
- [[global model training creates systematic failure mode where AI models provide generic responses to local context-specific queries, as evidenced by Sri Lanka election evaluation]] — Samiksha may reveal similar failures in Indian contexts
