{
  "@context": "https://schema.org",
  "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-10-13-nemotron-personas-india-synthesized-data-for-sovereign-ai/#webpage",
  "@type": [
    "WebPage",
    "ArchiveComponent"
  ],
  "articleBody": "![Nemotron-Personas-India](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/VXDHXVACuC0FOkOgkJtZI.png) *A compound AI approach to Indian personas grounded in real-world distributions*\n\n## Open Data for India's AI Future\n\nIndia represents one of the world's largest AI opportunities — with over 700 million internet users, a multitude of languages, and a rapidly growing developer ecosystem. Yet, most open datasets reflect Western norms and English-only contexts, creating a data gap that limits AI adoption in India's multilingual, multi-script environment.\n\nToday, we're releasing [Nemotron-Personas-India](https://huggingface.co/datasets/nvidia/Nemotron-Personas-India), the first open synthetic dataset of Indic personas aligned to India's real-world demographic, geographic, and cultural distributions. Licensed under **CC BY 4.0**, this dataset offers a privacy-preserving, regulation-ready foundation for scaling AI systems that reflect Indian society—without relying on sensitive personal data.\n\nBuilt with [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html), NVIDIA's enterprise-grade synthetic data generation microservice, Nemotron-Personas-India extends our global collection of [Sovereign AI](https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/) datasets. It builds on the success of our [US](https://huggingface.co/datasets/nvidia/Nemotron-Personas) and [Japan](https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan) persona datasets and includes new features designed specifically for India's culturally rich landscape.\n\nThis dataset integrates seamlessly with **Nemotron models** and other open-source LLMs, making it easy to fine-tune AI systems for Indian use cases—from multilingual chatbots to culturally-grounded specialized copilots.\n\nThis release complements our earlier suite of Hindi evaluation datasets — including [ChatRAG-Hi](https://huggingface.co/datasets/nvidia/ChatRAG-Hi), [IFEval-Hi](https://huggingface.co/datasets/nvidia/IFEval-Hi), [MT-Bench-Hi](https://huggingface.co/datasets/nvidia/MT-Bench-Hi), [GSM8K-Hi](https://huggingface.co/datasets/nvidia/GSM8K-Hi), and [BFCL-Hi](https://huggingface.co/datasets/nvidia/BFCL-Hi) — supporting a complete pipeline from synthetic data generation to rigorous model evaluation for Indian AI systems.\n\n## What’s in the Dataset?\n\n[![image/png](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png)](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png)\n\n- **21 million personas total** (3M records × 7 personas each)\n- **Multilingual support:** English and Hindi, in both Devanagari and Latin scripts\n- **27 fields per record:** Persona traits + contextual attributes grounded in official census and labor statistics, including age, gender, education, occupation, state, district, and more\n- **7.7 billion tokens total**, including 2.9B persona tokens\n  - English: 1B tokens total, 394M persona tokens\n  - Hindi (Devanagari): 4.7B tokens total, 1.8B persona tokens\n  - Hindi (Latin): 2B tokens total, 746M persona tokens\n- **\\~560k unique full names**, reflecting India's vast linguistic diversity\n- **2.9k occupational categories**, including informal, formal, and traditional sectors\n- **All 36 states of India** and 640 districts represented\n- **Natural language fields** cultural background, linguistic background, skills and expertise, hobbies and interests\n- **Persona types:** Includes general, professional, linguistic, culinary, sports, arts, and travel personas\n- **Licensed under CC BY 4.0** for commercial and non-commercial use\n\n## How We Built It\n\n### Data Generation Pipeline\n\nProduced using [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html), NVIDIA's microservice for synthetic data generation. This compound AI system enables generation with complex Jinja templating, Pydantic validation, structured outputs, automated retries, and supports multiple generation backends – the necessary tooling to scale a synthetic dataset of this size. We also leveraged the following models:\n\n1. **Probabilistic Graphical Model (Apache-2.0)** for statistical grounding\n2. **GPT-OSS-120B (Apache-2.0)** for narrative generation in English, Hindi (Devanagari), and Hindi (Latin)\n\n### Embedded Cultural Context\n\nThis dataset was aligned to India’s official demographic distributions from the 2011 Census and expanded to include attributes essential for trustworthy AI training:\n\n- **Education:** Expanded degree levels to reflect India’s diverse academic pathways\n- **Occupations:** Includes formal, informal, and traditional sectors like farming, tailoring, and street vending\n- **Life Stages:** Student, homemaker, retired, and unemployed categories included\n- **Cultural Traits:** Family structures, regional festivals, marriage traditions, and norms\n- **Digital Divide:** Modeled usage patterns across urban/rural, age, and income lines\n- **Linguistic Diversity:** Included incredible diversity with respect to first, second, and third spoken languages for each synthetic persona\n\n### Private By Design\n\nNo real names. No re-identification risk.\n\nAll personas are fully synthetic. While grounded in real-world distributions from the [2011 Census](https://censusindia.gov.in) and [Parsed Indian Electoral Rolls](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/MUEGDT) data, no data is tied to any living or deceased individual. This ensures developers can safely train AI systems without privacy risks or regulatory barriers.\n\n## Who This Is For\n\n**Built for India, Ready for the World**\n\nNemotron‑Personas‑India is designed for developers building [Sovereign AI](https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/) systems for the **Indian market**, as well as global teams looking to adapt models to India’s unique linguistic, cultural, and social context.\n\nMost open datasets today reflect English-speaking, Western norms—limiting AI performance in India’s multilingual, multi-script, and demographically complex environments.\n\n## Practical AI Applications\n\nWith Nemotron‑Personas‑India, teams can:\n\n- Generate diverse, realistic training data in **Indian languages and scripts**\n- Fine-tune models to capture **local social, occupational, and cultural nuance**\n- Build **region-aware AI agents** that generalize across India’s many communities\n- Develop **domain-specific copilots** tuned to Indian professional and civic workflows\n- Create **multilingual systems** capable of handling **complex multi-turn conversations** and varying levels of digital fluency\n\n## Why It Matters\n\nIndia's 1.4 billion people speak hundreds of languages and live across vast cultural, economic, and geographic divides. India's National AI Portal estimates over [7,000 AI startups and research institutions](https://indiaai.gov.in/indiaaiportal) are working to build locally relevant AI systems, and the Digital India initiative and government programs like IndiaAI are accelerating adoption.\n\nBut progress is constrained by a fundamental gap: high-quality, culturally grounded training data that reflects India's demographic reality. Without representative datasets, AI systems struggle with code-switching between English and Hindi, fail to understand regional occupational categories, and miss cultural context essential for trust and adoption.\n\nThe dataset improves diversity of synthetically-generated data, mitigates biases, and prevents [model collapse](https://medium.com/data-science/addressing-concerns-of-model-collapse-from-synthetic-data-in-ai-7cd380208d14) (degradation caused by uncurated training on another model's outputs) by reflecting India's real geographic and demographic distributions.\n\n**Nemotron-Personas-India** supports Indian model builders in developing Sovereign AI systems that incorporate important region-specific demographics and cultural context.\n\n## Start Building with Nemotron-Personas-India\n\nWant to build AI systems that understand India's culture, languages, and people?\n\n**To start experimenting today:**\n\n```python\nfrom datasets import load_dataset\n\n# English personas\nnemotron_personas_en = load_dataset(\"nvidia/Nemotron-Personas-India\", \"en_IN\")\n# Hindi personas in Devanagari\nnemotron_personas_hi_deva = load_dataset(\"nvidia/Nemotron-Personas-India\", \"hi_Deva_IN\")\n# Hindi personas in Latin\nnemotron_personas_hi_latn = load_dataset(\"nvidia/Nemotron-Personas-India\", \"hi_Latn_IN\")\n```\n\nWhether you're an Indian model builder developing Sovereign AI or a global developer seeking better regional adoption, Nemotron-Personas-India provides the authentic, privacy-safe foundation your applications need.\n\nDownload it. Fine-tune it. Build AI that understands India. If you’re ready to go deeper, an extended version of Nemotron-Personas-India (which includes e.g., first/last names, religion, and synthetic addresses) is available in [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html).\n",
  "authors": [],
  "captured_at": "2026-10-04T12:50:30.804479567+00:00",
  "category": "ai",
  "content_html": "<p><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/VXDHXVACuC0FOkOgkJtZI.png\" alt=\"Nemotron-Personas-India\" loading=\"eager\" decoding=\"async\" fetchpriority=\"high\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture> <em>A compound AI approach to Indian personas grounded in real-world distributions</em></p>\n<h2>Open Data for India's AI Future</h2>\n<p>India represents one of the world's largest AI opportunities — with over 700 million internet users, a multitude of languages, and a rapidly growing developer ecosystem. Yet, most open datasets reflect Western norms and English-only contexts, creating a data gap that limits AI adoption in India's multilingual, multi-script environment.</p>\n<p>Today, we're releasing <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/Nemotron-Personas-India\">Nemotron-Personas-India</a>, the first open synthetic dataset of Indic personas aligned to India's real-world demographic, geographic, and cultural distributions. Licensed under <strong>CC BY 4.0</strong>, this dataset offers a privacy-preserving, regulation-ready foundation for scaling AI systems that reflect Indian society—without relying on sensitive personal data.</p>\n<p>Built with <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html\">NeMo Data Designer</a>, NVIDIA's enterprise-grade synthetic data generation microservice, Nemotron-Personas-India extends our global collection of <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/\">Sovereign AI</a> datasets. It builds on the success of our <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/Nemotron-Personas\">US</a> and <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan\">Japan</a> persona datasets and includes new features designed specifically for India's culturally rich landscape.</p>\n<p>This dataset integrates seamlessly with <strong>Nemotron models</strong> and other open-source LLMs, making it easy to fine-tune AI systems for Indian use cases—from multilingual chatbots to culturally-grounded specialized copilots.</p>\n<p>This release complements our earlier suite of Hindi evaluation datasets — including <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/ChatRAG-Hi\">ChatRAG-Hi</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/IFEval-Hi\">IFEval-Hi</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/MT-Bench-Hi\">MT-Bench-Hi</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/GSM8K-Hi\">GSM8K-Hi</a>, and <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/BFCL-Hi\">BFCL-Hi</a> — supporting a complete pipeline from synthetic data generation to rigorous model evaluation for Indian AI systems.</p>\n<h2>What’s in the Dataset?</h2>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png\" alt=\"image/png\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<ul>\n<li><strong>21 million personas total</strong> (3M records × 7 personas each)</li>\n<li><strong>Multilingual support:</strong> English and Hindi, in both Devanagari and Latin scripts</li>\n<li><strong>27 fields per record:</strong> Persona traits + contextual attributes grounded in official census and labor statistics, including age, gender, education, occupation, state, district, and more</li>\n<li><strong>7.7 billion tokens total</strong>, including 2.9B persona tokens\n<ul>\n<li>English: 1B tokens total, 394M persona tokens</li>\n<li>Hindi (Devanagari): 4.7B tokens total, 1.8B persona tokens</li>\n<li>Hindi (Latin): 2B tokens total, 746M persona tokens</li>\n</ul>\n</li>\n<li><strong>~560k unique full names</strong>, reflecting India's vast linguistic diversity</li>\n<li><strong>2.9k occupational categories</strong>, including informal, formal, and traditional sectors</li>\n<li><strong>All 36 states of India</strong> and 640 districts represented</li>\n<li><strong>Natural language fields</strong> cultural background, linguistic background, skills and expertise, hobbies and interests</li>\n<li><strong>Persona types:</strong> Includes general, professional, linguistic, culinary, sports, arts, and travel personas</li>\n<li><strong>Licensed under CC BY 4.0</strong> for commercial and non-commercial use</li>\n</ul>\n<h2>How We Built It</h2>\n<h3>Data Generation Pipeline</h3>\n<p>Produced using <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html\">NeMo Data Designer</a>, NVIDIA's microservice for synthetic data generation. This compound AI system enables generation with complex Jinja templating, Pydantic validation, structured outputs, automated retries, and supports multiple generation backends – the necessary tooling to scale a synthetic dataset of this size. We also leveraged the following models:</p>\n<ol>\n<li><strong>Probabilistic Graphical Model (Apache-2.0)</strong> for statistical grounding</li>\n<li><strong>GPT-OSS-120B (Apache-2.0)</strong> for narrative generation in English, Hindi (Devanagari), and Hindi (Latin)</li>\n</ol>\n<h3>Embedded Cultural Context</h3>\n<p>This dataset was aligned to India’s official demographic distributions from the 2011 Census and expanded to include attributes essential for trustworthy AI training:</p>\n<ul>\n<li><strong>Education:</strong> Expanded degree levels to reflect India’s diverse academic pathways</li>\n<li><strong>Occupations:</strong> Includes formal, informal, and traditional sectors like farming, tailoring, and street vending</li>\n<li><strong>Life Stages:</strong> Student, homemaker, retired, and unemployed categories included</li>\n<li><strong>Cultural Traits:</strong> Family structures, regional festivals, marriage traditions, and norms</li>\n<li><strong>Digital Divide:</strong> Modeled usage patterns across urban/rural, age, and income lines</li>\n<li><strong>Linguistic Diversity:</strong> Included incredible diversity with respect to first, second, and third spoken languages for each synthetic persona</li>\n</ul>\n<h3>Private By Design</h3>\n<p>No real names. No re-identification risk.</p>\n<p>All personas are fully synthetic. While grounded in real-world distributions from the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://censusindia.gov.in\">2011 Census</a> and <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/MUEGDT\">Parsed Indian Electoral Rolls</a> data, no data is tied to any living or deceased individual. This ensures developers can safely train AI systems without privacy risks or regulatory barriers.</p>\n<h2>Who This Is For</h2>\n<p><strong>Built for India, Ready for the World</strong></p>\n<p>Nemotron‑Personas‑India is designed for developers building <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/\">Sovereign AI</a> systems for the <strong>Indian market</strong>, as well as global teams looking to adapt models to India’s unique linguistic, cultural, and social context.</p>\n<p>Most open datasets today reflect English-speaking, Western norms—limiting AI performance in India’s multilingual, multi-script, and demographically complex environments.</p>\n<h2>Practical AI Applications</h2>\n<p>With Nemotron‑Personas‑India, teams can:</p>\n<ul>\n<li>Generate diverse, realistic training data in <strong>Indian languages and scripts</strong></li>\n<li>Fine-tune models to capture <strong>local social, occupational, and cultural nuance</strong></li>\n<li>Build <strong>region-aware AI agents</strong> that generalize across India’s many communities</li>\n<li>Develop <strong>domain-specific copilots</strong> tuned to Indian professional and civic workflows</li>\n<li>Create <strong>multilingual systems</strong> capable of handling <strong>complex multi-turn conversations</strong> and varying levels of digital fluency</li>\n</ul>\n<h2>Why It Matters</h2>\n<p>India's 1.4 billion people speak hundreds of languages and live across vast cultural, economic, and geographic divides. India's National AI Portal estimates over <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://indiaai.gov.in/indiaaiportal\">7,000 AI startups and research institutions</a> are working to build locally relevant AI systems, and the Digital India initiative and government programs like IndiaAI are accelerating adoption.</p>\n<p>But progress is constrained by a fundamental gap: high-quality, culturally grounded training data that reflects India's demographic reality. Without representative datasets, AI systems struggle with code-switching between English and Hindi, fail to understand regional occupational categories, and miss cultural context essential for trust and adoption.</p>\n<p>The dataset improves diversity of synthetically-generated data, mitigates biases, and prevents <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://medium.com/data-science/addressing-concerns-of-model-collapse-from-synthetic-data-in-ai-7cd380208d14\">model collapse</a> (degradation caused by uncurated training on another model's outputs) by reflecting India's real geographic and demographic distributions.</p>\n<p><strong>Nemotron-Personas-India</strong> supports Indian model builders in developing Sovereign AI systems that incorporate important region-specific demographics and cultural context.</p>\n<h2>Start Building with Nemotron-Personas-India</h2>\n<p>Want to build AI systems that understand India's culture, languages, and people?</p>\n<p><strong>To start experimenting today:</strong></p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">datasets</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">load_dataset</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> English personas\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">nemotron_personas_en</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_dataset</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">nvidia/Nemotron-Personas-India<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">en_IN<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Hindi personas in Devanagari\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">nemotron_personas_hi_deva</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_dataset</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">nvidia/Nemotron-Personas-India<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">hi_Deva_IN<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Hindi personas in Latin\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">nemotron_personas_hi_latn</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_dataset</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">nvidia/Nemotron-Personas-India<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">hi_Latn_IN<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<p>Whether you're an Indian model builder developing Sovereign AI or a global developer seeking better regional adoption, Nemotron-Personas-India provides the authentic, privacy-safe foundation your applications need.</p>\n<p>Download it. Fine-tune it. Build AI that understands India. If you’re ready to go deeper, an extended version of Nemotron-Personas-India (which includes e.g., first/last names, religion, and synthetic addresses) is available in <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html\">NeMo Data Designer</a>.</p>\n",
  "content_markdown": "![Nemotron-Personas-India](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/VXDHXVACuC0FOkOgkJtZI.png) *A compound AI approach to Indian personas grounded in real-world distributions*\n\n## Open Data for India's AI Future\n\nIndia represents one of the world's largest AI opportunities — with over 700 million internet users, a multitude of languages, and a rapidly growing developer ecosystem. Yet, most open datasets reflect Western norms and English-only contexts, creating a data gap that limits AI adoption in India's multilingual, multi-script environment.\n\nToday, we're releasing [Nemotron-Personas-India](https://huggingface.co/datasets/nvidia/Nemotron-Personas-India), the first open synthetic dataset of Indic personas aligned to India's real-world demographic, geographic, and cultural distributions. Licensed under **CC BY 4.0**, this dataset offers a privacy-preserving, regulation-ready foundation for scaling AI systems that reflect Indian society—without relying on sensitive personal data.\n\nBuilt with [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html), NVIDIA's enterprise-grade synthetic data generation microservice, Nemotron-Personas-India extends our global collection of [Sovereign AI](https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/) datasets. It builds on the success of our [US](https://huggingface.co/datasets/nvidia/Nemotron-Personas) and [Japan](https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan) persona datasets and includes new features designed specifically for India's culturally rich landscape.\n\nThis dataset integrates seamlessly with **Nemotron models** and other open-source LLMs, making it easy to fine-tune AI systems for Indian use cases—from multilingual chatbots to culturally-grounded specialized copilots.\n\nThis release complements our earlier suite of Hindi evaluation datasets — including [ChatRAG-Hi](https://huggingface.co/datasets/nvidia/ChatRAG-Hi), [IFEval-Hi](https://huggingface.co/datasets/nvidia/IFEval-Hi), [MT-Bench-Hi](https://huggingface.co/datasets/nvidia/MT-Bench-Hi), [GSM8K-Hi](https://huggingface.co/datasets/nvidia/GSM8K-Hi), and [BFCL-Hi](https://huggingface.co/datasets/nvidia/BFCL-Hi) — supporting a complete pipeline from synthetic data generation to rigorous model evaluation for Indian AI systems.\n\n## What’s in the Dataset?\n\n[![image/png](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png)](https://cdn-uploads.huggingface.co/production/uploads/67ee906af503c8811bebcbf3/-ef4HnfGJO5m4ov3QXcL5.png)\n\n- **21 million personas total** (3M records × 7 personas each)\n- **Multilingual support:** English and Hindi, in both Devanagari and Latin scripts\n- **27 fields per record:** Persona traits + contextual attributes grounded in official census and labor statistics, including age, gender, education, occupation, state, district, and more\n- **7.7 billion tokens total**, including 2.9B persona tokens\n  - English: 1B tokens total, 394M persona tokens\n  - Hindi (Devanagari): 4.7B tokens total, 1.8B persona tokens\n  - Hindi (Latin): 2B tokens total, 746M persona tokens\n- **\\~560k unique full names**, reflecting India's vast linguistic diversity\n- **2.9k occupational categories**, including informal, formal, and traditional sectors\n- **All 36 states of India** and 640 districts represented\n- **Natural language fields** cultural background, linguistic background, skills and expertise, hobbies and interests\n- **Persona types:** Includes general, professional, linguistic, culinary, sports, arts, and travel personas\n- **Licensed under CC BY 4.0** for commercial and non-commercial use\n\n## How We Built It\n\n### Data Generation Pipeline\n\nProduced using [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html), NVIDIA's microservice for synthetic data generation. This compound AI system enables generation with complex Jinja templating, Pydantic validation, structured outputs, automated retries, and supports multiple generation backends – the necessary tooling to scale a synthetic dataset of this size. We also leveraged the following models:\n\n1. **Probabilistic Graphical Model (Apache-2.0)** for statistical grounding\n2. **GPT-OSS-120B (Apache-2.0)** for narrative generation in English, Hindi (Devanagari), and Hindi (Latin)\n\n### Embedded Cultural Context\n\nThis dataset was aligned to India’s official demographic distributions from the 2011 Census and expanded to include attributes essential for trustworthy AI training:\n\n- **Education:** Expanded degree levels to reflect India’s diverse academic pathways\n- **Occupations:** Includes formal, informal, and traditional sectors like farming, tailoring, and street vending\n- **Life Stages:** Student, homemaker, retired, and unemployed categories included\n- **Cultural Traits:** Family structures, regional festivals, marriage traditions, and norms\n- **Digital Divide:** Modeled usage patterns across urban/rural, age, and income lines\n- **Linguistic Diversity:** Included incredible diversity with respect to first, second, and third spoken languages for each synthetic persona\n\n### Private By Design\n\nNo real names. No re-identification risk.\n\nAll personas are fully synthetic. While grounded in real-world distributions from the [2011 Census](https://censusindia.gov.in) and [Parsed Indian Electoral Rolls](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/MUEGDT) data, no data is tied to any living or deceased individual. This ensures developers can safely train AI systems without privacy risks or regulatory barriers.\n\n## Who This Is For\n\n**Built for India, Ready for the World**\n\nNemotron‑Personas‑India is designed for developers building [Sovereign AI](https://www.nvidia.com/en-us/lp/industries/global-public-sector/sovereign-ai-technical-overview/) systems for the **Indian market**, as well as global teams looking to adapt models to India’s unique linguistic, cultural, and social context.\n\nMost open datasets today reflect English-speaking, Western norms—limiting AI performance in India’s multilingual, multi-script, and demographically complex environments.\n\n## Practical AI Applications\n\nWith Nemotron‑Personas‑India, teams can:\n\n- Generate diverse, realistic training data in **Indian languages and scripts**\n- Fine-tune models to capture **local social, occupational, and cultural nuance**\n- Build **region-aware AI agents** that generalize across India’s many communities\n- Develop **domain-specific copilots** tuned to Indian professional and civic workflows\n- Create **multilingual systems** capable of handling **complex multi-turn conversations** and varying levels of digital fluency\n\n## Why It Matters\n\nIndia's 1.4 billion people speak hundreds of languages and live across vast cultural, economic, and geographic divides. India's National AI Portal estimates over [7,000 AI startups and research institutions](https://indiaai.gov.in/indiaaiportal) are working to build locally relevant AI systems, and the Digital India initiative and government programs like IndiaAI are accelerating adoption.\n\nBut progress is constrained by a fundamental gap: high-quality, culturally grounded training data that reflects India's demographic reality. Without representative datasets, AI systems struggle with code-switching between English and Hindi, fail to understand regional occupational categories, and miss cultural context essential for trust and adoption.\n\nThe dataset improves diversity of synthetically-generated data, mitigates biases, and prevents [model collapse](https://medium.com/data-science/addressing-concerns-of-model-collapse-from-synthetic-data-in-ai-7cd380208d14) (degradation caused by uncurated training on another model's outputs) by reflecting India's real geographic and demographic distributions.\n\n**Nemotron-Personas-India** supports Indian model builders in developing Sovereign AI systems that incorporate important region-specific demographics and cultural context.\n\n## Start Building with Nemotron-Personas-India\n\nWant to build AI systems that understand India's culture, languages, and people?\n\n**To start experimenting today:**\n\n```python\nfrom datasets import load_dataset\n\n# English personas\nnemotron_personas_en = load_dataset(\"nvidia/Nemotron-Personas-India\", \"en_IN\")\n# Hindi personas in Devanagari\nnemotron_personas_hi_deva = load_dataset(\"nvidia/Nemotron-Personas-India\", \"hi_Deva_IN\")\n# Hindi personas in Latin\nnemotron_personas_hi_latn = load_dataset(\"nvidia/Nemotron-Personas-India\", \"hi_Latn_IN\")\n```\n\nWhether you're an Indian model builder developing Sovereign AI or a global developer seeking better regional adoption, Nemotron-Personas-India provides the authentic, privacy-safe foundation your applications need.\n\nDownload it. Fine-tune it. Build AI that understands India. If you’re ready to go deeper, an extended version of Nemotron-Personas-India (which includes e.g., first/last names, religion, and synthetic addresses) is available in [NeMo Data Designer](https://docs.nvidia.com/nemo/microservices/latest/generate-synthetic-data/index.html).\n",
  "dateCreated": "2026-10-04T12:50:30.804479567+00:00",
  "date_created": "2025-10-13T23:00:42+00:00",
  "date_modified": "2025-10-13T23:00:42+00:00",
  "date_published": "2025-10-13T23:00:42+00:00",
  "description": "A compound AI approach to Indian personas grounded in real-world distributions Open Data for India's AI Future India represents one of the world's largest AI opportunities — with over 700 million internet users, a multitude of languages,…",
  "external_url": "https://huggingface.co/blog/nvidia/nemotron-personas-india",
  "git": {
    "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e",
    "path": "items/huggingface-co/2025/10/2025-10-13-nemotron-personas-india-synthesized-data-for-sovereign-ai.md",
    "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2025/10/2025-10-13-nemotron-personas-india-synthesized-data-for-sovereign-ai.md"
  },
  "id": "urn:aggr:item:4656e1e53d8cad3bd7cb3315654b8aa1d9734c5e",
  "isBasedOn": {
    "@id": "https://huggingface.co/blog/nvidia/nemotron-personas-india"
  },
  "mainEntity": {
    "@id": "https://huggingface.co/blog/nvidia/nemotron-personas-india",
    "@type": "CreativeWork",
    "archivedAt": {
      "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-10-13-nemotron-personas-india-synthesized-data-for-sovereign-ai/#webpage"
    },
    "dateModified": "2025-10-13T23:00:42+00:00",
    "datePublished": "2025-10-13T23:00:42+00:00",
    "headline": "Nemotron-Personas-India: Synthesized Data for Sovereign AI",
    "timeRequired": "PT5M",
    "url": "https://huggingface.co/blog/nvidia/nemotron-personas-india",
    "wordCount": 1020
  },
  "name": "Archived snapshot: Nemotron-Personas-India: Synthesized Data for Sovereign AI",
  "preview": null,
  "reading_minutes": 5,
  "replicated_at": null,
  "source": "huggingface-co",
  "source_name": "Hugging Face - Blog",
  "summary": null,
  "tags": [],
  "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00",
  "title": "Nemotron-Personas-India: Synthesized Data for Sovereign AI",
  "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-10-13-nemotron-personas-india-synthesized-data-for-sovereign-ai/",
  "word_count": 1020
}