{
  "@context": "https://schema.org",
  "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-09-02-sair-accelerating-pharma-r-d-with-ai-powered-structural/#webpage",
  "@type": [
    "WebPage",
    "ArchiveComponent"
  ],
  "articleBody": "This summer, SandboxAQ released the **Structurally Augmented IC50 Repository (**[**SAIR**](https://huggingface.co/datasets/SandboxAQ/SAIR)**)**, **the largest dataset of co-folded 3D protein-ligand structures paired with experimentally measured IC₅₀ labels, directly linking molecular structure to drug potency** and overcoming a longstanding scarcity in training data. This dataset is now available on Hugging Face, and for the first time, researchers have open access to more than **5 million** AI‑generated, high‑accuracy protein-ligand 3D structures, each paired with validated empirical binding potency data.\n\n[![image/png](https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png)](https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png)\n\n**SAIR is an open-sourced dataset** and is publicly available for free under a permissive CC BY 4.0 license, making it immediately actionable for commercial and non-commercial R&D pipelines. More than just a dataset, SAIR is a **strategic asset** that bridges the long-standing data gap in AI-powered drug design. It empowers pharmaceutical, biotech, and tech‑bio leaders to accelerate R&D, expand target horizons, and supercharge AI models – moving more of the costly, lengthy drug design and optimization from the wet lab to *in silico*. This means shorter hit‑to‑lead timelines, more efficient lead optimization, fewer dead‑end projects, and a more predictable path from initial idea to clinical candidate.\n\n## Leapfrogging Past AI Achievements\n\nAI and computer-aided design have great potential in dramatically accelerating the development of new drugs. For decades, scientists have dreamed about AI that could identify or design a potent, non-toxic, and efficacious compound from a prompt describing the disease pathway, practically compressing years of drug R&D into a few minutes on a computer. However, this vision is bottlenecked by AI's ability to predict critical drug properties like potency, toxicity, etc., based solely on its molecular structure.\n\nFurthermore, traditional structure‑based discovery is often slowed early by the determination of reliable 3D structures. Three‑dimensional molecular structure dictates a molecule’s functionality, dynamics and interactions, which is especially important when a potential drug candidate is expected to bind to a human protein target.\n\nExperimental methods, such as X-ray crystallography and cryo-EM, require extensive time and investment, and many promising disease targets still lack experimentally validated structural information. Computer simulations have helped lower the barrier of getting 3D structures and predicting binding affinity. However, earlier generations of algorithms for protein folding and docking (like AlphaFold and Vina respectively) only predict static snapshots of molecules and proteins (which, in reality, are inherently dynamic and shape-changing).\n\nSAIR solves that constraint by compiling over **1 million unique computationally co-folded protein–ligand pairs**, ultimately yielding **5.24 million** distinct 3D complexes (five different co-folded structures per pair). Each structure is paired with a curated IC₅₀ measurement from [ChEMBL](https://www.ebi.ac.uk/chembl/) or [BindingDB](https://www.bindingdb.org/rwd/bind/index.jsp), providing for the first time a scalable link between high-quality 3D structures and drug potency, and bridging the historic data gap that has hindered AI-driven discovery. Deep-learned affinity models such as Boltz-2, trained on similar data, [have been shown](https://www.rxrx.ai/boltz-2) to yield up to a 1,000x speed-up over the traditional, first-principle approach.\n\n## Optimized for the cutting-edge of computing\n\nCreating SAIR was a major feat of high-performance AI computing. It took more than 130,000 GPU hours to compute the SAIR dataset using Boltz1, a cofolding AI model, on a cluster of 760 NVIDIA H100 processors, leveraging the NVIDIA DGX Cloud through Google Cloud Platform.\n\nCapturing highly granular node, operator, scheduler, and GPU metrics, as well as a close collaboration on both infrastructure and workload optimization, helped NVIDIA AI Accelerator and SandboxAQ engineering teams identify bottlenecks and optimize configurations to achieve the highest workload throughput.\n\nConsequently, the two teams were able to achieve > 95% GPU compute utilization for generating the SAIR dataset. This enabled us to create SAIR in three weeks – as opposed to the original estimate of three months (more than a 4X speed up) – and resulted in a highly optimized, GPU-native computational workflow that seamlessly integrates with today’s cutting-edge enterprise compute environments.\n\n## Unprecedented scale, accuracy and power\n\nGenerating such a massive volume of data is only half the story. Equally important is confidence in its quality, which is why every predicted complex underwent rigorous validation with [**PoseBusters**](https://pubs.rsc.org/en/content/articlelanding/2024/sc/d3sc04185a), an industry-standard, open-source tool for benchmarking structure-related AI in drug discovery. This tool checks chemical sanity and physical plausibility.\n\n**The end result was that 97 percent** of SAIR’s structures passed all checks. In addition to PoseBusters validation, we benchmarked leading affinity prediction methods, such as empirical scoring functions, 3D CNNs, and graph neural networks, across SAIR’s synthetic structures and experimental IC₅₀ values. The detailed results of these studies are available in our [scientific manuscript on bioRxiv](https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1).\n\nSAIR data is a reliable foundation for benchmarking new models as well as downstream modelling, screening, and design.\n\n## Bringing “dark” targets into the light\n\nA persistent challenge in drug discovery is the “dark proteome,” or disease‑relevant proteins for which experimental structures simply do not exist. SAIR illuminates these uncharted regions by providing credible, AI‑predicted complexes wherever experimental data is scarce. For example, more than 40 percent of the proteins in the SAIR dataset have no available structures in the Protein Data Bank (PDB) whatsoever, with or without a ligand. SAIR addresses one of the biggest challenges with existing AI models, low generalizability due to data scarcity. With SAIR, scientists can now explore targets that were previously deemed undruggable, armed with structural hypotheses to guide virtual screening and lead optimization using trustworthy model predictions.\n\nMoreover, SAIR’s cross‑target breadth uncovers polypharmacology patterns and elucidates how a single molecule might interact with multiple proteins. Leveraging this rich tapestry of interactions, you can train AI models to predict off‑target effects or identify new repurposing opportunities, equipping your organization with a deeper understanding of compound profiles before any lab work begins.\n\n* * *\n\n## Accessing SAIR\n\nSAIR is freely available on [Hugging Face](https://huggingface.co/datasets/SandboxAQ/SAIR). Here’s a quick guide to pull SAIR from Hugging Face, peek at the main table, and (optionally) download a few structure archives.\n\n### 1. Install essentials\n\nWe use the Hub to fetch files and pandas+pyarrow to read the Parquet.\n\n```bash\npip install huggingface_hub pandas pyarrow\n```\n\n### 2. Authenticate\n\nAuthenticate to Hugging Face:\n\n```python\nimport huggingface_hub\nhuggingface_hub.login(token=\"your_auth_token\")\n```\n\n### 3. Load the main table (`sair.parquet`)\n\nThis grabs the file from the Hub and loads it into a DataFrame.\n\n```python\nfrom huggingface_hub import hf_hub_download\nimport pandas as pd\n\nparquet_path = hf_hub_download(\n    repo_id=\"SandboxAQ/SAIR\",\n    filename=\"sair.parquet\",\n    repo_type=\"dataset\"\n)\n\ndf = pd.read_parquet(parquet_path)\ndf.head()\n```\n\n### 4. (Optional) List available structure archives\n\nStructure files are shipped as many `.tar.gz` archives under `structures_compressed/`. List them and pick what you need.\n\n```python\nfrom huggingface_hub import list_repo_files\n\nfiles = [f.split(\"/\")[-1] for f in list_repo_files(\"SandboxAQ/SAIR\", repo_type=\"dataset\")\n         if f.startswith(\"structures_compressed/\") and f.endswith(\".tar.gz\")]\nfiles[:5]\n```\n\n### 5. (Optional) Download and extract structures\n\nEach archive can be large (≈10 GB). Download only the ones you need and extract them locally.\n\n```python\nimport os, tarfile\nfrom huggingface_hub import hf_hub_download\n\ndest = \"sair_structures\"\nos.makedirs(dest, exist_ok=True)\n\nto_get = [\n    \"sair_structures_1006049_to_1016517.tar.gz\",\n    \"sair_structures_100623_to_111511.tar.gz\",\n]\n\nfor name in to_get:\n    tar_path = hf_hub_download(\n        repo_id=\"SandboxAQ/SAIR\",\n        filename=f\"structures_compressed/{name}\",\n        repo_type=\"dataset\",\n        local_dir=dest,\n        local_dir_use_symlinks=False,\n    )\n    with tarfile.open(tar_path, \"r:gz\") as tar:\n        tar.extractall(dest)\n    os.remove(tar_path)  # free disk space\n```\n\nA full version of this script, including more robust logging and validation, is available in the [README](https://huggingface.co/datasets/SandboxAQ/SAIR/blob/main/README.md) file for your convenience. For more details, visit [the SAIR homepage](https://www.sandboxaq.com/sair), read our [manuscript on bioRxiv](https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1), or watch our 25-minute [joint webinar with NVIDIA](https://www.youtube.com/watch?v=y96t8vu9nfg), where we demonstrate SAIR and explain how data is structured within it. Extensive documentation, tutorials, and example benchmarks are available to facilitate its use and accelerate internal adoption.\n\nThe future of drug discovery is data-driven, AI-accelerated, and grounded in scalable, high-quality structural insights. While we don’t yet have AI that can design effective drug therapies with just a prompt, SAIR brings researchers ever closer to that goal with new data and insights that can potentially shave years from even AI-accelerated R&D pipelines.\n\nWe can’t wait to see what researchers will build using SAIR, and SandboxAQ experts are here to support them throughout the discovery process.\n\n* * *\n\n### Questions?\n\nContact the authors or post on the SAIR dataset discussion page.\n\nAuthors: [Arman Zaribafiyan](mailto:arman.zaribafiyan@sandboxquantum.com), [Georgia Channing](mailto:georgia@huggingface.co), [Zane Beckwith](mailto:zane.beckwith@sandboxquantum.com), and [Rudi Plesch](mailto:rudi.plesch@sandboxquantum.com)\n",
  "authors": [],
  "captured_at": "2026-10-04T12:50:30.804479567+00:00",
  "category": "ai",
  "content_html": "<p>This summer, SandboxAQ released the <strong>Structurally Augmented IC50 Repository (</strong><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/SandboxAQ/SAIR\"><strong>SAIR</strong></a><strong>)</strong>, <strong>the largest dataset of co-folded 3D protein-ligand structures paired with experimentally measured IC₅₀ labels, directly linking molecular structure to drug potency</strong> and overcoming a longstanding scarcity in training data. This dataset is now available on Hugging Face, and for the first time, researchers have open access to more than <strong>5 million</strong> AI‑generated, high‑accuracy protein-ligand 3D structures, each paired with validated empirical binding potency data.</p>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png\" alt=\"image/png\" loading=\"eager\" decoding=\"async\" fetchpriority=\"high\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<p><strong>SAIR is an open-sourced dataset</strong> and is publicly available for free under a permissive CC BY 4.0 license, making it immediately actionable for commercial and non-commercial R&amp;D pipelines. More than just a dataset, SAIR is a <strong>strategic asset</strong> that bridges the long-standing data gap in AI-powered drug design. It empowers pharmaceutical, biotech, and tech‑bio leaders to accelerate R&amp;D, expand target horizons, and supercharge AI models – moving more of the costly, lengthy drug design and optimization from the wet lab to <em>in silico</em>. This means shorter hit‑to‑lead timelines, more efficient lead optimization, fewer dead‑end projects, and a more predictable path from initial idea to clinical candidate.</p>\n<h2>Leapfrogging Past AI Achievements</h2>\n<p>AI and computer-aided design have great potential in dramatically accelerating the development of new drugs. For decades, scientists have dreamed about AI that could identify or design a potent, non-toxic, and efficacious compound from a prompt describing the disease pathway, practically compressing years of drug R&amp;D into a few minutes on a computer. However, this vision is bottlenecked by AI's ability to predict critical drug properties like potency, toxicity, etc., based solely on its molecular structure.</p>\n<p>Furthermore, traditional structure‑based discovery is often slowed early by the determination of reliable 3D structures. Three‑dimensional molecular structure dictates a molecule’s functionality, dynamics and interactions, which is especially important when a potential drug candidate is expected to bind to a human protein target.</p>\n<p>Experimental methods, such as X-ray crystallography and cryo-EM, require extensive time and investment, and many promising disease targets still lack experimentally validated structural information. Computer simulations have helped lower the barrier of getting 3D structures and predicting binding affinity. However, earlier generations of algorithms for protein folding and docking (like AlphaFold and Vina respectively) only predict static snapshots of molecules and proteins (which, in reality, are inherently dynamic and shape-changing).</p>\n<p>SAIR solves that constraint by compiling over <strong>1 million unique computationally co-folded protein–ligand pairs</strong>, ultimately yielding <strong>5.24 million</strong> distinct 3D complexes (five different co-folded structures per pair). Each structure is paired with a curated IC₅₀ measurement from <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.ebi.ac.uk/chembl/\">ChEMBL</a> or <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.bindingdb.org/rwd/bind/index.jsp\">BindingDB</a>, providing for the first time a scalable link between high-quality 3D structures and drug potency, and bridging the historic data gap that has hindered AI-driven discovery. Deep-learned affinity models such as Boltz-2, trained on similar data, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.rxrx.ai/boltz-2\">have been shown</a> to yield up to a 1,000x speed-up over the traditional, first-principle approach.</p>\n<h2>Optimized for the cutting-edge of computing</h2>\n<p>Creating SAIR was a major feat of high-performance AI computing. It took more than 130,000 GPU hours to compute the SAIR dataset using Boltz1, a cofolding AI model, on a cluster of 760 NVIDIA H100 processors, leveraging the NVIDIA DGX Cloud through Google Cloud Platform.</p>\n<p>Capturing highly granular node, operator, scheduler, and GPU metrics, as well as a close collaboration on both infrastructure and workload optimization, helped NVIDIA AI Accelerator and SandboxAQ engineering teams identify bottlenecks and optimize configurations to achieve the highest workload throughput.</p>\n<p>Consequently, the two teams were able to achieve &gt; 95% GPU compute utilization for generating the SAIR dataset. This enabled us to create SAIR in three weeks – as opposed to the original estimate of three months (more than a 4X speed up) – and resulted in a highly optimized, GPU-native computational workflow that seamlessly integrates with today’s cutting-edge enterprise compute environments.</p>\n<h2>Unprecedented scale, accuracy and power</h2>\n<p>Generating such a massive volume of data is only half the story. Equally important is confidence in its quality, which is why every predicted complex underwent rigorous validation with <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://pubs.rsc.org/en/content/articlelanding/2024/sc/d3sc04185a\"><strong>PoseBusters</strong></a>, an industry-standard, open-source tool for benchmarking structure-related AI in drug discovery. This tool checks chemical sanity and physical plausibility.</p>\n<p><strong>The end result was that 97 percent</strong> of SAIR’s structures passed all checks. In addition to PoseBusters validation, we benchmarked leading affinity prediction methods, such as empirical scoring functions, 3D CNNs, and graph neural networks, across SAIR’s synthetic structures and experimental IC₅₀ values. The detailed results of these studies are available in our <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1\">scientific manuscript on bioRxiv</a>.</p>\n<p>SAIR data is a reliable foundation for benchmarking new models as well as downstream modelling, screening, and design.</p>\n<h2>Bringing “dark” targets into the light</h2>\n<p>A persistent challenge in drug discovery is the “dark proteome,” or disease‑relevant proteins for which experimental structures simply do not exist. SAIR illuminates these uncharted regions by providing credible, AI‑predicted complexes wherever experimental data is scarce. For example, more than 40 percent of the proteins in the SAIR dataset have no available structures in the Protein Data Bank (PDB) whatsoever, with or without a ligand. SAIR addresses one of the biggest challenges with existing AI models, low generalizability due to data scarcity. With SAIR, scientists can now explore targets that were previously deemed undruggable, armed with structural hypotheses to guide virtual screening and lead optimization using trustworthy model predictions.</p>\n<p>Moreover, SAIR’s cross‑target breadth uncovers polypharmacology patterns and elucidates how a single molecule might interact with multiple proteins. Leveraging this rich tapestry of interactions, you can train AI models to predict off‑target effects or identify new repurposing opportunities, equipping your organization with a deeper understanding of compound profiles before any lab work begins.</p>\n<hr />\n<h2>Accessing SAIR</h2>\n<p>SAIR is freely available on <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/SandboxAQ/SAIR\">Hugging Face</a>. Here’s a quick guide to pull SAIR from Hugging Face, peek at the main table, and (optionally) download a few structure archives.</p>\n<h3>1. Install essentials</h3>\n<p>We use the Hub to fetch files and pandas+pyarrow to read the Parquet.</p>\n<pre><code class=\"language-bash\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">pip</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> install huggingface_hub pandas pyarrow</span>\n</span></span></code></pre>\n<h3>2. Authenticate</h3>\n<p>Authenticate to Hugging Face:</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">huggingface_hub</span></span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">huggingface_hub</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">login</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">token</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">your_auth_token<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<h3>3. Load the main table (<code>sair.parquet</code>)</h3>\n<p>This grabs the file from the Hub and loads it into a DataFrame.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">huggingface_hub</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">hf_hub_download</span></span>\n<span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pandas</span></span> <span class=\"syntax-keyword syntax-control syntax-import syntax-as syntax-python\">as</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pd</span></span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">parquet_path</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">hf_hub_download</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-variable syntax-parameter syntax-python\">repo_id</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">SandboxAQ/SAIR<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    <span class=\"syntax-variable syntax-parameter syntax-python\">filename</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">sair.parquet<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    <span class=\"syntax-variable syntax-parameter syntax-python\">repo_type</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">dataset<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">df</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pd</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">read_parquet</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">parquet_path</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">df</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">head</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<h3>4. (Optional) List available structure archives</h3>\n<p>Structure files are shipped as many <code>.tar.gz</code> archives under <code>structures_compressed/</code>. List them and pick what you need.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">huggingface_hub</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">list_repo_files</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">files</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-structure syntax-list syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-list syntax-begin syntax-python\">[</span><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">f</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">split</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">/<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-keyword syntax-operator syntax-arithmetic syntax-python\">-</span><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">1</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span> <span class=\"syntax-meta syntax-expression syntax-generator syntax-python\"><span class=\"syntax-keyword syntax-control syntax-flow syntax-for syntax-generator syntax-python\">for</span> <span class=\"syntax-meta syntax-generic-name syntax-python\">f</span> <span class=\"syntax-keyword syntax-control syntax-flow syntax-for syntax-in syntax-python\">in</span></span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">list_repo_files</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">SandboxAQ/SAIR<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">repo_type</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">dataset<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n         <span class=\"syntax-keyword syntax-control syntax-flow syntax-if syntax-inline syntax-python\">if</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">f</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">startswith</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">structures_compressed/<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span> <span class=\"syntax-keyword syntax-operator syntax-logical syntax-python\">and</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">f</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">endswith</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">.tar.gz<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-section syntax-list syntax-end syntax-python\">]</span></span>\n<span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">files</span></span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-punctuation syntax-separator syntax-slice syntax-python\">:</span><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">5</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span>\n</span></span></code></pre>\n<h3>5. (Optional) Download and extract structures</h3>\n<p>Each archive can be large (≈10 GB). Download only the ones you need and extract them locally.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">os</span></span><span class=\"syntax-punctuation syntax-separator syntax-import-list syntax-python\">,</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tarfile</span></span></span>\n<span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">huggingface_hub</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">hf_hub_download</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">dest</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">sair_structures<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">os</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">makedirs</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">dest</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">exist_ok</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">to_get</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-structure syntax-list syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-list syntax-begin syntax-python\">[</span>\n    <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">sair_structures_1006049_to_1016517.tar.gz<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-list syntax-python\">,</span>\n    <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">sair_structures_100623_to_111511.tar.gz<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-list syntax-python\">,</span>\n<span class=\"syntax-punctuation syntax-section syntax-list syntax-end syntax-python\">]</span></span>\n\n<span class=\"syntax-meta syntax-statement syntax-for syntax-python\"><span class=\"syntax-keyword syntax-control syntax-flow syntax-for syntax-python\">for</span> <span class=\"syntax-meta syntax-generic-name syntax-python\">name</span> <span class=\"syntax-meta syntax-statement syntax-for syntax-python\"><span class=\"syntax-keyword syntax-control syntax-flow syntax-for syntax-in syntax-python\">in</span></span></span><span class=\"syntax-meta syntax-statement syntax-for syntax-python\"> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">to_get</span></span><span class=\"syntax-punctuation syntax-section syntax-block syntax-for syntax-python\">:</span></span>\n    <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tar_path</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">hf_hub_download</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n        <span class=\"syntax-variable syntax-parameter syntax-python\">repo_id</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">SandboxAQ/SAIR<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n        <span class=\"syntax-variable syntax-parameter syntax-python\">filename</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-storage syntax-type syntax-string syntax-python\">f</span><span class=\"syntax-meta syntax-string syntax-interpolated syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-interpolated syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">structures_compressed/</span><span class=\"syntax-meta syntax-interpolation syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-interpolation syntax-begin syntax-python\">{</span><span class=\"syntax-source syntax-python syntax-embedded\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">name</span></span></span></span><span class=\"syntax-meta syntax-interpolation syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-interpolation syntax-end syntax-python\">}</span></span><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n        <span class=\"syntax-variable syntax-parameter syntax-python\">repo_type</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">dataset<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n        <span class=\"syntax-variable syntax-parameter syntax-python\">local_dir</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">dest</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n        <span class=\"syntax-variable syntax-parameter syntax-python\">local_dir_use_symlinks</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">False</span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    </span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n    <span class=\"syntax-meta syntax-statement syntax-with syntax-python\"><span class=\"syntax-keyword syntax-control syntax-flow syntax-with syntax-python\">with</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tarfile</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">open</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tar_path</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">r:gz<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span> <span class=\"syntax-meta syntax-statement syntax-with syntax-python\"><span class=\"syntax-keyword syntax-control syntax-flow syntax-with syntax-as syntax-python\">as</span></span></span><span class=\"syntax-meta syntax-statement syntax-with syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">tar</span><span class=\"syntax-punctuation syntax-section syntax-block syntax-with syntax-python\">:</span></span>\n        <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tar</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">extractall</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">dest</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n    <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">os</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">remove</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">tar_path</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>  <span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> free disk space\n</span></span></span></code></pre>\n<p>A full version of this script, including more robust logging and validation, is available in the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/SandboxAQ/SAIR/blob/main/README.md\">README</a> file for your convenience. For more details, visit <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.sandboxaq.com/sair\">the SAIR homepage</a>, read our <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1\">manuscript on bioRxiv</a>, or watch our 25-minute <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.youtube.com/watch?v=y96t8vu9nfg\">joint webinar with NVIDIA</a>, where we demonstrate SAIR and explain how data is structured within it. Extensive documentation, tutorials, and example benchmarks are available to facilitate its use and accelerate internal adoption.</p>\n<p>The future of drug discovery is data-driven, AI-accelerated, and grounded in scalable, high-quality structural insights. While we don’t yet have AI that can design effective drug therapies with just a prompt, SAIR brings researchers ever closer to that goal with new data and insights that can potentially shave years from even AI-accelerated R&amp;D pipelines.</p>\n<p>We can’t wait to see what researchers will build using SAIR, and SandboxAQ experts are here to support them throughout the discovery process.</p>\n<hr />\n<h3>Questions?</h3>\n<p>Contact the authors or post on the SAIR dataset discussion page.</p>\n<p>Authors: <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"mailto:arman.zaribafiyan@sandboxquantum.com\">Arman Zaribafiyan</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"mailto:georgia@huggingface.co\">Georgia Channing</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"mailto:zane.beckwith@sandboxquantum.com\">Zane Beckwith</a>, and <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"mailto:rudi.plesch@sandboxquantum.com\">Rudi Plesch</a></p>\n",
  "content_markdown": "This summer, SandboxAQ released the **Structurally Augmented IC50 Repository (**[**SAIR**](https://huggingface.co/datasets/SandboxAQ/SAIR)**)**, **the largest dataset of co-folded 3D protein-ligand structures paired with experimentally measured IC₅₀ labels, directly linking molecular structure to drug potency** and overcoming a longstanding scarcity in training data. This dataset is now available on Hugging Face, and for the first time, researchers have open access to more than **5 million** AI‑generated, high‑accuracy protein-ligand 3D structures, each paired with validated empirical binding potency data.\n\n[![image/png](https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png)](https://cdn-uploads.huggingface.co/production/uploads/680ff4388f704be391757780/KOXzLW72JLyU3rtMrb3yn.png)\n\n**SAIR is an open-sourced dataset** and is publicly available for free under a permissive CC BY 4.0 license, making it immediately actionable for commercial and non-commercial R&D pipelines. More than just a dataset, SAIR is a **strategic asset** that bridges the long-standing data gap in AI-powered drug design. It empowers pharmaceutical, biotech, and tech‑bio leaders to accelerate R&D, expand target horizons, and supercharge AI models – moving more of the costly, lengthy drug design and optimization from the wet lab to *in silico*. This means shorter hit‑to‑lead timelines, more efficient lead optimization, fewer dead‑end projects, and a more predictable path from initial idea to clinical candidate.\n\n## Leapfrogging Past AI Achievements\n\nAI and computer-aided design have great potential in dramatically accelerating the development of new drugs. For decades, scientists have dreamed about AI that could identify or design a potent, non-toxic, and efficacious compound from a prompt describing the disease pathway, practically compressing years of drug R&D into a few minutes on a computer. However, this vision is bottlenecked by AI's ability to predict critical drug properties like potency, toxicity, etc., based solely on its molecular structure.\n\nFurthermore, traditional structure‑based discovery is often slowed early by the determination of reliable 3D structures. Three‑dimensional molecular structure dictates a molecule’s functionality, dynamics and interactions, which is especially important when a potential drug candidate is expected to bind to a human protein target.\n\nExperimental methods, such as X-ray crystallography and cryo-EM, require extensive time and investment, and many promising disease targets still lack experimentally validated structural information. Computer simulations have helped lower the barrier of getting 3D structures and predicting binding affinity. However, earlier generations of algorithms for protein folding and docking (like AlphaFold and Vina respectively) only predict static snapshots of molecules and proteins (which, in reality, are inherently dynamic and shape-changing).\n\nSAIR solves that constraint by compiling over **1 million unique computationally co-folded protein–ligand pairs**, ultimately yielding **5.24 million** distinct 3D complexes (five different co-folded structures per pair). Each structure is paired with a curated IC₅₀ measurement from [ChEMBL](https://www.ebi.ac.uk/chembl/) or [BindingDB](https://www.bindingdb.org/rwd/bind/index.jsp), providing for the first time a scalable link between high-quality 3D structures and drug potency, and bridging the historic data gap that has hindered AI-driven discovery. Deep-learned affinity models such as Boltz-2, trained on similar data, [have been shown](https://www.rxrx.ai/boltz-2) to yield up to a 1,000x speed-up over the traditional, first-principle approach.\n\n## Optimized for the cutting-edge of computing\n\nCreating SAIR was a major feat of high-performance AI computing. It took more than 130,000 GPU hours to compute the SAIR dataset using Boltz1, a cofolding AI model, on a cluster of 760 NVIDIA H100 processors, leveraging the NVIDIA DGX Cloud through Google Cloud Platform.\n\nCapturing highly granular node, operator, scheduler, and GPU metrics, as well as a close collaboration on both infrastructure and workload optimization, helped NVIDIA AI Accelerator and SandboxAQ engineering teams identify bottlenecks and optimize configurations to achieve the highest workload throughput.\n\nConsequently, the two teams were able to achieve > 95% GPU compute utilization for generating the SAIR dataset. This enabled us to create SAIR in three weeks – as opposed to the original estimate of three months (more than a 4X speed up) – and resulted in a highly optimized, GPU-native computational workflow that seamlessly integrates with today’s cutting-edge enterprise compute environments.\n\n## Unprecedented scale, accuracy and power\n\nGenerating such a massive volume of data is only half the story. Equally important is confidence in its quality, which is why every predicted complex underwent rigorous validation with [**PoseBusters**](https://pubs.rsc.org/en/content/articlelanding/2024/sc/d3sc04185a), an industry-standard, open-source tool for benchmarking structure-related AI in drug discovery. This tool checks chemical sanity and physical plausibility.\n\n**The end result was that 97 percent** of SAIR’s structures passed all checks. In addition to PoseBusters validation, we benchmarked leading affinity prediction methods, such as empirical scoring functions, 3D CNNs, and graph neural networks, across SAIR’s synthetic structures and experimental IC₅₀ values. The detailed results of these studies are available in our [scientific manuscript on bioRxiv](https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1).\n\nSAIR data is a reliable foundation for benchmarking new models as well as downstream modelling, screening, and design.\n\n## Bringing “dark” targets into the light\n\nA persistent challenge in drug discovery is the “dark proteome,” or disease‑relevant proteins for which experimental structures simply do not exist. SAIR illuminates these uncharted regions by providing credible, AI‑predicted complexes wherever experimental data is scarce. For example, more than 40 percent of the proteins in the SAIR dataset have no available structures in the Protein Data Bank (PDB) whatsoever, with or without a ligand. SAIR addresses one of the biggest challenges with existing AI models, low generalizability due to data scarcity. With SAIR, scientists can now explore targets that were previously deemed undruggable, armed with structural hypotheses to guide virtual screening and lead optimization using trustworthy model predictions.\n\nMoreover, SAIR’s cross‑target breadth uncovers polypharmacology patterns and elucidates how a single molecule might interact with multiple proteins. Leveraging this rich tapestry of interactions, you can train AI models to predict off‑target effects or identify new repurposing opportunities, equipping your organization with a deeper understanding of compound profiles before any lab work begins.\n\n* * *\n\n## Accessing SAIR\n\nSAIR is freely available on [Hugging Face](https://huggingface.co/datasets/SandboxAQ/SAIR). Here’s a quick guide to pull SAIR from Hugging Face, peek at the main table, and (optionally) download a few structure archives.\n\n### 1. Install essentials\n\nWe use the Hub to fetch files and pandas+pyarrow to read the Parquet.\n\n```bash\npip install huggingface_hub pandas pyarrow\n```\n\n### 2. Authenticate\n\nAuthenticate to Hugging Face:\n\n```python\nimport huggingface_hub\nhuggingface_hub.login(token=\"your_auth_token\")\n```\n\n### 3. Load the main table (`sair.parquet`)\n\nThis grabs the file from the Hub and loads it into a DataFrame.\n\n```python\nfrom huggingface_hub import hf_hub_download\nimport pandas as pd\n\nparquet_path = hf_hub_download(\n    repo_id=\"SandboxAQ/SAIR\",\n    filename=\"sair.parquet\",\n    repo_type=\"dataset\"\n)\n\ndf = pd.read_parquet(parquet_path)\ndf.head()\n```\n\n### 4. (Optional) List available structure archives\n\nStructure files are shipped as many `.tar.gz` archives under `structures_compressed/`. List them and pick what you need.\n\n```python\nfrom huggingface_hub import list_repo_files\n\nfiles = [f.split(\"/\")[-1] for f in list_repo_files(\"SandboxAQ/SAIR\", repo_type=\"dataset\")\n         if f.startswith(\"structures_compressed/\") and f.endswith(\".tar.gz\")]\nfiles[:5]\n```\n\n### 5. (Optional) Download and extract structures\n\nEach archive can be large (≈10 GB). Download only the ones you need and extract them locally.\n\n```python\nimport os, tarfile\nfrom huggingface_hub import hf_hub_download\n\ndest = \"sair_structures\"\nos.makedirs(dest, exist_ok=True)\n\nto_get = [\n    \"sair_structures_1006049_to_1016517.tar.gz\",\n    \"sair_structures_100623_to_111511.tar.gz\",\n]\n\nfor name in to_get:\n    tar_path = hf_hub_download(\n        repo_id=\"SandboxAQ/SAIR\",\n        filename=f\"structures_compressed/{name}\",\n        repo_type=\"dataset\",\n        local_dir=dest,\n        local_dir_use_symlinks=False,\n    )\n    with tarfile.open(tar_path, \"r:gz\") as tar:\n        tar.extractall(dest)\n    os.remove(tar_path)  # free disk space\n```\n\nA full version of this script, including more robust logging and validation, is available in the [README](https://huggingface.co/datasets/SandboxAQ/SAIR/blob/main/README.md) file for your convenience. For more details, visit [the SAIR homepage](https://www.sandboxaq.com/sair), read our [manuscript on bioRxiv](https://www.biorxiv.org/content/10.1101/2025.06.17.660168v1), or watch our 25-minute [joint webinar with NVIDIA](https://www.youtube.com/watch?v=y96t8vu9nfg), where we demonstrate SAIR and explain how data is structured within it. Extensive documentation, tutorials, and example benchmarks are available to facilitate its use and accelerate internal adoption.\n\nThe future of drug discovery is data-driven, AI-accelerated, and grounded in scalable, high-quality structural insights. While we don’t yet have AI that can design effective drug therapies with just a prompt, SAIR brings researchers ever closer to that goal with new data and insights that can potentially shave years from even AI-accelerated R&D pipelines.\n\nWe can’t wait to see what researchers will build using SAIR, and SandboxAQ experts are here to support them throughout the discovery process.\n\n* * *\n\n### Questions?\n\nContact the authors or post on the SAIR dataset discussion page.\n\nAuthors: [Arman Zaribafiyan](mailto:arman.zaribafiyan@sandboxquantum.com), [Georgia Channing](mailto:georgia@huggingface.co), [Zane Beckwith](mailto:zane.beckwith@sandboxquantum.com), and [Rudi Plesch](mailto:rudi.plesch@sandboxquantum.com)\n",
  "dateCreated": "2026-10-04T12:50:30.804479567+00:00",
  "date_created": "2025-09-02T16:54:29+00:00",
  "date_modified": "2025-09-02T16:54:29+00:00",
  "date_published": "2025-09-02T16:54:29+00:00",
  "description": "This summer, SandboxAQ released the Structurally Augmented IC50 Repository (SAIR), the largest dataset of co-folded 3D protein-ligand structures paired with experimentally measured IC₅₀ labels, directly linking molecular structure to drug…",
  "external_url": "https://huggingface.co/blog/SandboxAQ/sair-data-accelerating-drug-discovery-with-ai",
  "git": {
    "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e",
    "path": "items/huggingface-co/2025/09/2025-09-02-sair-accelerating-pharma-r-d-with-ai-powered-structural.md",
    "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2025/09/2025-09-02-sair-accelerating-pharma-r-d-with-ai-powered-structural.md"
  },
  "id": "urn:aggr:item:214cb7dfe1ae699605a06edf66c69516fa78806e",
  "isBasedOn": {
    "@id": "https://huggingface.co/blog/SandboxAQ/sair-data-accelerating-drug-discovery-with-ai"
  },
  "mainEntity": {
    "@id": "https://huggingface.co/blog/SandboxAQ/sair-data-accelerating-drug-discovery-with-ai",
    "@type": "CreativeWork",
    "archivedAt": {
      "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-09-02-sair-accelerating-pharma-r-d-with-ai-powered-structural/#webpage"
    },
    "dateModified": "2025-09-02T16:54:29+00:00",
    "datePublished": "2025-09-02T16:54:29+00:00",
    "headline": "SAIR: Accelerating Pharma R&D with AI-Powered Structural Intelligence",
    "timeRequired": "PT7M",
    "url": "https://huggingface.co/blog/SandboxAQ/sair-data-accelerating-drug-discovery-with-ai",
    "wordCount": 1356
  },
  "name": "Archived snapshot: SAIR: Accelerating Pharma R&D with AI-Powered Structural Intelligence",
  "preview": null,
  "reading_minutes": 7,
  "replicated_at": null,
  "source": "huggingface-co",
  "source_name": "Hugging Face - Blog",
  "summary": null,
  "tags": [],
  "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00",
  "title": "SAIR: Accelerating Pharma R&D with AI-Powered Structural Intelligence",
  "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-09-02-sair-accelerating-pharma-r-d-with-ai-powered-structural/",
  "word_count": 1356
}