{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/ampcode-com/2025-07-30-amp-tab-30-faster/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "Response times of [Amp Tab](https://ampcode.com/manual#amp-tab), our in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage.\n\nWe worked together with [Baseten](https://www.baseten.co/) to optimize our custom deployment. The new infrastructure delivers roughly 2x performance improvements by switching to TensorRT-LLM as the inference engine and implementing KV caching with speculative decoding.\n\nThis new infrastructure also includes a modified version of lookahead decoding that uses an improved n-gram candidate selection algorithm and variable-length speculations, which reduces both draft tokens and compute per iteration compared to standard implementations.\n\n![Chart showing Amp Tab latency improvements over time](https://static.ampcode.com/news/next-cursor-latency-trend-dark.png)\n", "authors": [], "captured_at": "2026-10-04T12:50:30.804479567+00:00", "category": "programming", "content_html": "

Response times of Amp Tab, our in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage.

\n

We worked together with Baseten to optimize our custom deployment. The new infrastructure delivers roughly 2x performance improvements by switching to TensorRT-LLM as the inference engine and implementing KV caching with speculative decoding.

\n

This new infrastructure also includes a modified version of lookahead decoding that uses an improved n-gram candidate selection algorithm and variable-length speculations, which reduces both draft tokens and compute per iteration compared to standard implementations.

\n

\"Chart

\n", "content_markdown": "Response times of [Amp Tab](https://ampcode.com/manual#amp-tab), our in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage.\n\nWe worked together with [Baseten](https://www.baseten.co/) to optimize our custom deployment. The new infrastructure delivers roughly 2x performance improvements by switching to TensorRT-LLM as the inference engine and implementing KV caching with speculative decoding.\n\nThis new infrastructure also includes a modified version of lookahead decoding that uses an improved n-gram candidate selection algorithm and variable-length speculations, which reduces both draft tokens and compute per iteration compared to standard implementations.\n\n![Chart showing Amp Tab latency improvements over time](https://static.ampcode.com/news/next-cursor-latency-trend-dark.png)\n", "dateCreated": "2026-10-04T12:50:30.804479567+00:00", "date_created": "2025-07-30T00:00:00+00:00", "date_modified": "2025-07-30T00:00:00+00:00", "date_published": "2025-07-30T00:00:00+00:00", "description": "Response times of Amp Tab, our in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage. We worked together with Baseten to optimize our custom deployment. The new infrastructure delivers roughly 2x…", "external_url": "https://ampcode.com/news/amp-tab-30-percent-faster", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/ampcode-com/2025/07/2025-07-30-amp-tab-30-faster.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/ampcode-com/2025/07/2025-07-30-amp-tab-30-faster.md" }, "id": "urn:aggr:item:2acd6175ff8584b4bbed07c9084bed9db36ee9db", "isBasedOn": { "@id": "https://ampcode.com/news/amp-tab-30-percent-faster" }, "mainEntity": { "@id": "https://ampcode.com/news/amp-tab-30-percent-faster", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/ampcode-com/2025-07-30-amp-tab-30-faster/#webpage" }, "dateModified": "2025-07-30T00:00:00+00:00", "datePublished": "2025-07-30T00:00:00+00:00", "headline": "Amp Tab 30% Faster", "image": { "@type": "ImageObject", "caption": "Chart showing Amp Tab latency improvements over time", "height": 90, "url": "https://static.ampcode.com/news/next-cursor-latency-trend-dark.png", "width": 160 }, "timeRequired": "PT1M", "url": "https://ampcode.com/news/amp-tab-30-percent-faster", "wordCount": 93 }, "name": "Archived snapshot: Amp Tab 30% Faster", "preview": { "alt": "Chart showing Amp Tab latency improvements over time", "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://static.ampcode.com/news/next-cursor-latency-trend-dark.png", "width": 160 }, "reading_minutes": 1, "replicated_at": null, "source": "ampcode-com", "source_name": "Amp News", "summary": "Response times of Amp Tab, our in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage. We worked together with Baseten to optimize our custom deployment. The new infrastructure delivers roughly 2x performance improvements by switching to TensorRT-LLM as the inference engine and implementing KV caching with speculative decoding. This new infrastructure also includes a modified version of lookahead decoding that uses an improved n-gram candidate selection algorithm and variable-length speculations, which reduces both draft tokens and compute per iteration compared to standard implementations.", "tags": [], "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00", "title": "Amp Tab 30% Faster", "url": "https://aggr.aymericbeaumet.com/items/ampcode-com/2025-07-30-amp-tab-30-faster/", "word_count": 93 }