![\"unnamed\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/unnamed.webp\")
I did a fireside chat at Sequoia Ascent 2026. The YouTube video is here:
\n\nAs an experiment, I fed an LLM all of my recent blog posts and tweets, then I had it read this video's transcript and produce 1) a summary and 2) a cleaned up transcript (correcting all transcription mistakes, getting rid of fill words, etc). I am posting both of these below. These can be useful for both people who may want to just read the summary in text format, but also for LLMs so that my content is legible and available to them.
\nAI generated content below for this talk follows. I used a top capability model (in this case Codex 5.5) and read the content and it reads ok without glaring mistakes.
\nI recently joined Stephanie Zhan for a fireside chat at Sequoia Ascent 2026, speaking with founders about the recent shift in AI agents, what it means for software, and how I think about the next wave of AI-native companies.
\nThe transcript from the event is a bit noisy, so I wanted to write up the main intellectual content in a cleaner form. The short version is that I think we have crossed a new threshold. LLMs are no longer just chatbots or autocomplete. They are becoming a new programmable layer for digital work.
\nThis is the compact version of the conversation.
\nI said recently that I have never felt more behind as a programmer.
\nThe reason is not that programming became harder in the old sense. It is that the default workflow changed. For much of 2025, tools like Claude Code, Codex, and Cursor-like agents were useful but still required frequent correction. Around December 2025, I felt a step change: the generated chunks got larger, more coherent, and more reliable. I started trusting the agents with more of the work.
\nThe unit of programming changed from typing lines of code to delegating larger "macro actions":
\nThis is why I think the profession is being refactored. The programmer is increasingly not just a code writer, but an orchestrator of agents.
\nI think of this as the next step in a sequence:
\nIn Software 3.0, the context window becomes the main lever. The LLM is an interpreter over that context, performing computation over digital information.
\nOne example is installation. In the old world, installing a complex tool across many environments required a brittle shell script full of conditionals. In the Software 3.0 world, the installer can be a block of instructions you paste into an agent. The agent reads the local environment, debugs errors, adapts to the machine, and completes the setup.
\nThat is a different kind of program. It is less precise, but more adaptive.
\nI used MenuGen as an example of a deeper shift.
\nMenuGen was a traditional web app: take a picture of a restaurant menu, OCR the dish names, generate images of the dishes, and render the result in a UI. It required frontend code, APIs, image generation, deployment, auth, payments, secrets, and infrastructure.
\nBut later, I saw the Software 3.0 version: take a photo of the menu, give it to a multimodal model, and ask it to render dish images directly onto the menu image.
\nIn that version, much of the app disappears. The neural network directly transforms input media into output media. The old software stack was scaffolding around a transformation the model can now perform directly.
\nThis is one of the most important founder implications: AI is not just a faster way to build the old apps. Some apps should stop existing as apps.
\nThe shift is broader than coding. LLMs automate forms of information processing that were not previously programmable.
\nMy LLM Wiki pattern is the clearest example. Instead of using retrieval-augmented generation to answer questions from raw documents each time, an agent incrementally compiles raw sources into a persistent Markdown wiki: summaries, entity pages, concept pages, contradictions, cross-links, logs, and evolving synthesis.
\nNo classical program could robustly maintain that kind of knowledge base across messy human documents. But an LLM can.
\nThe lesson: do not only ask, "What existing workflow can AI speed up?" Also ask, "What information transformation was impossible before, but is now natural?"
\nMy core automation framework is:
\nIf a task has an automatic reward or success signal, models can practice it. This is why math, coding, tests, benchmarks, games, and many engineering tasks improve so quickly. They are resettable, repeatable, and rewardable.
\nThis also explains why coding agents feel dramatically better than many ordinary chatbot experiences. Coding gives the model feedback: tests pass or fail, programs run or crash, diffs can be inspected, benchmarks can be measured.
\nThe interview added an important refinement to the verifiability thesis.
\nModel capability is not only about whether a task is verifiable. It also depends on whether the task was emphasized by labs during training, post-training, synthetic data generation, and reinforcement learning.
\nA rough formula:
\ncapability spike ~= verifiability x training attention x data coverage x economic value\nChess is a good example. When GPT-4 improved at chess, that was not necessarily because general intelligence smoothly improved everywhere. It may also have been because much more chess data was included in the training mix.
\nThis matters because frontier models do not come with a manual. They are artifacts of pretraining mixtures, RL environments, benchmark pressure, product priorities, and economic incentives. They spike in some places and behave strangely in others.
\nSo the practical question for a founder is: are you on the model's rails?
\nIf your task sits inside a region that is verifiable and heavily trained, the model may fly. If not, it may fail in surprisingly basic ways. You may need better context, tools, fine-tuning, your own evals, or your own reinforcement learning environment.
\nI distinguish two related but different ideas:
\nVibe coding is fine for prototypes and personal tools. Agentic engineering is what serious teams need.
\nThe agentic engineer does not blindly accept generated code. They design specs, supervise plans, inspect diffs, write tests, create evaluation loops, manage permissions, isolate worktrees, and preserve quality.
\nMy MenuGen payment bug is a useful example. The agent tried to match Stripe purchases to Google accounts using email addresses. That is plausible code, but bad system design: the Stripe email and Google login email can differ. A human needs enough product and engineering judgment to insist on persistent user IDs.
\nThe frontier skill is not memorizing every API detail. Agents can remember whether a tensor library uses dim, axis, keepdim, reshape, or permute. The human still needs to understand the underlying concepts: storage, views, memory copies, invariants, identity, security boundaries, and the shape of the system.
If agentic engineering is the new professional skill, hiring should test it directly.
\nTraditional coding puzzles are increasingly mismatched. A better interview might be: build a substantial project with agents, deploy it, make it secure, and then have adversarial agents try to break it.
\nThis tests the real skill:
\nThe old "10x engineer" idea may become much more extreme. People who master agentic workflows may outperform others by far more than 10x.
\nFor founders, one important opportunity is finding domains that are valuable, verifiable, and undertrained by frontier labs.
\nIf you can create a domain-specific environment where models can try actions and receive reliable rewards, you may be able to improve performance with fine-tuning or reinforcement learning even if the base model is not already excellent there.
\nThe most obvious domains, like coding and math, are already heavily targeted by labs. But many economically important domains may have latent verifiable structure that has not yet been exploited.
\nThat is a startup wedge.
\nMost software is still built for humans clicking through screens.
\nDocs say things like "go to this URL, click this button, open this settings panel." But increasingly the user is not the human directly. The user is the human's agent.
\nThis means products need agent-native surfaces:
\nI think about this in terms of sensors and actuators. A sensor turns some state of the world into digital information. An actuator lets an agent change something. The future stack is agents using sensors and actuators on behalf of people and organizations.
\nThe MenuGen deployment story remains a useful benchmark. Building the app was easy compared to wiring Vercel, auth, payments, DNS, secrets, and production settings. In a mature agent-native world, I should be able to say "build MenuGen" and have the agent deploy the whole thing without manual clicking.
\nMy Animals vs. Ghosts framing is a way to avoid bad intuitions.
\nLLMs are not animals. They do not have biological drives, embodied survival pressure, curiosity, play, or intrinsic motivation in the animal sense. They are statistical simulations of human artifacts, shaped by pretraining, post-training, RL, product feedback, and economic incentives.
\nThis matters because anthropomorphic expectations mislead us. These systems can be brilliant in one moment and bizarrely dumb in the next. They are not smooth human minds. They are jagged, alien tools.
\nThe right posture is neither dismissal nor blind trust. It is empirical familiarity: learn where they work, where they fail, what they were trained for, and how to build guardrails around them.
\nWe ended on education. There is a line I keep returning to:
\n\n\nYou can outsource your thinking, but you can't outsource your understanding.
\n
Even if agents do more of the work, the human still needs understanding to direct them. You need to know what is worth building, what question matters, what result is suspicious, and what tradeoff is acceptable.
\nThis is why I am interested in LLM knowledge bases. They are not just answer machines. They are tools for transforming information into understanding.
\nThis also connects to my tiny microGPT project: a complete GPT training and inference implementation in a single dependency-free Python file. The educational artifact becomes small enough for both humans and agents to inspect. The human expert contributes the distilled artifact and the taste behind it; the agent can then explain it interactively to each learner.
The main thesis of the conversation is that AI is becoming a new operating layer for digital work.
\nThe scarce thing is shifting:
\nFor founders, the most important questions are:
\nMy current worldview is not that AI simply makes everyone faster at the old work. It is that the work itself is being reorganized around agents. Software, research, education, infrastructure, and knowledge work are all becoming variations of the same pattern:
\ndefine the context\ndefine the tools\ndefine the feedback loop\ndefine the guardrails\nlet agents work\npreserve human understanding\nEdited transcript. Lightly cleaned for readability, with obvious transcription errors corrected, filler removed, and a few relevant links added.
\nKonstantine: Someone you all know, someone who has become, in this AI revolution, a teacher of AI. In every revolution there is the technologist, but there is also the teacher, the person who actually informs and instructs how this transformation is going to happen. Andrej has become that teacher to the world.
\nEarly at Autopilot at Tesla, co-founder of OpenAI, he left it all to start Eureka Labs, where he leaned into the idea of an AI that was a true instructor. We're happy to have Andrej Karpathy with our partner Stephanie Zhan.
\nStephanie: Hi everyone. We're excited for our first special guest. He has helped build modern AI, explain modern AI, and occasionally rename modern AI.
\nHe helped co-found OpenAI. He helped get Autopilot working at Tesla. And he has a rare gift for making the most complex technical shifts feel both accessible and inevitable.
\nYou all know him for having coined the term vibe coding last year. But just in the last few months, he said something even more startling: he has never felt more behind as a programmer. That's where we're starting today. Thank you, Andrej, for joining us.
\nAndrej: Hello. Excited to be here and to kick us off.
\nStephanie: A couple of months ago, you said you've never felt more behind as a programmer. That's startling to hear from you, of all people. Can you help us unpack that? Was that feeling exhilarating or unsettling?
\nAndrej: A mixture of both, for sure.
\nLike many of you, I've been using agentic tools like Claude Code, Codex, and adjacent things for a while, maybe over the last year. They were very good at chunks of code, but sometimes they would mess up and you had to edit them. They were helpful.
\nThen I would say December was a clear point. I was on a break, so I had more time. I think many other people were similar. I started to notice that with the latest models, the chunks just came out fine. Then I kept asking for more and they still came out fine. I couldn't remember the last time I corrected it. I started trusting the system more and more.
\nI do think it was a stark transition. A lot of people experienced AI last year as a ChatGPT-adjacent thing, but you really had to look again as of December, because things changed fundamentally, especially in this agentic, coherent workflow. It really started to work.
\nThat realization sent me down the rabbit hole of infinite side projects. My side-projects folder is extremely full with random things. I was coding all the time. That happened in December, and I've been looking at the repercussions since.
\nStephanie: You've talked about LLMs as a new computer. It isn't just better software; it's a new computing paradigm. Software 1.0 was explicit rules. Software 2.0 was learned weights. Software 3.0 is this. If that is true, what does a team build differently the day they actually believe it?
\nAndrej: Software 1.0 is writing code. Software 2.0 is programming by creating datasets and training neural networks. Programming becomes arranging datasets, objectives, and neural network architectures.
\nThen what happened is that if you train GPT models or LLMs on a sufficiently large set of tasks, implicitly, because the internet contains many tasks, these models become programmable computers in a certain sense.
\nSoftware 3.0 is about programming through prompting. What's in the context window is your lever over the interpreter, and the interpreter is the LLM. It interprets your context and performs computation in digital information space.
\nA few examples drove this home for me. When OpenClaw came out, to install it you would normally expect a shell script. But to target many platforms and many kinds of computers, shell scripts usually balloon and become extremely complex. You're stuck in the Software 1.0 universe of wanting to write exact code.
\nThe OpenClaw installation was instead a block of text that you copy and paste into your agent. It is like a little skill: copy this, give it to your agent, and it will install OpenClaw. That is more powerful because you're working in the Software 3.0 paradigm. You don't have to spell out every detail. The agent has intelligence. It looks at your environment, performs intelligent actions, and debugs in the loop.
\nThat is a different way of thinking. What is the piece of text to copy-paste into your agent? That is now part of the programming paradigm.
\nAnother example is MenuGen. You sit down at a restaurant, get a menu, and there are no pictures. I don't know what many of these things are. I wanted to take a photo of the menu and get pictures of what those dishes might look like in a generic sense.
\nSo I built an app. You upload a photo, it OCRs all the titles, uses an image generator to get pictures, and shows them to you. It runs on Vercel and rerenders the menu.
\nThen I saw the Software 3.0 version, which blew my mind. You take the photo, give it to Gemini, and say: use Nano Banana to overlay the things onto the menu. It returns an image of the menu I took, but with pictures rendered into the pixels.
\nAll of MenuGen is spurious in that framing. It is working in the old paradigm. That app shouldn't exist. In the Software 3.0 paradigm, the neural network does more of the work. Your prompt or context is the image, and the output is an image. There is no need for all the app machinery in between.
\nPeople have to reframe. Don't only work in the existing paradigm and think of AI as a speedup of what exists. New things are available now.
\nAnd it is not just programming becoming faster. This is more general information processing that is now automatable. Previous code worked over structured data. You wrote code over structured data.
\nWith my LLM knowledge bases project, you get LLMs to create wikis for your organization or for you personally. This is not a program in the old sense. There was no code that could create a knowledge base based on a bunch of messy facts. But now you can take documents, recompile them, reorder them, and create something new and interesting as a reframing of the data.
\nThese are new things that weren't possible before. I keep trying to come back to that: not only what can we do faster, but what couldn't be possible before? That is more exciting.
\nStephanie: I love the MenuGen progression. If you extrapolate further, what is the 2026 equivalent of building websites in the 90s, mobile apps in the 2010s, or SaaS in the cloud era? What will look obvious in hindsight that is still mostly unbuilt today?
\nAndrej: Going with the MenuGen example, a lot of this code shouldn't exist. The neural network should be doing most of the work.
\nThe extrapolation looks very weird. You could imagine completely neural computers in a certain sense. Imagine a device that takes raw video or audio into a neural net and uses diffusion to render a UI unique for that moment.
\nIn the early days of computing, people were a little confused about whether computers would look like calculators or neural nets. In the 1950s and 1960s, it was not obvious which way it would go. We went down the calculator path and built classical computing.
\nNeural nets are currently running virtualized on existing computers. But you can imagine a flip where the neural net becomes the host process and CPUs become coprocessors. Intelligence compute and neural-network compute become the dominant spend of FLOPs.
\nYou can imagine something foreign, where neural nets do most of the heavy lifting and use tools as a historical appendage for deterministic tasks. What is really running the show is neural nets networked in some way.
\nThat is the extrapolation, but I think we will get there piece by piece.
\nStephanie: I'd love to talk about verifiability: the idea that AI will automate faster and more easily in domains where the output can be verified. If that framework is right, what work is about to move much faster than people realize? And what professions do people think are safe, but are actually highly verifiable?
\nAndrej: Traditional computers automate what you can specify in code. This latest round of LLMs can automate what you can verify.
\nWhen frontier labs train these LLMs, they train them in giant reinforcement learning environments with verification rewards. Because of that, models progress and become jagged entities. They peak in capability in verifiable domains like math, code, and adjacent areas, and they stagnate or remain rough around the edges where things are not in that space.
\nI wrote about verifiability because I was trying to understand why these things are so jagged. Some of it has to do with how labs train the models. Some of it also has to do with what labs focus on and what they put into the data distribution. Some things are significantly more valuable economically, so labs create more environments for those settings. Code is a good example.
\nThere are probably many verifiable environments that you could think about that did not make it into the mix because they are not as economically useful to have capability around.
\nOne favorite example for a while was: how many letters are in "strawberry"? Models famously got this wrong. That has now been patched. The newer example is: I want to go to a car wash to wash my car, and it's 50 meters away. Should I drive or walk? State-of-the-art models may tell you to walk because it's close.
\nHow is it possible that a state-of-the-art model can refactor a 100,000-line codebase or find zero-day vulnerabilities, yet tells me to walk to the car wash? That's jaggedness. To the extent models remain jagged, it means you need to be in the loop. You need to treat them as tools and stay in touch with what they are doing.
\nMy writing on verifiability is trying to understand this pattern. I think it is some combination of "verifiable" plus "labs care."
\nAnother anecdote is chess. From GPT-3.5 to GPT-4, people noticed that chess improved a lot. Some people thought that was just general capability progress. But I think it is public information that a large amount of chess data made it into the pretraining set. Because it was in the data distribution, the model improved much more than it would by default.
\nSomeone at OpenAI decided to add that data, and now there is a capability spike. That is why I stress this dimension: we are slightly at the mercy of what the labs do and what they put into the mix. You have to explore the model they give you. It has no manual. It works in some settings and not others.
\nIf you are in the circuits that were part of reinforcement learning, you fly. If you are outside the data distribution, you struggle. You have to figure out which circuits your application is in. If you are not in those circuits, then you have to look at fine-tuning or doing some of your own work, because it may not come out of the LLM out of the box.
\nStephanie: If you were a founder today, and you were solving a tractable, verifiable problem, but you looked around and saw that the labs have started getting to escape velocity in obvious domains like math and coding, what would your advice be?
\nAndrej: Verifiability makes something tractable in the current paradigm because you can throw a huge amount of reinforcement learning at it.
\nThat remains true even if the labs are not focusing on it directly. If you are in a verifiable setting where you can create reinforcement learning environments or examples, then you can potentially do your own fine-tuning and benefit from it. That technology fundamentally works. If you have diverse datasets or RL environments, you can use a fine-tuning framework, pull the lever, and get something that works pretty well.
\nI don't want to give away specific examples, but there are valuable reinforcement learning environments that people could think of that are not part of the current frontier-lab mix.
\nStephanie: On the flip side, what still feels automatable only from a distance? What domains or professions are safer than others?
\nAndrej: Ultimately, almost everything can be made verifiable to some extent, some things more easily than others. Even for writing, you can imagine having a council of LLM judges and getting something reasonable.
\nSo it is more about what is easy or hard.
\nStephanie: Last year you coined the term vibe coding. Today we are in a world that feels more serious, more agentic engineering. What is the difference between the two, and what would you call what we are in today?
\nAndrej: Vibe coding is about raising the floor for everyone in terms of what they can do in software. Everyone can vibe code anything, and that is amazing.
\nAgentic engineering is about preserving the quality bar of professional software. You are not allowed to introduce vulnerabilities because of vibe coding. You are still responsible for your software, just as before. But can you go faster? Spoiler: you can. The question is how to do that properly.
\nI call it agentic engineering because it is an engineering discipline. You have agents, which are spiky entities. They are fallible and stochastic, but extremely powerful. How do you coordinate them to go faster without sacrificing your quality bar?
\nVibe coding raises the floor. Agentic engineering is about extrapolating the ceiling. I think there is a very high ceiling on agentic-engineer capability. People used to talk about the 10x engineer. I think this is magnified a lot more. 10x is not the speedup people can gain. People who are very good at this can peak much higher than that.
\nStephanie: Last year Sam Altman came to Ascent and said people of different generations use ChatGPT differently. If you're in your 30s, you use it as a Google search replacement. If you're in your teens, ChatGPT is your gateway to the internet.
\nWhat is the parallel in coding? If we watched two people code using OpenClaw, Claude Code, or Codex, one mediocre and one fully AI-native, how would you describe the difference?
\nAndrej: It is about getting the most out of the tools available, using their features, and investing in your own setup.
\nEngineers have always done this with tools like Vim or VS Code. Now the tools are Claude Code, Codex, and so on. You invest in your setup and use what is available.
\nOne related thought is hiring. Many people want to hire strong agentic engineers, but most hiring processes have not been refactored for agentic-engineer capability. If you are giving out small puzzles to solve, that is still the old paradigm.
\nHiring should look more like: give someone a big project and see them implement it. For example, write a Twitter clone for agents, make it good and secure, then have agents simulate activity on it. Then I will use ten Codex agents to try to break the website you deployed, and they should not be able to break it.
\nWatching people in that setting, building a bigger project and using the tooling, is closer to what I would look for.
\nStephanie: As agents do more, what human skill becomes more valuable, not less?
\nAndrej: Right now the agents are like interns. You still have to be in charge of aesthetics, judgment, taste, and oversight.
\nOne of my favorite examples is from MenuGen. You sign up with a Google account, but you purchase credits using Stripe. Both have email addresses. My agent tried to assign purchased credits by matching the Stripe email address to the Google email address.
\nBut those can be different emails. The user might not get the credits they purchased. Why would you use email addresses to cross-correlate funds? You need a persistent user ID. This is the kind of mistake agents still make.
\nPeople have to be in charge of the spec and plan. I don't even fully like "plan mode" as a concept, though it is useful. There is something more general: you work with your agent to design a detailed spec, maybe basically the docs, and get agents to write them. You are in charge of oversight and the top-level categories. The agents do much of the work underneath.
\nAs another example, with tensors in neural networks, there are many details across PyTorch, NumPy, pandas, and so on: dim versus axis, reshape, permute, transpose, keepdim. I don't remember this stuff anymore because I don't have to. These details are handled by the intern because agents have good recall.
But you still have to understand the fundamentals. You need to know that there is underlying tensor storage, that you can manipulate a view of the same storage, or create different storage, which is less efficient. You still need to know enough to avoid copying memory unnecessarily.
\nSo you are in charge of taste, engineering, design, and whether the system makes sense. You ask for the right things: for example, we tie everything to unique user IDs. The agents fill in the blanks.
\nStephanie: Do you think taste and judgment matter less over time, or does the ceiling just keep rising?
\nAndrej: I hope it improves. The reason it does not improve right now is probably that it is not part of the reinforcement learning. There may be no aesthetics reward, or it is not good enough.
\nWhen I look at the code, sometimes I get a heart attack. It is not always amazing code. It can be bloated, copy-pasted, awkwardly abstracted, brittle. It works, but it is gross. I hope this improves in future models.
\nA good example is my microGPT project, where I tried to simplify LLM training as much as possible. The models hate this. They can't do it. I kept trying to prompt an LLM to simplify more and more, and it just couldn't. You feel like you are outside the RL circuits. It feels like pulling teeth.
So people remain in charge of this for now. But I don't think there is anything fundamental preventing improvement. The labs just haven't done it yet.
\nStephanie: I'd love to come back to jagged forms of intelligence. You wrote a thought-provoking piece around Animals vs. Ghosts: we are not building animals, we are summoning ghosts. These are jagged forms of intelligence shaped by data and reward functions, but not by intrinsic motivation, fun, curiosity, or empowerment in the way evolution shaped animals.
\nWhy does that framing matter? What does it change about how you build, deploy, evaluate, or trust them?
\nAndrej: I wrote about it because I am trying to wrap my head around what these things are. If you have a good model of what they are and are not, you will be more competent at using them.
\nI don't know if the framing has direct practical power. It is a little philosophical. But it is about coming to terms with the fact that these things are not animal intelligence. If you yell at them, they are not going to work better or worse. They are statistical simulation circuits. The substrate is pretraining, then reinforcement learning bolted on top.
\nIt is a mindset: what am I interacting with, what is likely to work, what is not likely to work, and how do I modify it? I don't have five obvious outcomes that make your system better. It is more about being suspicious of the system and figuring it out empirically over time.
\nStephanie: You are deep in working with agents that do not just chat. They have real permissions, local context, and actually take action on your behalf. What does the world look like when we all live in that world?
\nAndrej: A lot of people here are probably excited about what the agent-native environment looks like. Everything has to be rewritten. Most things are still fundamentally written for humans.
\nWhen I use frameworks or libraries, the docs are still written for humans. This is my favorite pet peeve. Why are people still telling me what to do? I don't want to do anything. What is the thing I should copy-paste to my agent?
\nEvery time I am told "go to this URL" or "click here," I think: no. The industry has to decompose workloads into sensors and actuators over the world. How do we make things agent-native? How do we describe them to agents first, and build automation around data structures that are legible to LLMs?
\nI hope there is a lot of agent-first infrastructure. With MenuGen, the hard part was not writing the code. The trouble was deploying it on Vercel, wiring services, settings, DNS, auth, payments, secrets, and production configuration.
\nI would hope I could prompt an LLM: build MenuGen. Then I don't touch anything, and it is deployed on the internet. That would be a good test of whether our infrastructure is becoming agent-native.
\nUltimately, I do think we are going toward a world where people and organizations have agent representation. My agent will talk to your agent to figure out meeting details and other tasks. That is roughly where things are going.
\nStephanie: We have to end on education. You are probably one of the best in the world at making complex technical concepts simple, and you think deeply about education. What remains worth learning deeply when intelligence gets cheap?
\nAndrej: There was a tweet that blew my mind recently, and I keep thinking about it:
\n\n\nYou can outsource your thinking, but you can't outsource your understanding.
\n
That is nicely put. I am still part of the system. Information still has to make it into my brain. I am becoming the bottleneck of even knowing what we are trying to build, why it is worth doing, and how to direct my agents.
\nSomething still has to direct the thinking and processing. That is constrained by understanding.
\nThis is one reason I am excited about LLM knowledge bases. They are a way for me to process information. Whenever I see a different projection onto information, I feel like I gain insight. It is synthetic data generation over fixed data.
\nWhen I read an article, I have my wiki being built up from those articles. I love asking questions about it. Ultimately these are tools to enhance understanding. Understanding is still the bottleneck because you cannot be a good director if you do not understand.
\nThe LLMs do not fully excel at understanding. You are still uniquely in charge of that. Tools that enhance understanding are incredibly interesting and exciting.
\nStephanie: I'm excited to come back here in a couple of years and see if we have been fully automated out of the loop, and whether they take care of understanding as well. Thank you so much, Andrej.
\nAndrej: Thank you.
\nKonstantine: Stephanie, Andrej, thank you so much.
\n", "date_modified": "2026-04-30T16:30:08.883848+00:00", "date_published": "2026-04-30T16:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/sequoia-ascent-2026/", "id": "urn:aggr:item:3db965f4a325e028f9e712e84e82e40e865008b2", "summary": "Summary of my talk at Sequoia Ascent", "tags": [ "programming" ], "title": "Sequoia Ascent 2026 summary", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2026-04-30-sequoia-ascent-2026-summary/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "
2025 has been a strong and eventful year of progress in LLMs. The following is a list of personally notable and mildly surprising "paradigm changes" - things that altered the landscape and stood out to me conceptually.
\nAt the start of 2025, the LLM production stack in all labs looked something like this:
\nThis was the stable and proven recipe for training a production-grade LLM for a while. In 2025, Reinforcement Learning from Verifiable Rewards (RLVR) emerged as the de facto new major stage to add to this mix. By training LLMs against automatically verifiable rewards across a number of environments (e.g. think math/code puzzles), the LLMs spontaneously develop strategies that look like "reasoning" to humans - they learn to break down problem solving into intermediate calculations and they learn a number of problem solving strategies for going back and forth to figure things out (see DeepSeek R1 paper for examples). These strategies would have been very difficult to achieve in the previous paradigms because it's not clear what the optimal reasoning traces and recoveries look like for the LLM - it has to find what works for it, via the optimization against rewards.
\nUnlike the SFT and RLHF stage, which are both relatively thin/short stages (minor finetunes computationally), RLVR involves training against objective (non-gameable) reward functions which allows for a lot longer optimization. Running RLVR turned out to offer high capability/$, which gobbled up the compute that was originally intended for pretraining. Therefore, most of the capability progress of 2025 was defined by the LLM labs chewing through the overhang of this new stage and overall we saw ~similar sized LLMs but a lot longer RL runs. Also unique to this new stage, we got a whole new knob (and and associated scaling law) to control capability as a function of test time compute by generating longer reasoning traces and increasing "thinking time". OpenAI o1 (late 2024) was the very first demonstration of an RLVR model, but the o3 release (early 2025) was the obvious point of inflection where you could intuitively feel the difference.
\n2025 is where I (and I think the rest of the industry also) first started to internalize the "shape" of LLM intelligence in a more intuitive sense. We're not "evolving/growing animals", we are "summoning ghosts". Everything about the LLM stack is different (neural architecture, training data, training algorithms, and especially optimization pressure) so it should be no surprise that we are getting very different entities in the intelligence space, which are inappropriate to think about through an animal lens. Supervision bits-wise, human neural nets are optimized for survival of a tribe in the jungle but LLM neural nets are optimized for imitating humanity's text, collecting rewards in math puzzles, and getting that upvote from a human on the LM Arena. As verifiable domains allow for RLVR, LLMs "spike" in capability in the vicinity of these domains and overall display amusingly jagged performance characteristics - they are at the same time a genius polymath and a confused and cognitively challenged grade schooler, seconds away from getting tricked by a jailbreak to exfiltrate your data.
\n![\"G6zymj4a0AMNJkJ\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/g6zymj4a0amnjkj.webp\")
Related to all this is my general apathy and loss of trust in benchmarks in 2025. The core issue is that benchmarks are almost by construction verifiable environments and are therefore immediately susceptible to RLVR and weaker forms of it via synthetic data generation. In the typical benchmaxxing process, teams in LLM labs inevitably construct environments adjacent to little pockets of the embedding space occupied by benchmarks and grow jaggies to cover them. Training on the test set is a new art form.
\nWhat does it look like to crush all the benchmarks but still not get AGI?
\nI have written a lot more on the topic of this section here:
\n\nWhat I find most notable about Cursor (other than its meteoric rise this year) is that it convincingly revealed a new layer of an "LLM app" - people started to talk about "Cursor for X". As I highlighted in my Y Combinator talk this year (transcript and video), LLM apps like Cursor bundle and orchestrate LLM calls for specific verticals:
\nA lot of chatter has been spent in 2025 on how "thick" this new app layer is. Will the LLM labs capture all applications or are there green pastures for LLM apps? Personally I suspect that LLM labs will trend to graduate the generally capable college student, but LLM apps will organize, finetune and actually animate teams of them into deployed professionals in specific verticals by supplying private data, sensors and actuators and feedback loops.
\nClaude Code (CC) emerged as the first convincing demonstration of what an LLM Agent looks like - something that in a loopy way strings together tool use and reasoning for extended problem solving. In addition, CC is notable to me in that it runs on your computer and with your private environment, data and context. I think OpenAI got this wrong because they focused their early codex / agent efforts on cloud deployments in containers orchestrated from ChatGPT instead of simply localhost. And while agent swarms running in the cloud feels like the "AGI endgame", we live in an intermediate and slow enough takeoff world of jagged capabilities that it makes more sense to run the agents directly on the developer's computer. Note that the primary distinction that matters is not about where the "AI ops" happen to run (in the cloud, locally or whatever), but about everything else - the already-existing and booted up computer, its installation, context, data, secrets, configuration, and the low-latency interaction. Anthropic got this order of precedence correct and packaged CC into a delightful, minimal CLI form factor that changed what AI looks like - it's not just a website you go to like Google, it's a little spirit/ghost that "lives" on your computer. This is a new, distinct paradigm of interaction with an AI.
2025 is the year that AI crossed a capability threshold necessary to build all kinds of impressive programs simply via English, forgetting that the code even exists. Amusingly, I coined the term "vibe coding" in this shower of thoughts tweet totally oblivious to how far it would go :). With vibe coding, programming is not strictly reserved for highly trained professionals, it is something anyone can do. In this capacity, it is yet another example of what I wrote about in Power to the people: How LLMs flip the script on technology diffusion, on how (in sharp contrast to all other technology so far) regular people benefit a lot more from LLMs compared to professionals, corporations and governments. But not only does vibe coding empower regular people to approach programming, it empowers trained professionals to write a lot more (vibe coded) software that would otherwise never be written. In nanochat, I vibe coded my own custom highly efficient BPE tokenizer in Rust instead of having to adopt existing libraries or learn Rust at that level. I vibe coded many projects this year as quick app demos of something I wanted to exist (e.g. see menugen, llm-council, reader3, HN time capsule). And I've vibe coded entire ephemeral apps just to find a single bug because why not - code is suddenly free, ephemeral, malleable, discardable after single use. Vibe coding will terraform software and alter job descriptions.
\nGoogle Gemini Nano banana is one of the most incredible, paradigm-shifting models of 2025. In my world view, LLMs are the next major computing paradigm similar to computers of the 1970s, 80s, etc. Therefore, we are going to see similar kinds of innovations for fundamentally similar kinds of reasons. We're going to see equivalents of personal computing, of microcontrollers (cognitive core), or internet (of agents), etc etc. In particular, in terms of the UIUX, "chatting" with LLMs is a bit like issuing commands to a computer console in the 1980s. Text is the raw/favored data representation for computers (and LLMs), but it is not the favored format for people, especially at the input. People actually dislike reading text - it is slow and effortful. Instead, people love to consume information visually and spatially and this is why the GUI has been invented in traditional computing. In the same way, LLMs should speak to us in our favored format - in images, infographics, slides, whiteboards, animations/videos, web apps, etc. The early and present version of this of course are things like emoji and Markdown, which are ways to "dress up" and lay out text visually for easier consumption with titles, bold, italics, lists, tables, etc. But who is actually going to build the LLM GUI? In this world view, nano banana is a first early hint of what that might look like. And importantly, one notable aspect of it is that it's not just about the image generation itself, it's about the joint capability coming from text generation, image generation and world knowledge, all tangled up in the model weights.
\nTLDR. 2025 was an exciting and mildly surprising year of LLMs. LLMs are emerging as a new kind of intelligence, simultaneously a lot smarter than I expected and a lot dumber than I expected. In any case they are extremely useful and I don't think the industry has realized anywhere near 10% of their potential even at present capability. Meanwhile, there are so many ideas to try and conceptually the field feels wide open. And as I mentioned on my Dwarkesh pod earlier this year, I simultaneously (and on the surface paradoxically) believe that we will both see rapid and continued progress and that yet there is a lot of work to be done. Strap in.
\n", "date_modified": "2025-12-20T00:24:51.072798+00:00", "date_published": "2025-12-19T18:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/year-in-review-2025/", "id": "urn:aggr:item:f0531616b3b06b794acc6bf2ebb4932763f1b49e", "summary": "2025 Year in Review of LLM paradigm changes", "tags": [ "programming" ], "title": "2025 LLM Year in Review", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-12-19-2025-llm-year-in-review/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "Following up on digital hygiene, I wanted to write up my (evolving, opinionated) guide to chemical hygiene. I keep ranting about this topic to all of my friends recently (you can tell I'm really fun at parties), so I thought it would be worth writing it up to have it all in one place/url:
\nWater
\nStarting out with controlling your water system, which is the easiest in terms of concrete, high confidence recommendations that in my experience still only <5% of my friends have adopted:
\nWater is the easiest section in this entire article because it has well-understood ways to spend $/risk reduction compared to a lot more complex categories we'll see later (food especially). I would recommend contacting a company in your local area to install both a whole home filter and an under-the-sink reverse osmosis system, to handle the ~yearly maintenance (filter changes), and conduct tests to demonstrate the improvement.
\nAir
\nSimilar to water, air is relatively well-understood and simple to control in your home:
\nFood
\nFood is the hardest category to control because it involves extensively deep supply chains that have been ruthlessly efficiency-maxxed over the last few decades with little to no regard for public health externalities. The industry has a clear and immediate financial incentive to trade something 10% cheaper at the cost of something 10X more harmful to you as long as it shows up over a long enough time period that the accounting is impractical. And it just turns out that in food there are many, many ways to cut corners. Sadly, the US Government has been woefully inadequate in constraining the industry and lags far behind other countries (e.g. Europe especially), hence the recent MAHA efforts. I'll split this section into 1) food sourcing and 2) cooking/preparation.
\nFood: 1) sourcing
\n![\"GvWK5b1X0AEevjd\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/gvwk5b1x0aeevjd.webp\")
Food: 2) Cooking & preparation
\nFabrics
\nOur bodies come into frequent contact with all kinds of fabrics (clothing, bedding, furniture, rugs, mats, ...). As you handle these materials, they shed particles, which you end up breathing in.
\nCleaning supplies: soap, dish washing, laundry, toilet, spray cleaners
\nDental hygiene
\nThis is a category that I was not able to make a dent into in my personal life, despite a number of attempts. The goal with all of this is go after the 80:20 low hanging fruit and this category for me falls into the latter category:
\nSunscreen
\nWellness
\nLearn more
\nTLDR. Keep your home unsophisticated. Filter your water and air. Eat real food (not edible food-like substances) from well-treated animals and with few, sensible ingredients and minimally sophisticated supply chains and processing steps. Say no to as many dyes and fragrances as you can. Surround yourself with simple, natural materials or strong and inert materials (e.g. stainless steel). Avoid plastics, especially if they are handled, heated, frozen - the risk is not just related to the tiny particles of these exotic materials accumulating all over your body and interfering with its chemistry, but the large zoo of chemical plasticizers that are added to plastics and then leech out. The government is significantly lagging behind the industry on chemical regulation and this is your responsibility.
\nThis guide isn't perfect. It's a work in progress. I am not a professional toxicologist or food scientist so my tone above is my frustration that the government is forcing me to be a part-time investigative journalist just to exist in a modern society and not feel like I am poisoning myself and my family. And I didn't even go into and cover all of the environmental aspects of these industries. This state of affairs is much worse here in the US than e.g. in Europe - the EU bans or restricts many food additives, dyes, chemicals and food processing practices that are routine here. The FDA "Generally Recognized As Safe" (GRAS) system lets manufacturers self-certify ingredients without independent review and a new exotic chemical or process is innocent until proven guilty, while in Europe the default is often the reverse. So treat all of this as a starting point, ask your favorite LLM for more information on any of the items, let me know your thoughts (e.g. X/Instagram DMs) and I will aim to update this guide over time.
\n", "date_modified": "2025-12-22T18:17:22.643380+00:00", "date_published": "2025-12-18T18:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/chemical-hygiene/", "id": "urn:aggr:item:5fc51c3a66bbea821b13ebc7037283dee79a74b7", "summary": "An evolving guide of protecting your health from a pricemaxxing industry.", "tags": [ "programming" ], "title": "Chemical hygiene", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-12-18-chemical-hygiene/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "![\"hnhero\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/hnhero.webp\")
TLDR: https://karpathy.ai/hncapsule/
\nYesterday I stumbled on this HN thread Show HN: Gemini Pro 3 hallucinates the HN front page 10 years from now, where Gemini 3 was hallucinating the frontpage of 10 years from now. One of the comments struck me a bit more though - Bjartr linked to the HN frontpage from exactly 10 years ago, i.e. December 2015. I was reading through the discussions of 10 years ago and mentally grading them for prescience when I realized that an LLM might actually be a lot better at this task. I copy pasted one of the article+comment threads manually into ChatGPT 5.1 Thinking and it gave me a beautiful analysis of what people thought + what actually happened in retrospect, even better and significantly more detailed than what I was doing manually. I realized that this task is actually a really good fit for LLMs and I was looking for excuses to vibe code something with the newly released Opus 4.5, so I got to work. I'm going to get all the front pages of December (31 days, 30 articles per day), get ChatGPT 5.1 Thinking to do the analysis, and present everything in a nice way for historical reading.
\nThere are two macro reasons for why I think the exercise is interesting more generally:
\nVibe coding the actual project was relatively painless and took about 3 hours with Opus 4.5, with a few hickups but overall very impressive. The repository is on GitHub here: karpathy/hn-time-capsule. Here is the progression of what the code does:
\nThe following is an article that appeared on Hacker News 10 years ago, and the discussion thread.\n\nLet's use our benefit of hindsight now in 6 sections:\n\n1. Give a brief summary of the article and the discussion thread.\n2. What ended up happening to this topic? (research the topic briefly and write a summary)\n3. Give out awards for "Most prescient" and "Most wrong" comments, considering what happened.\n4. Mention any other fun or notable aspects of the article or discussion.\n5. Give out grades to specific people for their comments, considering what happened.\n6. At the end, give a final score (from 0-10) for how interesting this article and its retrospect analysis was.\n\nAs for the format of Section 5, use the header "Final grades" and follow it with simply an unordered list of people and their grades in the format of "name: grade (optional comment)". Here is an example:\n\nFinal grades\n- speckx: A+ (excellent predictions on ...)\n- tosh: A (correctly predicted this or that ...)\n- keepamovin: A\n- bgwalter: D\n- fsflover: F (completely wrong on ...)\n\nYour list may contain more people of course than just this toy example. Please follow the format exactly because I will be parsing it programmatically. The idea is that I will accumulate the grades for each account to identify the accounts that were over long periods of time the most prescient or the most wrong.\n\nAs for the format of Section 6, use the prefix "Article hindsight analysis interestingness score:" and then the score (0-10) as a number. Give high scores to articles/discussions that are prominent, notable, or interesting in retrospect. Give low scores in cases where few predictions are made, or the topic is very niche or obscure, or the discussion is not very interesting in retrospect.\n\nHere is an example:\nArticle hindsight analysis interestingness score: 8\n---\ndata directory if someone else would like to play. It's the file data.zip under the exact same url prefix (intentionally avoiding a direct link).I spent a few hours browsing around and found it to be very interesting. A few example threads just for fun:
\nAnd then when you navigate over to the Hall of Fame, you can find the top commenters of Hacker News in December 2015, sorted by imdb-style score of their grade point average. In particular, congratulations to pcwalton, tptacek, paulmd, cstross, greglindahl, moxie, hannob, 0xcde4c3db, Manishearth, johncolanduoni - GPT 5.1 Thinking found your comments very insightful and prescient. You can also scroll all the way down to find the noise of HN, which I think we're all familiar with too :)
\nMy code (wait, Opus' code?) on GitHub can be used to reproduce or tweak the results. Running 31 days of 30 articles through GPT 5.1 Thinking meant 31 * 30 = 930 LLM queries and cost about $58 and somewhere around ~1 hour. The LLM megaminds of the future might find this kind of a thing a lot easier, a lot faster and a lot cheaper.
The space of intelligences is large and animal intelligence (the only kind we've ever known) is only a single point (or a little cloud), arising from a very specific kind of optimization that is fundamentally distinct from that of our technology.
\n
Animal intelligence optimization pressure:
\nMeanwhile, LLM intelligence optimization pressure:
\nThe computational substrate is different (transformers vs. brain tissue and nuclei), the learning algorithms are different (SGD vs. ???), the present-day implementation is very different (continuously learning embodied self vs. an LLM with a knowledge cutoff that boots up from fixed weights, processes tokens and then dies). But most importantly (because it dictates asymptotics), the optimization pressure / objective is different. LLMs are shaped a lot less by biological evolution and a lot more by commercial evolution. It's a lot less survival of tribe in the jungle and a lot more solve the problem / get the upvote. LLMs are humanity's "first contact" with non-animal intelligence. Except it's muddled and confusing because they are still rooted within it by reflexively digesting human artifacts, which is why I attempted to give it a different name earlier (ghosts/spirits or whatever). People who build good internal models of this new intelligent entity will be better equipped to reason about it today and predict features of it in the future. People who don't will be stuck thinking about it incorrectly like an animal.
\n", "date_modified": "2025-11-29T21:40:09.651972+00:00", "date_published": "2025-11-29T18:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/the-space-of-minds/", "id": "urn:aggr:item:cff534376e1d9e777cff57aedefe8fe6155a2eeb", "summary": "On the space of minds and the optimizations that give rise to them.", "tags": [ "programming" ], "title": "The space of minds", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-11-29-the-space-of-minds/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "AI has been compared to various historical precedents: electricity, industrial revolution, etc., I think the strongest analogy is that of AI as a new computing paradigm because both are fundamentally about the automation of digital information processing.
\nIf you were to forecast the impact of computing on the job market in ~1980s, the most predictive feature of a task/job you'd look at is specifiability, i.e. are you just mechanically transforming information according to rote, easy to specify algorithm (examples being typing, bookkeeping, human calculators, etc.)? Back then, this was the class of programs that the computing capability of that era allowed us to write (by hand, manually). I call hand-written programs "Software 1.0".
\nWith AI now, we are able to write new programs that we could never hope to write by hand before. We do it by specifying objectives (e.g. classification accuracy, reward functions), and we search the program space via gradient descent to find neural networks that work well against that objective. This is my Software 2.0 blog post from a while ago. In this new programming paradigm then, the new most predictive feature to look at is verifiability. If a task/job is verifiable, then it is optimizable directly or via reinforcement learning, and a neural net can be trained to work extremely well. It's about to what extent an AI can "practice" something. The environment has to be:
\nThe more a task/job is verifiable, the more amenable it is to automation in the new programming paradigm. If it is not verifiable, it has to fall out from neural net magic of generalization fingers crossed, or via weaker means like imitation. This is what's driving the "jagged" frontier of progress in LLMs. Tasks that are verifiable progress rapidly, including possibly beyond the ability of top experts (e.g. math, code, amount of time spent watching videos, anything that looks like puzzles with correct answers), while many others lag by comparison (creative, strategic, tasks that combine real-world knowledge, state, context and common sense).
\nFinally had a chance to listen through this Dwarkesh pod with Sutton, which was interesting and amusing.
\nAs background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed, just look at LLM scaling laws where if you put compute on the x-axis, number go up and to the right. So it's amusing to see that Sutton, the author of the post, is not so sure that LLMs are "bitter lesson pilled" at all. They are trained on giant datasets of fundamentally human data, which is both 1) human generated and 2) finite. What do you do when you run out? How do you prevent a human bias? So there you have it, bitter lesson pilled LLM researchers taken down by the author of the bitter lesson - rough!
\nIn some sense, Dwarkesh (who represents the LLM researchers viewpoint in the pod) and Sutton are slightly speaking past each other because Sutton has a very different architecture in mind and LLMs break a lot of its principles. He calls himself a "classicist" and evokes the original concept of Alan Turing of building a "child machine" - a system capable of learning through experience by dynamically interacting with the world. There's no giant pretraining stage of imitating internet webpages. There's also no supervised finetuning, which he points out is absent in the animal kingdom (it's a subtle point but Sutton is right in the strong sense: animals may of course observe demonstrations, but their actions are not directly forced/"teleoperated" by other animals). Another important note he makes is that even if you just treat pretraining as an initialization of a prior before you finetune with reinforcement learning, Sutton sees the approach as tainted with human bias and fundamentally off course, a bit like when AlphaZero (which has never seen human games of Go) beats AlphaGo (which initializes from them). In Sutton's world view, all there is is an interaction with a world via reinforcement learning, where the reward functions are partially environment specific, but also intrinsically motivated, e.g. "fun", "curiosity", and related to the quality of the prediction in your world model. And the agent is always learning at test time by default, it's not trained once and then deployed thereafter. Overall, Sutton is a lot more interested in what we have common with the animal kingdom instead of what differentiates us. "If we understood a squirrel, we'd be almost done".
\nAs for my take...
\nFirst, I should say that I think Sutton was a great guest for the pod and I like that the AI field maintains entropy of thought and that not everyone is exploiting the next local iteration LLMs. AI has gone through too many discrete transitions of the dominant approach to lose that. And I also think that his criticism of LLMs as not bitter lesson pilled is not inadequate. Frontier LLMs are now highly complex artifacts with a lot of humanness involved at all the stages - the foundation (the pretraining data) is all human text, the finetuning data is human and curated, the reinforcement learning environment mixture is tuned by human engineers. We do not in fact have an actual, single, clean, actually bitter lesson pilled, "turn the crank" algorithm that you could unleash upon the world and see it learn automatically from experience alone.
\nDoes such an algorithm even exist? Finding it would of course be a huge AI breakthrough. Two "example proofs" are commonly offered to argue that such a thing is possible. The first example is the success of AlphaZero learning to play Go completely from scratch with no human supervision whatsoever. But the game of Go is clearly such a simple, closed, environment that it's difficult to see the analogous formulation in the messiness of reality. I love Go, but algorithmically and categorically, it is essentially a harder version of tic tac toe. The second example is that of animals, like squirrels. And here, personally, I am also quite hesitant whether it's appropriate because animals arise by a very different computational process and via different constraints than what we have practically available to us in the industry. Animal brains are nowhere near the blank slate they appear to be at birth. First, a lot of what is commonly attributed to "learning" is imo a lot more "maturation". And second, even that which clearly is "learning" and not maturation is a lot more "finetuning" on top of something clearly powerful and preexisting. Example. A baby zebra is born and within a few dozen minutes it can run around the savannah and follow its mother. This is a highly complex sensory-motor task and there is no way in my mind that this is achieved from scratch, tabula rasa. The brains of animals and the billions of parameters within have a powerful initialization encoded in the ATCGs of their DNA, trained via the "outer loop" optimization in the course of evolution. If the baby zebra spasmed its muscles around at random as a reinforcement learning policy would have you do at initialization, it wouldn't get very far at all. Similarly, our AIs now also have neural networks with billions of parameters. These parameters need their own rich, high information density supervision signal. We are not going to re-run evolution. But we do have mountains of internet documents. Yes it is basically supervised learning that is ~absent in the animal kingdom. But it is a way to practically gather enough soft constraints over billions of parameters, to try to get to a point where you're not starting from scratch. TLDR: Pretraining is our crappy evolution. It is one candidate solution to the cold start problem, to be followed later by finetuning on tasks that look more correct, e.g. within the reinforcement learning framework, as state of the art frontier LLM labs now do pervasively.
\nI still think it is worth to be inspired by animals. I think there are multiple powerful ideas that LLM agents are algorithmically missing that can still be adapted from animal intelligence. And I still think the bitter lesson is correct, but I see it more as something platonic to pursue, not necessarily to reach, in our real world and practically speaking. And I say both of these with double digit percent uncertainty and cheer the work of those who disagree, especially those a lot more ambitious bitter lesson wise.
\nSo that brings us to where we are. Stated plainly, today's frontier LLM research is not about building animals. It is about summoning ghosts. You can think of ghosts as a fundamentally different kind of point in the space of possible intelligences. They are muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity's documents with some sprinkle on top. They are not platonically bitter lesson pilled, but they are perhaps "practically" bitter lesson pilled, at least compared to a lot of what came before. It seems possibly to me that over time, we can further finetune our ghosts more and more in the direction of animals; That it's not so much a fundamental incompatibility but a matter of initialization in the intelligence space. But it's also quite possible that they diverge even further and end up permanently different, un-animal-like, but still incredibly helpful and properly world-altering. It's possible that ghosts:animals :: planes:birds.
\nAnyway, in summary, overall and actionably, I think this pod is solid "real talk" from Sutton to the frontier LLM researchers, who might be gear shifted a little too much in the exploit mode. Probably we are still not sufficiently bitter lesson pilled and there is a very good chance of more powerful ideas and paradigms, other than exhaustive benchbuilding and benchmaxxing. And animals might be a good source of inspiration. Intrinsic motivation, fun, curiosity, empowerment, multi-agent self-play, culture. Use your imagination.
\nVery often, I sit down at a restaurant, look through their menu, and feel... kind of stuck. What is Pâté again? What is a Tagine? Cavatappi... that's a pasta right? Sweetbread sounds delicious (I have a huge sweet tooth). It can get really out of hand sometimes. "Confit tubers folded with matured curd and finished with a beurre noisette infusion." okay so... what is this exactly? I've spent so much of my life googling pictures of foods that when the time came to attend a recent vibe coding hackathon, I knew it was the perfect opportunity to finally build the app I always wanted, but could nowhere find. And here it is in flesh, I call it... 🥁🥁🥁 ... MenuGen:
\n![\"Screenshot](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/02pm.webp\")
MenuGen is super simple. You take a picture of a menu and it generates images for all the menu items. It visualizes the menu. Obviously it's not exactly what you will be served in that specific restaurant, but it gives you the basic idea: Some of these dishes are salads, this is a fish, this is a soup, etc. I found it so helpful in my personal use that after the hackathon (where I got the first version to work on localhost) I continued vibe coding a bit to deploy it, add authentication, payments, and generally make it real. So here it is, give it a shot the next time you go out :): menugen.app!
\nMenuGen is my first end-to-end vibe coded app, where I (someone who tinkers but has little to no actual web development experience) went from scratch all the way to a real product that people can sign up for, pay for, get utility out of, and where I pocket some good and honest 10% markup. It's pretty cool. But in addition to the utility of the app, MenuGen was interesting to me as an exploration of vibe coding apps and how feasible it is today. As such, I did not write any code directly; 100% of the code was written by Cursor+Claude and I basically don't really know how MenuGen works in the conventional sense that I am used to. So now that the project is "done" (as in the first version seems to work), I wanted to write up this quick post on my experience - what it looks like today for a non-webdev to vibe code a web app.
\nFirst, local version. In what is a relatively common experience in vibe coding, the very first prototype of the app running on my local machine took very little time. I took Cursor + Claude 3.7, I gave it the description of the app, and it wrote all the React frontend components very quickly, laying out a beautiful web page with smooth, multicolored fonts, little CSS animations, responsive design and all that, except for the actual backend functionality. Seeing a new website materialize so quickly is a strong hook. I felt like I was 80% done but (foreshadowing...) it was a bit closer to 20%.
\nOpenAI API. Around here is where some of the troubles started. I needed to call OpenAI APIs to OCR the menu items from the image. I had to get the OpenAI API keys. I had to navigate slightly convoluted menus asking me about "projects" and detailed permissions. Claude kept hallucinating deprecated APIs, model names, and input/output conventions that have all changed recently, which was confusing, but it resolved them after I copy pasted the docs back and forth for a while. Once the individual API calls were working, I immediately ran into some heavy rate limiting of the API calls, allowing me to only issue a few queries every 10 minutes.
\nReplicate API. Next, I needed to generate images given the descriptions. I signed up for a new Replicate API key and ran into similar issues relatively quickly. My queries didn't work because LLM knowledge was deprecated, but in addition, this time even the official docs were a little bit out of date due to recent changes in the API, which now don't return the JSON directly but instead some kind of a Streaming object that neither I or Claude understood. I then faced rate limiting on the API so it was difficult to debug the app. I was told later that these are common protection measures by these services to mitigate fraud, but they also make it harder to get started with new, legitimate accounts. I'm told Replicate is moving to a different approach where you pre-purchase credits, which might help going forward.
\nVercel deploy. At this point at least, the app was working locally so I was quite happy. It was time to deploy the basic first version. Sign up for Vercel, add project, configure it, point it at my GitHub repo, push to master, watch a new Deployment build and... ERROR. The logs showed some linting errors due to unused variables and other basic things like that, but it was hard to understand or debug because everything worked fine on local and only broke on Vercel build, so I debugged the issues by pushing fake debugging commits to master to force redeploys. Once I fixed these issues, the site still refused to work. I asked Claude. I asked ChatGPT. I consulted docs. I googled around. 1 hour later I finally realized my silly mistake - My .env.local file stored the API keys to OpenAI and Replicate, but this file is (correctly!) part of .gitignore and doesn't get pushed to git, so you have to manually navigate to Vercel project settings, find the right place, and add your environment keys manually. I kind of understood the issue relatively quickly, but I could see an aspiring vibe coder get stuck on this for a while. Once the deployment finally succeeded, Vercel happily offered a URL. This surprised me again because my project was a private git repo that was not ready to see the light of day. I didn't realize that Vercel will take your !private! repo of an unfinished project and auto-deploy it on a totally public and easy to guess url just like that, hah.
Clerk authentication. Claude suggested that we use Clerk for authentication, so I went along with it. Signed up for Clerk, configured the project, got my API keys. At this point Claude hallucinated about 1000 lines of code that appeared to be deprecated Clerk APIs. I had to copy paste a lot of the docs back and forth to get things gradually unstuck. Next, so far, Clerk was running in a "Development" deployment. To move to a "Production" deployment, there were more hoops to jump through. Clerk demands that you host your app on a custom domain that you own. menugen.vercel.com will not work. So I had to purchase the domain name menugen.app. Then I had to wire the domain to my Vercel project. Then I had to change the DNS records. Then I had to pick an OAuth provider, e.g. I went with Google. But to do that was its own configuration adventure . I had to enable an "SSO connection". I had to go over to Google Cloud Console and create a new project, and add a new OAuth Credential. I had to wait some time for an approval process around here. I then had to go back and forth between the nested settings of all of Vercel, Clerk and Google for a while to wire it up properly. I thought of quitting the project around here, but I felt better when I woke up the next morning.
Stripe payments. Next I wanted to add payments so that people can purchase credits. This means another website, another account, more docs, more keys. I select "Next.js" as the backend, copy paste the very first snippet of code from the "getting started" docs into my app and... ERROR. I realized later that Stripe gives you JavaScript code when you select Next.js, but my app is built in TypeScript, so every time I pasted a snippet of code it made Cursor unhappy with linter errors, but Claude patched things up ok over time after I told it to "fix errors" a few times and after I threatened to switch to ChatGPT. Then back in the Stripe dashboard we create a Product, we create a Price, we find the price key (not the product key!), copy paste all the keys around. Around here, I caught Claude using a really bad idea approach to match up a successful Stripe payment to user credits (it tried to match up the email addresses, but the email the user might give in the Stripe checkout may not be the email of the Google account they signed up with, so the user might not actually get the credits that they purchased). I point this out to Claude and it immediately apologizes and rewrites it correctly by passing around unique user ids in the request metadata. It thanks me for pointing out the issue and tells me that it will do it correctly in the future, which I know is just gaslighting. But since our quick test works, only a few more clicks to upgrade the deployment from Development to Production, now re-do a new Product, redo a new Price, re-copy paste all the keys and ids, locally and in the Vercel settings... and then it worked :)
\nDatabase? Work queues? So far, all of the processing is done "in the moment" - it's just requests and results right there and then, nothing is cached, saved, or etc. So the results are ephemeral and if the response takes too long (e.g. because the menu is too long and has too many items, or because the APIs show too much latency), the request can time out and break. If you refresh the page, everything is gone too. The correct way to do this is to have a database where we register and keep track of work, and the client just displays the latest state as it's ready. I realized I'd have to connect a database from the Marketplace, something like Supabase PostgreSQL (even when Claude pitched me on using Vercel KV, which I know is actually deprecated). And then we'd also need some queue service like Upstash or so to run the actual processing. It would mean more services. More logins. More API keys. More configurations. More docs. More suffering. It was too much bear. Leave as future work.
\nTLDR. Vibe coding menugen was exhilarating and fun escapade as a local demo, but a bit of a painful slog as a deployed, real app. Building a modern app is a bit like assembling IKEA future. There are all these services, docs, API keys, configurations, dev/prod deployments, team and security features, rate limits, pricing tiers... Meanwhile the LLMs have slightly outdated knowledge of everything, they make subtle but critical design mistakes when you watch them closely, and sometimes they hallucinate or gaslight you about solutions. But the most interesting part to me was that I didn't even spend all that much work in the code editor itself. I spent most of it in the browser, moving between tabs and settings and configuring and gluing a monster. All of this work and state is not even accessible or manipulatable by an LLM - how are we supposed to be automating society by 2027 like this?
\nGoing forward. As an exploration of what it's like to vibe code an app today if you have little to no web dev background, I'm left with an equal mix of amazement (it's actually possible and much easier/faster than what was possible before!) and a bit of frustration of what could be. Part of the pain of course is that none of this infrastructure was really designed to be used like this. The intended target audience are teams of professional web developers living in a pre-LLM world. Not vibe coding solo devs prototyping apps. Some thoughts on solutions that could make super simple apps like MenuGen a lot easier to create:
\nFor now, I'm pretty happy to have vibe coded my first super custom app through the finish line of something that is real, solves a need I've had for a long time, and is shareable with friends. Thank you to all the services above that I've used to build it. In principle, it could earn some $ if others like it too, in a completely passive way - the @levelsio dream. Ultimately, vibe coding full web apps today is kind of messy and not a good idea for anything of actual importance. But there are clear hints of greatness and I think the industry just needs a bit of time to adapt to the new world of LLMs. I'm personally quite excited to see the barrier to app drop to ~zero, where anyone could build and publish an app just as easily as they can make a TikTok. These kinds of hyper-custom automations could become a beautiful new canvas for human creativity.
\nThe companion tweet (and the "comments section") is on my X @karpathy.
\n", "date_modified": "2025-05-01T15:30:51.441806+00:00", "date_published": "2025-04-27T12:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/vibe-coding-menugen/", "id": "urn:aggr:item:97d4e982abf72956c437e7cf57fb5222b81edfa9", "summary": "Work log of vibe coding menugen app", "tags": [ "programming" ], "title": "Vibe coding MenuGen", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-04-27-vibe-coding-menugen/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "![\"a-beautifully-illustrated-watercolor-ima_TcCW0iyURQi-raDmaFRZLw_GIGBjDOWQ6CN6Qf_yUU4tA](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/a-beautifully-illustrated-watercolor-ima_tccw0iyurqi-radmafrzlw_gigbjdowq6cn6qf_yuu4ta-1.webp\")
Transformative technologies usually follow a top-down diffusion path: originating in government or military contexts, passing through corporations, and eventually reaching individuals - think electricity, cryptography, computers, flight, the internet, or GPS. This progression feels intuitive, new and powerful technologies are usually scarce, capital-intensive, and their use requires specialized technical expertise in the early stages.
\nSo it strikes me as quite unique and remarkable that LLMs display a dramatic reversal of this pattern - they generate disproportionate benefit for regular people, while their impact is a lot more muted and lagging in corporations and governments. ChatGPT is the fastest growing consumer application in history, with 400 million weekly active users who use it for writing, coding, translation, tutoring, summarization, deep research, brainstorming, etc. This isn't a minor upgrade to what existed before, it is a major multiplier to an individual's power level across a broad range of capabilities. And the barrier to use is incredibly low - the models are cheap (free, even), fast, available to anyone on demand behind a url (or even local machine), and they speak anyone's native language, including tone, slang or emoji. This is insane. As far as I can tell, the average person has never experienced a technological unlock this dramatic, this fast.
\nWhy then are the benefits a lot more muted in the corporate and government realms? I think the first reason is that LLMs offer a very specific profile of capability - that of merely quasi-expert knowledge/performance, but simultaneously across a very wide variety of domains. In other words, they are simultaneously versatile but also shallow and fallible. Meanwhile, an organization's unique superpower is the ability to concentrate diverse expertise into a single entity by employing engineers, researchers, analysts, lawyers, marketers, etc. While LLMs can certainly make these experts more efficient individually (e.g. drafting initial legal clauses, generating boilerplate code, etc.), the improvement to the organization takes the form of becoming a bit better at the things it could already do. In contrast, an individual will usually only be an expert in at most one thing, so the broad quasi-expertise offered by the LLM fundamentally allows them to do things they couldn't do before. People can now vibe code apps. They can approach legal documents. They can grok esoteric research papers. They can do data analytics. They can generate multimodal content for branding and marketing. They can do all of this at an adequate capability without involving an additional expert.
\n![\"download2\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/download2.webp\")
Second, organizations deal with problems of a lot greater complexity and necessary coordination, think: various integrations, legacy systems, corporate brand or style guides, stringent security protocols, privacy considerations, internationalization, regulatory compliance and legal risk. There are a lot more variables, a lot more constraints, a lot more considerations, and a lot lower margin for error. It's not so easy to put all of it into a context window. You can't just vibe code something. You might be one disastrous hallucination away from losing your job. And third, there is the well-documented inertia of a larger organization, featuring culture, historical precedents, political turf wars that escalate in periods of rapid change, communication overhead, re-training challenges of a distributed workforce and good old-fashioned bureaucracy. These are major headwinds when it comes to rapid adoption of a sparkling new, versatile-but-shallow-and-fallible tool. I don't wish to downplay the impacts of LLMs in corporations or governments, but at least for the moment and in aggregate across society, they have been significantly more life altering for individuals than they have been for organizations. Mary, Jim and Joes are experiencing the majority of the benefit, not Google or the government of the United States.
\nLooking forward, the continued diffusion of LLMs of course depends on continued performance improvement and its capability profile. The "benefit distribution" overall is particularly interesting to chart, and depends heavily on the dynamic range of the performance as a function of capital expenditure. Today, frontier-grade LLM performance is very accessible and cheap. Beyond this point, you cannot spend a marginal dollar to get better performance, reliability or autonomy. Money can't buy better ChatGPT. Bill Gates talks to GPT 4o just like you do. But can this be expected to last? Train-time scaling (increase parameters, data), test-time scaling (increase time) and model ensembles (increase batch) are forces increasing the dynamic range. On the other hand, model distillation (the ability to train disproportionately powerful small models by training to mimic the big model) has been a force decreasing dynamic range. Certainly, the moment money can buy dramatically better ChatGPT, things change. Large organizations get to concentrate their vast resources to buy more intelligence. And within the category of "individual" too, the elite may once again split away from the rest of society. Their child will be tutored by GPT-8-pro-max-high, yours by GPT-6 mini.
\nBut at least at this moment in time, we find ourselves in a unique and unprecedented situation in the history of technology. If you go back through various sci-fi you'll see that very few would have predicted that the AI revolution would feature this progression. It was supposed to be a top secret government megabrain project wielded by the generals, not ChatGPT appearing basically overnight and for free on a device already in everyone's pocket. Remember that William Gibson quote "The future is already here, it's just not evenly distributed"? Surprise - the future is already here, and it is shockingly distributed. Power to the people. Personally, I love it.
\nA version of this post that allows community comments is here on X.
\n", "date_modified": "2025-04-08T08:21:28.839125+00:00", "date_published": "2025-04-07T18:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/power-to-the-people/", "id": "urn:aggr:item:6b3970aef2046c8d70640508826fdc51110b782e", "summary": "Yes", "tags": [ "programming" ], "title": "Power to the people: How LLMs flip the script on technology diffusion", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-04-07-power-to-the-people-how-llms-flip-the-script-on-technology/" }, { "authors": [ { "name": "karpathy" } ], "content_html": "About 2 months ago I stumbled by this Bryan Johnson video on How I FIXED My Terrible Sleep - 10 Habits. I resolved that day to listen to Bryan and try to improve my sleep. But before we can improve it, first - how should we measure it? Bryan Johnson seems to use Whoop, but at that time I only had my Apple Watch (coupled with one of the popular sleep apps - AutoSleep). And then a long time ago I used and liked Oura. And I also had an order in for the new and fancy 8Sleep Pod 4 Ultra, which I was aware offers some sleep tracking too. So I found myself in a bit of a pickle - which one should I pick to track my sleep? And the answer of course is... to initiate a comprehensive tracking project to compare the 4 major candidates and find the. best. sleep. tracker. So that's what I did. This is me fully geared up and ready for bed:
\n![\"sleep_edit\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/sleep_edit.webp\")
I've now gathered roughly 2 months of data. I kept the raw data in a simple spreadsheet, recording some of the basic measurements: the amount of sleep (Light, REM, Deep, and Awake tossing and turning), heart rate measurements (Resting Heart Rate (RHR), Heart Rate Variability (HRV)), and the sleep Score offered by each app. I'd log these every day right when I wake up so that I can compare and contrast the numbers and relate them to how I felt that morning. You can find my raw data in this spreadsheet, it looks like this:
\n![\"sleep_data\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/sleep_data.webp\")
Qualitative assessment. Now, to spare you some suspense, after 2 months of data collection and staring at the results basically every morning, it was very pretty easy to guess that Oura and Whoop are both "Tier 1" - fairly similar and quite high quality in their sleep tracking. They both give similar scores that also correlated with the way I felt in the morning most of the time. Next is 8Sleep, which is ok. And finally, I was sad to learn that Apple Watch + AutoSleep (which I had used in the past for many months) was really, really terrible. Its scores are basically almost random and they swing around wildly, with little correlation to how I felt in the morning in comparison.
\nLet's now look at some of the data. First, let's look at the values that all 4 signals take on over the 2 months, with their histograms:
\n![\"signals\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/signals.webp\")
As we can see, AutoSleep and 8Sleep are way too easy to please, giving out really high scores and pushing against the 100 score boundary. Whoop is also a little too easy to please, giving out 100 scores. Oura is the most difficult to please, shows a relatively nice gaussian distribution of scores, and offering the most dynamic range. I take this to be a good and nice property of Oura. Indeed, after 2 months my highest ever score on Oura was 92, while I can get 100 on Whoop fairly regularly. This means that I can keep going and striving for even more optimal sleep, one day.
\nNext, I was very curious about the correlation analysis between the trackers. We take all the scores and plot pairwise correlation scatter plots to see which of the trackers "agree the most" with each other. Here it is:
\n![\"corr\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/corr.webp\")
And here are the correlations sorted:
\nWhoop vs Oura: 0.65\nOura vs 8Sleep: 0.59\nOura vs AutoSleep: 0.47\n8Sleep vs AutoSleep: 0.42\nWhoop vs 8Sleep: 0.38\nWhoop vs AutoSleep: 0.14\nWhoop and Oura seem to enjoy the highest correlation at ~0.65, while the other trackers are a bit all over the place. In particular, Whoop and AutoSleep are almost uncorrelated (0.14!). If we think that Whoop is good (which I think it is), AutoSleep looks almost like a noise generator.
\nMatters of Heart Rate. Next, I was interested to look at the Resting Heart Rate (RHR) and Heart Rate Variability (HRV). First, all trackers except 8Sleep agree quite highly on the heart rate during the night, including the Apple Watch. 8 Sleep is the worst because... it's a mattress so it doesn't have a direct measurement of the heart rate. I'm actually a bit impressed that it has a correlation this high:
\n AutoSleep 8Sleep Oura Whoop\nAutoSleep 1.000000 0.947151 0.908987 0.942587\n8Sleep 0.947151 1.000000 0.947977 0.878552\nOura 0.908987 0.947977 1.000000 0.904023\nWhoop 0.942587 0.878552 0.904023 1.000000\nHaving established that all 3 devices (Oura, Whoop, AutoSleep) give a good and consistent measurement of resting heart rate during the night, I was curious if there is a correlation with the sleep score, as this is something Bryan mentioned a few times in his videos. In other words, is a lower RHR associated with better sleep score? Keep in mind that this is just correlation analysis, indeed, I have no idea if the apps take RHR as one of the measurements when they calculate the sleep score. For Whoop, it seems like there is a tiny bit of a correlation, i.e. lower RHR comes with higher sleep score (~0.13).
\n![\"whoopcorr\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/whoopcorr.webp\")
But for Oura there is none:
\n![\"ouracorr\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/ouracorr.webp\")
So... I'm not sure what to make of this. Going in, I thought that lower RHR would correlate quite well to better score but this doesn't seem to be the case.
\nLastly, during the 2 months of data collection I was exercising regularly, getting about 30 minutes on average of Zone 2 cardio every day, except twice a week also doing a 4x4x4 HIIT (4 min off, 4 min on, 4 times). I was curious if this showed up and indeed it seems like it does, pretty cool:
\n![\"improvement\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/improvement.webp\")
Using Whoop-Oura average measurement of both RHR and HRV, my resting heart rate has improved (decreased) by a bit less than 3 bpm over the duration of these 60 days (from ~51 bpm -> 48 ~bpm), which is awesome. In addition, my HRV has also improved (increased), (from ~49 -> 54). I love to see exercise adaptations in the data. For some unknown reason, notice also that the HRV values from Whoop seem to be inflated above those of Oura by about 5. I'm not exactly sure why, possibly they calculate it differently... but it's a bit surprising and unexplained.
\nLastly, over the duration of 2 months I tried to improve my sleep quality, but it's all mixed up with a bunch of random events, parties, injuries, and also random experiments I tried to run here and there. As another example, my last week was rough because I was obsessed with a technical problem and couldn't sleep well. So unfortunately, overall, I am not seeing a dramatic increase in my sleep quality just yet. But I see this as a long-term project and I hope to increase these scores on average over the duration of the year. Maybe if squint hard enough my sleep has improved a tiny amount (?), but let's face it this is cope hah:
\n![\"sleepovertime\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/sleepovertime.webp\")
Yes, sleep matters. Overall, I will say with absolute certainty that Bryan is basically right, and my sleep scores correlate strongly with the quality of work I am able to do that day. When my score is low, I lack agency, I lack courage, I lack creativity, I'm simply tired. When my sleep score is high, I can power through anything. On my best days, I can sit down and work through 14 hours and barely notice the passage of time. It's not subtle. The effects are not a function of a single day's sleep but of the accumulated sleep debt over a duration of last few days. So in other words a single bad night is usually ok. But a few in a row is bad news. And vice versa. Listen to Bryan.
\nShopping recommendations. Finally, I wanted to close with some recommendations to others who might want to undertake sleep tracking and improve their sleep.
\nOura is Tier 1 / super solid tracker. The app is excellent and I love the single "overview pane" with all the data about that sleep (Whoop needs a lot more clicking around the app). I love that Oura score doesn't saturate that easily, that its scores are a gaussian, and that it has dynamic range. Unfortunately, I find the ring form factor quite inconvenient because it's a little thick, and fingers are used extensively (e.g. hand washing, food preparation, etc.) When I go to the gym, I find myself removing the ring often because it interferes with my grip strength, and it could get scratched. The ring has to be sized correctly and your finger changes its size. Sometimes it's a little too snug, sometimes a little too loose. The ring also has to be rotated correctly for the best results (the notch has to be down), so you'll keep finding it rotated wrong and correcting it. I also don't love having to take the ring on and off to charge it.
\nWhoop is also a Tier 1 / super solid tracker. The app is excellent. It can be a bit overwhelming at first and requires quite a bit of moving around, but it is very comprehensive, full-featured and customizable, more than Oura. It also has a pretty neat and useful LLM integration. I also really like the Community feature, though it is severely undercooked, under-designed, and feels orphaned. I think Oura has a better "grand overview" page for a single dense summary of one night of sleep. I don't like that Whoop saturates at 100 fairly easily. I find that Whoop is significantly better when it comes to the form factor. Having the tracker on your wrist is just so significantly easier and less intrusive into your daily life. In addition, you never have to take it off because the charger attaches on and off onto it!
\nI didn't find 8Sleep to be very reliable in its sleep tracking. The scores don't make as much sense to me when I wake up, and as we saw above they don't correlate very strongly with Whoop or Oura.
\nAutoSleep is basically a random number generator. Maybe there is a better app on Apple Watch for sleep tracking, but I haven't found it. Do not use.
\n![\"apps\"](\"https://bear-images.sfo2.cdn.digitaloceanspaces.com/karpathy/apps.webp\")
Above: The 4 apps. Left to right: Oura - I love this "grand overview" summary page, it's dense with just the info you want, and it's super easy to swipe left/right for other days. Whoop - less dense, you have to move around a lot to "treasure hunt" the information you want. 8Sleep - pretty decent. AutoSleep - looks cool but the numbers are all wrong so ¯\\(ツ)/¯.
Summarizing all of that into my advice right now: Get Whoop for 9.5/10, reliable, convenient sleep tracking with an excellent app (once you get to know it a bit). Get Oura for 10/10 tracking, if you're ok with the ring form factor.
\nDid I skip your favorite obviously best sleep tracker? Let me know on X @karpathy.
\n", "date_modified": "2025-03-25T14:59:58.705090+00:00", "date_published": "2025-03-24T23:00:00+00:00", "external_url": "https://karpathy.bearblog.dev/finding-the-best-sleep-tracker/", "id": "urn:aggr:item:82c6c05c33ed9d3e3c4e257149f7d5003d61778d", "summary": "Finding the best sleep tracker with data", "tags": [ "programming" ], "title": "Finding the Best Sleep Tracker", "url": "https://aggr.aymericbeaumet.com/items/karpathy-bearblog-dev/2025-03-24-finding-the-best-sleep-tracker/" } ], "language": "en", "title": "karpathy", "version": "https://jsonfeed.org/version/1.1" }