--- title: 2024 Year in Review link: https://eugeneyan.com//writing/2024-review/ source: eugeneyan-com published: 2024-12-22T00:00:00Z updated: 2024-12-22T00:00:00Z first_seen: 2026-09-27T19:29:15.293681927Z labels: - life summary: A peaceful year of steady progress on my craft and health. content: extracted html: 2024-12-22-2024-year-in-review.html preview: file: 2024-12-22-2024-year-in-review.preview-4f80ea64f6ca.webp width: 256 height: 144 color: '#5b524f' images: - source: https://eugeneyan.com/assets/og_image/2024-review.jpg original: file: 2024-12-22-2024-year-in-review.image-0a62f12e94c9.jpg width: 1200 height: 675 color: '#181515' - source: https://eugeneyan.com/assets/aurora-2024.webp original: file: 2024-12-22-2024-year-in-review.image-4ccae49d9d99.webp width: 2400 height: 1800 variants: - file: 2024-12-22-2024-year-in-review.image-66bef68254e5.webp width: 320 height: 240 color: '#070704' - source: https://eugeneyan.com/assets/aligneval.webp original: file: 2024-12-22-2024-year-in-review.image-037c61949ce1.webp width: 1600 height: 1305 variants: - file: 2024-12-22-2024-year-in-review.image-dfc00983b38d.webp width: 320 height: 261 color: '#fafafa' - source: https://eugeneyan.com/assets/health-2024.webp original: file: 2024-12-22-2024-year-in-review.image-6d0f42c2d860.webp width: 1198 height: 679 variants: - file: 2024-12-22-2024-year-in-review.image-a0fc7763a427.webp width: 320 height: 181 color: '#fdfcfd' - source: https://eugeneyan.com/assets/search-2024.webp original: file: 2024-12-22-2024-year-in-review.image-8468b5a76688.webp width: 1200 height: 393 variants: - file: 2024-12-22-2024-year-in-review.image-960e4f99bd75.webp width: 320 height: 105 color: '#fcfcfc' - source: https://eugeneyan.com/assets/ga-2024.webp original: file: 2024-12-22-2024-year-in-review.image-f59185f09ad9.webp width: 1200 height: 623 color: '#fcfdfd' - source: https://eugeneyan.com/assets/ga-channels-2024.webp original: file: 2024-12-22-2024-year-in-review.image-d2afcf461ecb.webp width: 1200 height: 519 variants: - file: 2024-12-22-2024-year-in-review.image-92dd0eca7b66.webp width: 320 height: 138 color: '#fbfcfc' - source: https://eugeneyan.com/assets/ga-views-2024.webp original: file: 2024-12-22-2024-year-in-review.image-7601e59e42ea.webp width: 1200 height: 482 variants: - file: 2024-12-22-2024-year-in-review.image-b538c8d7efe1.webp width: 320 height: 129 color: '#f9fafb' - source: https://eugeneyan.com/assets/convertkit-2024.webp original: file: 2024-12-22-2024-year-in-review.image-942f76d7e58d.webp width: 1200 height: 475 variants: - file: 2024-12-22-2024-year-in-review.image-f910e0e5f55d.webp width: 320 height: 127 color: '#fafafc' - source: https://eugeneyan.com/assets/icon-twitter.svg original: file: 2024-12-22-2024-year-in-review.image-9f73746a86e9.png width: 512 height: 512 variants: - file: 2024-12-22-2024-year-in-review.image-10d32c0d6dac.webp width: 320 height: 320 - file: 2024-12-22-2024-year-in-review.image-1a1653690e12.webp width: 512 height: 512 color: '#000000' - source: https://eugeneyan.com/assets/icon-linkedin.svg original: file: 2024-12-22-2024-year-in-review.image-50dfb45d5f9e.png width: 505 height: 505 variants: - file: 2024-12-22-2024-year-in-review.image-b7fdeee63d68.webp width: 320 height: 320 - file: 2024-12-22-2024-year-in-review.image-003cf9435e7e.webp width: 505 height: 505 color: '#000000' - source: https://eugeneyan.com/assets/bluesky.svg original: file: 2024-12-22-2024-year-in-review.image-cd93481613cb.png width: 600 height: 530 variants: - file: 2024-12-22-2024-year-in-review.image-1790c4125e73.webp width: 600 height: 530 color: '#1084fd' - source: https://eugeneyan.com/assets/icon-facebook.svg original: file: 2024-12-22-2024-year-in-review.image-68df389f82c9.png width: 256 height: 256 variants: - file: 2024-12-22-2024-year-in-review.image-6035d4122baf.webp width: 256 height: 256 color: '#3b5998' - source: https://eugeneyan.com/assets/icon-mail.svg original: file: 2024-12-22-2024-year-in-review.image-bf2110bd7265.png width: 512 height: 512 variants: - file: 2024-12-22-2024-year-in-review.image-c80acd329b85.webp width: 320 height: 320 - file: 2024-12-22-2024-year-in-review.image-84c4e78a876d.webp width: 512 height: 512 color: '#000000' --- 2024 was a peaceful year of steady progress. With regard to my craft, the prototypes of 2023 were scaled and put into production, and I rediscovered the joy of building in public. On the personal side, I continued the prior year’s focus on health, further improving my diet and exercise habits, leading to measurable results. Past years: [2020](https://eugeneyan.com/writing/retrospective-2020/), [2021](https://eugeneyan.com/writing/2021-year-in-review/), [2022](https://eugeneyan.com/writing/2022-in-review/), [2023](https://eugeneyan.com/writing/2023-review/) ## Reviewing my 2024 goals ✅ **Work:** Shipped ML/LLM systems that serve customers at scale. 2024 was the year of productionizing the prototypes of 2023. I learned how to deploy these systems reliably, at scale, and cost-effectively, for both customer-facing UXes and internal pipelines. ✅ **Writing:** Published 6 substantive pieces by writing the following: - [How to Generate and Use Synthetic Data for Finetuning](https://eugeneyan.com/writing/synthetic/) - [Task-Specific LLM Evals that Do & Don’t Work](https://eugeneyan.com/writing/evals/) - [Prompting Fundamentals and How to Apply them Effectively](https://eugeneyan.com/writing/prompting/) - [Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)](https://eugeneyan.com/writing/llm-evaluators/) - [How to Interview and Hire ML/AI Engineers](https://eugeneyan.com/writing/how-to-interview/) A major highlight was collaborating with several friends to write [What We’ve Learned From a Year of Building with LLMs](https://applied-llms.org/). This was also published on O’Reilly as a [book](https://www.oreilly.com/library/view/what-we-learned/9781098176716/) and technical series (parts [I](https://www.oreilly.com/radar/what-we-learned-from-a-year-of-building-with-llms-part-i/), [II](https://www.oreilly.com/radar/what-we-learned-from-a-year-of-building-with-llms-part-ii/), [III](https://www.oreilly.com/radar/what-we-learned-from-a-year-of-building-with-llms-part-iii-strategy/)). I also had fun writing a few shorter, snappier pieces: - [Don’t Mock Machine Learning Models In Unit Tests](https://eugeneyan.com/writing/unit-testing-ml/) - [39 Lessons on Building ML Systems, Scaling, Execution, and More](https://eugeneyan.com/writing/conf-lessons/) - [How to Run a Weekly Paper Club](https://eugeneyan.com/writing/paper-club/) - [My Minimal MacBook Pro Setup Guide](https://eugeneyan.com/writing/mac-setup/) - [Seemingly Paradoxical Rules of Writing](https://eugeneyan.com/writing/paradox/) On public speaking, while it isn’t an explicit goal, I try to speak once a year for the sake of practice. In 2024, I was invited to speak at [Netflix’s internal RecSys conference](https://eugeneyan.com/speaking/netflix-prs/), deliver the [closing keynote](https://eugeneyan.com/speaking/aie-2024/) at the AI Engineer World’s Fair with my fellow authors, and judge the [Weights & Biases LLM-Judge hackathon](https://eugeneyan.com/speaking/hackathon-judge/). Also, I got to [present my paper](https://x.com/eugeneyalt/status/1846238823333662880) on detecting factual inconsistencies at the Amazon Machine Learning Conference. ✅ **Health:** Further improved my diet and exercise habits. After avoiding jogging in the first half of 2024, I restarted in June. I also bought a [body fat scale](https://www.amazon.com/dp/B01N1UX8RW) on Prime Day in July, and inspired by this [tweet](https://x.com/chrisalbon/status/1832946165446553716), I began intermittent fasting by skipping breakfast. Overall, I’m now exercising daily with a slight calorie deficit. Here’s my weekly routine: - At least two sessions of weight training - At least one session of jogging - The rest being Zone 2 - 3 cardio on the stair stepper or treadmill 🟠 **Travel:** One of two planned trips completed. We traveled to Alaska to catch the Northern Lights ([November 2024 - March 2026](https://abcnews.go.com/US/peak-northern-lights-activity-occur/story?id=113839425) is the the peak of the 11-year Solar Cycle). Next we have a snowboarding trip to Hakuba Valley in Febuary 2025. ![Northern Lights in Alaska](https://eugeneyan.com/assets/aurora-2024.webp "Northern Lights in Alaska") Northern Lights in Alaska ❌ **Snowboarding:** Did not do a black run in 2024. In hindsight, this was a stretch given our limited time on the slopes (~5 times a year). We ended 2024 with a lesson to get more comfortable on the blue runs. With our upcoming Hakuba Valley trip and a few more sessions on Crystal, we’re hoping to get comfortable enough to attempt a black run in 2025. ✅ **Learn something new:** Beyond diving into [synthetic data for finetuning](https://eugeneyan.com/writing/synthetic/), [application-specific evals](https://eugeneyan.com/writing/evals/), [LLM-evaluators](https://eugeneyan.com/writing/llm-evaluators/), and [various web frameworks](https://eugeneyan.com/writing/web-frameworks/), I also improved my product thinking via Shreya Doshi’s [Product Sense](https://maven.com/shreyas-doshi/product-sense) course. ## Building AI Coach & AlignEval Inspired by this [tweet](https://x.com/swyx/status/1765995892107317407), I explored voice as a modality by building [Tara, an AI Coach](https://eugeneyan.com/writing/ai-coach/) you can talk to on the go. Since launching, Tara has helped hundreds of people. A few of them have reached out to share positive, constructive feedback. My favorite story is how Tara convinced someone’s spouse to quit their horrible role after reminding them, in a gentle yet tough-love tone, that it was a “soul-sucking job”. While researching and writing about [LLM-evaluators](https://eugeneyan.com/writing/llm-evaluators/), I started developing an idiosyncratic take on how we should build LLMs as judges. To test these ideas, I built [AlignEval](https://aligneval.com/). ![AlignEval: Label -> Evaluate -> Optimize](https://eugeneyan.com/assets/aligneval.webp "AlignEval: Label -> Evaluate -> Optimize") AlignEval: Label -> Evaluate -> Optimize The response and usage has been more positive than I expected. So far, users have: - Uploaded 538 dataset files - Labeled ≥ 1 sample for 454 files - Labeled ≥ 20 samples for 289 files and ran an LLM-evaluator - Labeled ≥ 50 samples for 84 files and optimized their LLM-evaluator Building AI Coach and AlignEval was a blast, and the feedback from users has been valuable. I already have ideas for AlignEval v2. ## Doubling Down on Health In 2023, I started to watch my sugar, saturated fat, and alcohol intake, and exercised 3 - 4 times a week. 2024 is more of the same, with: - Limiting alcohol/snacks to three servings per week - Intermittent fasting by skipping breakfast (since September) - Tracking body fat percentage (since September) - Increasing protein intake (since September; [David bars](https://davidprotein.com/) are great) - Doing intervals for more heart-pumping exercise (since June) - At least 30 minutes of exercise every day (since June) These changes have helped lower my resting heart rate and body fat percentage. ![Resting heart rate and body fat percentage decreased since the middle of 2024](https://eugeneyan.com/assets/health-2024.webp "Resting heart rate and body fat percentage decreased since the middle of 2024") Resting heart rate and body fat percentage decreased since the middle of 2024 ## Statistics for 2024 Google Search traffic took a hit in 2024, with impressions falling from 2.71M to 1.97M (-27.3%) and clicks dropping proportionately from 77.6k to 56.5k (-27.2%). While this data is from a single site, it might hint at the scale of the decline in Google’s traffic. Also, despite my average position improving from 31.5 to 24.6, CTR held steady at 2.9%. ![Impressions and clicks from Google Search fell in 2024](https://eugeneyan.com/assets/search-2024.webp "Impressions and clicks from Google Search fell in 2024") Impressions and clicks from Google Search fell in 2024 Given the dip in search metrics, it’s surprising that eugeneyan.com saw a 10.5% increase in unique visitors, reaching 285k in 2024. The graph suggests this was driven by outlier traffic, likely from HackerNews. ![Despite the hit on Search traffic, users and views continued growing from 2023](https://eugeneyan.com/assets/ga-2024.webp "Despite the hit on Search traffic, users and views continued growing from 2023") Despite the hit on Search traffic, users and views continued growing from 2023 The distribution of incoming channels and geography remained similar to 2023. ![Image](https://eugeneyan.com/assets/ga-channels-2024.webp "Image") Incoming channels and geography remained similar to 2023 Seven of the top 10 pages were published in 2024 though three older pieces continued to resonate: (i) [Patterns for Building LLM Systems](https://eugeneyan.com/writing/llm-patterns/) with 62.4k views, (ii) [Simplicity](https://eugeneyan.com/writing/simplicity/) with 23.4k views, and (iii) [System Design for RecSys](https://eugeneyan.com/writing/system-design-for-discovery/) with 18k views. ![Top 10 most popular write-ups in 2024](https://eugeneyan.com/assets/ga-views-2024.webp "Top 10 most popular write-ups in 2024") Top 10 most popular write-ups in 2024 Social metrics continued growing - Email subscribers from 6.2k to 9.5k (+34.7%); 54.9% open rate and 6% click rate - Twitter followers from 15.8k to 21.3k (+25.8%) - LinkedIn followers from 32.7k to 37.7k (+13.3%) ![Subscriber growth continued in 2024, with the increase since June due to recommendations](https://eugeneyan.com/assets/convertkit-2024.webp "Subscriber growth continued in 2024, with the increase since June due to recommendations") Subscriber growth continued in 2024, with the increase since June due to recommendations Side note: Compiling these engagement metrics felt less insightful this year. I might skip them in the next annual review. ## Goals for 2025 - **Write 6 substantive pieces** - (New) **Prototype 4 apps in public**: Building AI Coach and AlignEval in public was educational, lots of fun, and yielded valuable feedback. Will do more in 2025. - **Health**: Stay consistent with diet and exercise until they become ingrained habits - **Snowboarding**: Try a black run by the end of the year Several goals from previous years, like meditating, learning, and traveling, have become so habitual that restating them annually feels unnecessary. On the other hand, some of my newer goals feel too personal to share publicly right now—perhaps in a future review! If you found this useful, please cite this write-up as: > Yan, Ziyou. (Dec 2024). 2024 Year in Review. eugeneyan.com. https://eugeneyan.com/writing/2024-review/. or ``` @article{yan20242024-review, title = {2024 Year in Review}, author = {Yan, Ziyou}, journal = {eugeneyan.com}, year = {2024}, month = {Dec}, url = {https://eugeneyan.com/writing/2024-review/} } ``` Share on: ![](https://eugeneyan.com/assets/icon-twitter.svg) ![](https://eugeneyan.com/assets/icon-linkedin.svg) ![](https://eugeneyan.com/assets/bluesky.svg) ![](https://eugeneyan.com/assets/icon-facebook.svg) ![](https://eugeneyan.com/assets/icon-mail.svg) Join **11,800+** readers getting updates on machine learning, RecSys, LLMs, and engineering.