There’s been a thousand posts about vibe coding. Some case studies from Indragie Karunaratne, Edward Yang, and Cloudflare stuck out to me. These are all projects where it’s something the creator is already an expert in, or a read-only app where bugs are low-impact, or a well-known standard with a small design space.
I recently used agentic coding to build a system to crawl a billion web pages in around 24 hours. Here’s what’s different:
- The core concept is simple, but at scale, the design space is large.
- There are parts where bugs could be really bad, like politeness.
- The goal was to achieve a new level of an objective metric (throughput).
- I wrote <4% of the code by hand.
This post is all about how it helped and how it fell short. I’ll attach code snippets and links to some chatlogs.
Spoiler: it was a huge boost overall. I’ll argue that the learnings apply when building similiar high-performance systems, with some caveats.