sketch.dev Says a Refactor Done by an LLM and Reviewed by a Human Turned a Break Into a Continue That Code Review Did Not Notice

sketch.dev says it uses its own product to build that product. On July 31, 2025 it published a short account of its first outage caused by code an LLM wrote: "We had a series of mini-outages at sketch.dev on July 15th, caused by LLM-written code." The account is the company's own. Parts of it follow, not the whole.

The Timeline

"A deployment appeared initially stable," the write-up says. "Some time afterwards, CPU spiked, and service incrementally slowed to a crawl." The profiles pointed at some very complicated SQL queries doing multiple full table scans, and the first hypothesis was that the database had reached a tipping point. The team fixed the queries and deployed again. "The same thing happened: initial stability followed by a downward spiral."

A bit of discussion indicated that the trigger for both spikes was the company's chief executive logging in, so the team redeployed for a clean start, permanently banned the chief executive from the service and moved on. The profiles kept pointing at database contention, which no longer seemed plausible. Walking up the stack, the team found one infrequent code path that triggered all of those queries, and that path "had recently been refactored." They reverted the refactoring, deployed, lifted the ban and began the analysis.

A Break That Became a Continue

"This refactoring was done by an LLM, and then reviewed by a human," the company says. "The bug was in a large chunk of code that moved from one file to another." Inside a loop that fetched repositories, an error used to end the loop with a break. After the move, the same place said continue. "The break became a continue, turning errors into infinite loops."

The review missed it. "This is a small enough change in a larger code movement that we didn't notice it during code review." The write-up says Git detects move-and-change at the file level, but not at the level of a patch hunk, and that "It's very easy to miss important changes in a sea of green and red that's otherwise mostly identical."

Root Causes and Prevention

The company's explanation is about how its agent edits. A human doing this refactor would cut the original text and paste it into the new file, so "Any changes after that would be intentional." An LLM coding agent works by writing patches, so to move code "they write two patches, a deletion and an insertion. This leaves room for transcription errors."

The original code carried a comment saying the program would log the error but continue with other installations, while the code itself said break. "Transcription said break. Local prediction said continue. Unfortunately for us, local prediction won." The write-up notes that the same kind of error had bitten one of its authors before LLMs, and that "this problem is exacerbated by LLM coding agents."

Under prevention, sketch.dev says it added clipboard support to the agent's environment, so that while patching files the agent can copy to and read from a clipboard, which reproduces code byte for byte, with support for adjusting indentation while pasting. It says the change "doesn't have many miles on it yet." It would also like to see Git add cross-hunk change detection, which, it says, will matter more "in a world in which more code is written by imperfect transcription."

How many moved blocks in a team's own history were read as moves, and never compared line by line?