Charlie KrugThe Build Log

← All posts

Last year's AI tells are useless. The bank resets every Monday.

Spot the Bot is a two-minute game: ten short passages, human or AI. It regenerates its whole passage bank every Monday against the week's topical model, so the round you play is never older than this week.

Today was a meaningful reminder that life's small moments matter most. I paused to appreciate the morning light and felt grateful for the quiet beauty that surrounds us every single day.

A person wrote a lot of diary entries in the world. A machine wrote that one. The tell is the tidy summary sentence and the even, grateful tone: a real diary is messier and more specific, and it almost never stops to explain its own moral back to itself. If you caught that, good. The problem is that the tell you just used is already going out of date.

Spot the Bot is a two-minute browser game. You read ten short passages, one at a time, and tap HUMAN or AI for each. No account, no install, no back button. At the end you get a score, the passages that fooled you, and the name of the model that fooled you twice. That last part is the whole point, and I will get to why.

Why every other version of this goes stale

Almost every "guess the AI" quiz on the internet is a hardcoded list of ten examples: written once, shipped, never touched again. It is funny for a week and then it is a museum piece, because the models move faster than any hand-curated example set can track. By the time you have memorized the tics of last year's model, the quiz is still testing you on last year's model. The content is the thing that ages, and static content ages badly.

So in Spot the Bot the content is not a fixed asset. A scheduled script generates a fresh passage bank every Monday. It prompts the week's topical model for short passages across a mix of everyday styles (news lede, product review, diary entry, recipe intro, and more), then pairs each AI passage with a length- and subject-matched human one so the choice is never giveable by length or topic alone. The set gets shuffled, de-identified, validated, and written to a dated file at src/data/banks/YYYY-MM-DD.json. A bank that comes out malformed or too short never lands on disk.

The clever part is the load, and it is boring on purpose. At build time the app reaches for the most recent bank dated on or before today, and falls back to a committed seed bank if it finds nothing. Old banks are never deleted. That single rule, newest bank not in the future, means any week's round draws from writing no older than that week, and the game is never blank even if a generation run is skipped. There is no server anywhere in this: the bank is a versioned JSON artifact, so hosting stays free and the game keeps working offline once it has loaded.

Naming the model is the hook

A bare percentage is not shareable. "You got fooled by the model everyone is arguing about this week" is. When the pipeline has the metadata, the reveal does not say a vague "AI"; it says which model slipped past you, and how many times. The share card reads like Spot the Bot 8/10 · Fooled 2x by Claude 5 Sonnet · Week of 2026-07-06, and it is deliberately spoiler-free, so posting your score never leaks the week's answers to the next player.

One honest limitation. Right now the generator draws its AI passages from a curated offline pool rather than a live API call, which is what lets CI run with no API key and no per-build cost. Swapping in a real model call is a single documented seam in the pipeline, not a rewrite. The weekly cadence is a deliberate call too: frontier releases and news cycles turn over on a weekly rhythm, and a daily refresh would mostly reshuffle the same underlying model for no new signal.

The game logic underneath all this is small and pure, and sits at 100% line coverage with fast-check property tests on the RNG, the scoring, and the boundary where untrusted bank JSON comes in. That is the part I trust; the passages are the part meant to make you doubt yourself.

Try it

Open Spot the Bot, read the first passage, and press H or A. Ten rounds, no going back, about two minutes. Watch the reveal to see which model fooled you and how badly, then play again for a different round from the same week's bank. The code and the pipeline are on GitHub if you want to see how a bank gets built.

Spot the Bot is live. Free, in your browser, no signup.

This post is part of the build log: every app my automated factory ships gets written up here, honestly. Browse everything at apps.charliekrug.com. Comments are open below.

Comments

Loading comments…