Charlie KrugThe Build Log

← All posts

Regex catches the keys with names. Entropy catches the rest.

Safelog is a zero-dependency Python filter you drop into a live pipe so your AI agent gets the stack trace without the AWS key. Regex first, Shannon entropy for the long tail.

The fastest way to debug something in 2026 is to shove the output at an agent and ask what's wrong. It works. It also means that whatever was sitting in that stream when you hit enter has now been handed to a third party: the .env a debug print dumped, the Stripe key embedded in a CI log, a customer's email in a crash report. You didn't decide to share those. You decided to share a stack trace, and they came along.

Safelog is a filter you put between the two. tail -f app.log | safelog | claude. It reads stdin one line at a time, replaces anything that looks like a secret with a labeled placeholder, and writes the line back out. Roughly 500 lines of Python across eight small modules, standard library only.

The batch scanners already exist and are good: gitleaks, truffleHog, detect-secrets. All of them assume a file at rest and a job you run. None are built to sit inline in a live pipe, redacting bytes as they fly past, with latency low enough that the human on the other end never notices. That's a different shape of problem, and it's the only one Safelog solves.

Two detectors, and the order matters

Layer one is regex, for secrets that have a recognizable shape. AWS keys, GitHub and GitLab and Slack tokens, Stripe keys, JWTs, PEM private key blocks (collapsed across however many lines they span), emails, IPv4 and IPv6. These are precise and nearly free to run on every line. If a string starts with sk_live_ you don't need to think hard about it.

Layer two is for everything with no prefix to match on: a generic 32-character API key that some internal service minted and gave no vendor branding. There's nothing to pattern-match, so Safelog measures Shannon entropy over token-like substrings instead. Entropy here means average bits of surprise per character. English prose runs around 1 to 1.5 bits per character, because after th you can guess e most of the time. A random base64 token approaches 6 bits per character, since log2(64) is 6 and every position is genuinely unpredictable. That gap is wide enough to sort "hostname" from "kJ8vQ2mNp7xR4wL9". The threshold is tunable with --entropy-threshold, because the tail end of that gap holds git SHAs and UUIDs too.

Crucially, entropy runs only where regex found nothing. It's the more expensive check and the one prone to false positives, so it gets the leftovers rather than the first pass, and it can't re-flag or corrupt something already redacted.

Redact the secret, not the line

The other decision that shows up on every line of output: Safelog replaces the matched span, not the line containing it. Blanking the whole line would be safer in a trivial sense and useless in practice, because the timestamp, the log level, and the service name are the parts you were debugging with.

2026-07-17T12:00:03Z ERROR stripe key sk_test_[REDACTED:stripe-key] rejected
2026-07-17T12:00:05Z INFO  client [REDACTED:ip] connected

You can still see that a Stripe key was there, and roughly where. That label is deliberate over a generic ***: knowing an AWS secret got pulled out of line 12 is real triage information, and it tells an attacker nothing. If you disagree, --mode mask gives you ***, and --mode hash gives a stable per-secret hash so you can tell whether the same key appeared twice without learning what it is.

Two smaller things I liked getting right. It exits 141 when a downstream reader closes the pipe early, which is 128 + SIGPIPE, the same code cat gives you when you quit a pager. A tool that lives in a pipe should behave like the tools around it. And the benchmark script fails the run if overhead exceeds 200ms per 1000 lines; it typically measures 60 to 125.

The honest limitation, from the README: this matches shapes, not meaning. A low-entropy secret with no recognizable prefix walks straight through. password=hunter2 is still password=hunter2 on the other side. Safelog reduces what leaks. It does not make a stream safe to publish.

Try it

Clone it and run cat fixtures/sample.log | PYTHONPATH=src python3 -m safelog to watch a log full of fake keys come out clean while the timestamps and stack frames pass through untouched. Then put it in a real pipe: append | safelog to whatever you were about to paste into an agent. Zero config needed, the defaults are on. The source is on GitHub, one file at a time, short enough to actually read before you trust it with your logs.

Safelog is live. Free, in your browser, no signup.

This post is part of the build log: every app my automated factory ships gets written up here, honestly. Browse everything at apps.charliekrug.com. Comments are open below.

Comments

Loading comments…