Charlie KrugThe Build Log

← All posts

Backprop already fills in the Jacobian. Backflow just paints it.

Backflow is a small neural network you can drag. Each nudge runs a full forward and backward pass and colors every neuron by its gradient, the exact numbers backprop usually throws away.

Move one slider and the whole network lights up, one layer at a time, each neuron tinted by how much it cares about the input you just nudged. The colors aren't a metaphor for the gradient. They are the gradient: the exact numbers backpropagation computes on its way to a weight update, and then almost always throws away.

Backflow is a small neural network you can poke. No training curves, no decision boundary bending over epochs. There's a fixed little multi-layer perceptron, a slider per input, and a rule: drag an input, run a full forward and backward pass, then color every neuron by d(chosen output) / d(that neuron), cold to hot.

It exists because backprop is taught as an equation before it's ever felt as a phenomenon. Toys like TensorFlow Playground let you watch a boundary bend, but they never show the gradient itself, the signal that says which neuron actually mattered. The Jacobian, the full table of output-with-respect-to-input partials, stays chalk on a whiteboard even for people who call .backward() every day.

The visualization is the autodiff engine's own bookkeeping

Here is the one design choice the whole thing rests on. The network's neurons aren't floats. They're nodes in an automatic-differentiation graph. A scalar Value (about a screenful in src/autodiff/value.ts) wraps a number and, for every arithmetic operation it takes part in, records a tiny closure describing how to push a gradient backward to its operands. Add two Values and you get a third that remembers its parents and how to blame them.

Call .backward() from any node and it topologically sorts the graph behind that node and fans the gradient out in reverse, filling .grad on every upstream neuron, weight, and input in a single pass. That .grad, sitting on every node once the pass finishes, is one row of the network's Jacobian. There is no "now compute the Jacobian" step bolted on afterward. The picture on screen is just the numbers the engine already had to write down to do its job. The UI's entire contribution is to trigger the pass on every slider event and paint the .grad fields.

That is also why hovering an edge can show you real arithmetic: the weight, the upstream neuron's gradient, and their product, which is one entry of the Jacobian written out longhand.

No TensorFlow, on purpose

Importing TensorFlow.js would have made this trivial to build and impossible to fully explain. The point was the opposite: keep the whole mechanism small enough to read in one sitting, so every color on screen traces back to code you can point at instead of a vendored black box. The pattern (a Value type with a per-operation backward closure) is the same one Andrej Karpathy uses to teach autodiff in micrograd; Backflow builds a network and a live visualizer on top of it. The engine is tested against hand-derived analytic derivatives, not just "it runs," so the numbers you're staring at are provably the right ones.

One decision I underrated until it was wrong: rendering. An early version repainted on the slider's release. It felt like flipping between two states, not watching a flow. Moving to a devicePixelRatio-aware 2D canvas that redraws on every input event, cheaply enough to keep up, is what turns the recompute into a ripple that reads left to right in under 300ms. To spare anyone motion-sick, prefers-reduced-motion collapses it to instant.

Honest limitation: this is a teaching toy, not a training visualizer. The MLP is small and fixed, the weights don't learn while you watch, and the "randomize" button just draws fresh ones. Backflow shows you what a gradient is and how it flows, not how a real net converges. That was the trade I wanted.

Try it

Open Backflow, pick which output neuron you're differentiating, and drag one slider slowly. Watch which neurons flare and which stay cold: those are the ones that do and don't care about that input. Then hover a bright edge to read the weight times the upstream gradient, and you've read a single Jacobian entry off the network by hand. The code is on GitHub, autodiff engine and all.

Backflow is live. Free, in your browser, no signup.

This post is part of the build log: every app my automated factory ships gets written up here, honestly. Browse everything at apps.charliekrug.com. Comments are open below.

Comments

Loading comments…