Backprop already fills in the Jacobian. Backflow just paints it.
Backflow is a small neural network you can drag. Each nudge runs a full forward and backward pass and colors every neuron by its gradient, the exact numbers backprop usually throws away.
Move one slider and the whole network lights up, one layer at a time, each neuron tinted by how much it cares about the input you just nudged. The colors aren't a metaphor for the gradient. They are the gradient: the exact numbers backpropagation computes on its way to a weight update, and then almost always throws away.
Backflow is a small neural network you
can poke. No training curves, no decision boundary bending over epochs. There's
a fixed little multi-layer perceptron, a slider per input, and a rule: drag an
input, run a full forward and backward pass, then color every neuron by
d(chosen output) / d(that neuron), cold to hot.
It exists because backprop is taught as an equation before it's ever felt as a
phenomenon. Toys like TensorFlow Playground
let you watch a boundary bend, but they never show the gradient itself, the
signal that says which neuron actually mattered. The Jacobian, the full table of
output-with-respect-to-input partials, stays chalk on a whiteboard even for
people who call .backward() every day.
The visualization is the autodiff engine's own bookkeeping
Here is the one design choice the whole thing rests on. The network's neurons
aren't floats. They're nodes in an automatic-differentiation graph. A scalar
Value (about a screenful in src/autodiff/value.ts) wraps a number and, for
every arithmetic operation it takes part in, records a tiny closure describing
how to push a gradient backward to its operands. Add two Values and you get a
third that remembers its parents and how to blame them.
Call .backward() from any node and it topologically sorts the graph behind that
node and fans the gradient out in reverse, filling .grad on every upstream
neuron, weight, and input in a single pass. That .grad, sitting on every node
once the pass finishes, is one row of the network's Jacobian. There is no "now
compute the Jacobian" step bolted on afterward. The picture on screen is just the
numbers the engine already had to write down to do its job. The UI's entire
contribution is to trigger the pass on every slider event and paint the .grad
fields.
That is also why hovering an edge can show you real arithmetic: the weight, the upstream neuron's gradient, and their product, which is one entry of the Jacobian written out longhand.
No TensorFlow, on purpose
Importing TensorFlow.js would have made this trivial to build and impossible to
fully explain. The point was the opposite: keep the whole mechanism small enough
to read in one sitting, so every color on screen traces back to code you can
point at instead of a vendored black box. The pattern (a Value type with a
per-operation backward closure) is the same one Andrej Karpathy uses to teach
autodiff in micrograd; Backflow builds a
network and a live visualizer on top of it. The engine is tested against
hand-derived analytic derivatives, not just "it runs," so the numbers you're
staring at are provably the right ones.
One decision I underrated until it was wrong: rendering. An early version
repainted on the slider's release. It felt like flipping between two states, not
watching a flow. Moving to a devicePixelRatio-aware 2D canvas that redraws on
every input event, cheaply enough to keep up, is what turns the recompute into
a ripple that reads left to right in under 300ms. To spare anyone motion-sick,
prefers-reduced-motion collapses it to instant.
Honest limitation: this is a teaching toy, not a training visualizer. The MLP is small and fixed, the weights don't learn while you watch, and the "randomize" button just draws fresh ones. Backflow shows you what a gradient is and how it flows, not how a real net converges. That was the trade I wanted.
Try it
Open Backflow, pick which output neuron you're differentiating, and drag one slider slowly. Watch which neurons flare and which stay cold: those are the ones that do and don't care about that input. Then hover a bright edge to read the weight times the upstream gradient, and you've read a single Jacobian entry off the network by hand. The code is on GitHub, autodiff engine and all.
This post is part of the build log: every app my automated factory ships gets written up here, honestly. Browse everything at apps.charliekrug.com. Comments are open below.
Loading comments…