We assumed the mathematics, and that was the wrong assumption
The library started at attention and backpropagation, which is where the interest is. Nineteen courses later it starts at what a letter is standing in for, and the reason is worth writing down.
The first courses we wrote were the ones people ask for: attention, backpropagation, cross entropy, how a model picks its next word. That is where the interest is, and writing them was the right place to start.
What it did was expose an assumption we had made without noticing. Each of those courses opens with a formula, because the subject is a formula, and the course can only be read by somebody who can read a formula. We had quietly addressed the whole library to that reader.
The honest version of the problem
The gap is not intelligence and it is not effort. It is a vocabulary of about thirty marks, plus a dozen conventions nobody states, that a reader is expected to have absorbed somewhere and usually has not. A sum sign is a for loop with an accumulator. A capital letter is probably a collection. The i under the sum does not exist once the sum is finished. None of that is hard. It is simply never said.
A reader missing it does not come back and tell you. They conclude the subject is above them and close the tab, which is the most expensive failure mode a learning product has, because it looks like nothing at all in the numbers.
What we did
Nineteen courses, written to sit underneath the ones that were already there, and then five reading orders through the whole library so that an arriving reader is given a route rather than a catalogue.
The on-ramp runs: reading the notation, logarithms and growth, counting without listing, chance and expectation, arguing that it always holds, vectors and distance, matrices as maps, change and accumulation. Then the shapes-of-work courses, how long it takes and finding it instantly and keeping things in order, because the second commonest place to stall is not a formula but a claim about scale.
From the library
Reading the Notation
The course that should have existed first: the thirty marks every other course here uses silently, each one named and said out loud.
Two things we would do differently
- 1
Write the prerequisite before the headline course, not after
It is tempting to write the course people search for and assume the rest. The cost of that assumption is invisible, which is exactly why it is worth being suspicious of.
- 2
Name a route by its outcome, not its subject
A reading order called evaluation does not get picked. The same one called put a model in front of people and know whether it is working does. Nothing about the courses changed, only the sentence on the front of them.
What has not changed
The courses in the library are the same kind of object they always were: eight lessons, figures drawn for each lesson, practice marked by instruments wherever an instrument exists. The point of writing more of them is not the catalogue. It is that a course written for you on request has to stand on something, and now it can stand lower down.
If you want the finished shape of this, the five tracks are the short version and the library is the long one. Forty-one of the eighty courses sit on a track today, and the courses we write next will be chosen by where the remaining gaps are rather than by what is interesting to write.