Library · Basis Lab

The mathematics underneath

17 courses

The arithmetic every model is made of, worked through by hand before any of it is handed to a library.

Answering With Random SamplesGet a usable number out of a problem too awkward to solve exactly by simulating it, and say honestly how far off that number is likely to be and how many more samples would be needed to halve the error8 lessons
Arguing That It Always HoldsSay exactly what a claim asserts and establish that it holds in every case without checking every case, using the handful of argument shapes that every proof, every invariant and every correctness argument in this library is built from8 lessons
Backpropagation in FullFollow a derivative all the way from a scalar loss back to every weight in a deep network, with the rule for each kind of operation, the memory the method costs, and the reasons the signal can arrive too small or too large8 lessons
Chance, Spread and What to ExpectBuild the small set of ideas that every later course quietly assumes, from a list of outcomes with weights on it, through expectation and spread, independence and conditioning, to why an average of many draws settles down and what a number read off a sample is really telling you8 lessons
Change and AccumulationAsk of any rule how fast it changes and how much it adds up to, find that these are the same question asked in two directions, and be able to answer either one by hand, by approximation or by stepping it forward with a known error8 lessons
Counting Without ListingCount how many ways something can happen without writing them all out, and recognise that count as the denominator sitting under every probability you will later meet8 lessons
Keeping The Numbers In RangeBe able to explain why deep networks fall apart numerically, what initialisation, normalisation and skip connections each do about it, and how to read a training run that is going wrong8 lessons
Logarithms, Growth and ScaleWork with quantities that span many orders of magnitude without losing your footing, by deriving what a logarithm answers, why it turns multiplication into addition, why nearly every score in machine learning is the log of something, and how to read a chart whose axis has been squashed8 lessons
Markov ChainsModel anything that moves at random between a known set of situations, work out where it spends its time in the long run, how long it takes to get there and how long it takes to arrive anywhere in particular, and then run the argument backwards to build a chain whose long run is a distribution you chose8 lessons
Matrices as MapsBe able to read a matrix as a thing that moves vectors, say what it stretches and what it destroys, and know what the common decompositions are actually telling you8 lessons
Reading the NotationRead any line of mathematics in this library without translating it first, by knowing what each symbol is standing for, which marks are names and which are instructions, and how to say a formula out loud as the sentence it was compressed from8 lessons
Systems That EvolveTreat anything that changes over time as a state plus a rule that advances it, find where it would sit still, decide whether it would stay there, and say what the long run looks like without solving anything exactly8 lessons
The Fourier TransformTake any recorded signal apart into the steady rhythms it is made of, say exactly what that costs and what it preserves, and read a spectrum well enough to know which of its features belong to the signal and which were put there by the act of measuring it8 lessons
The Mathematics of Cross EntropyDerive the loss almost every classifier and language model is trained on, from the probability of the data through to what its value does and does not promise, well enough to read a loss of 1.8 and say what the model believes8 lessons
The Mathematics of Gradient DescentDerive why gradient descent works, what breaks it, and what every optimiser since is repairing, well enough to read a training curve and know which number to change8 lessons
Vectors, Distance and SimilarityBe able to say what it means for two things to be close, choose the right measure for a given set of vectors, and know what goes wrong with all of these measures when the number of dimensions is large8 lessons
Why Learning Works At AllBe able to say why a model that fits its training data might still be useless, judge a reported result by how it was measured, and explain what capacity, regularisation and more data each actually do8 lessons

Show the other 11 coursesShow fewer

How models work

16 courses

What is actually inside the models everybody uses, taken one idea at a time.

Show the other 10 coursesShow fewer

Making it fast enough

5 courses

How long a piece of work takes before you run it, and the handful of arrangements of data that most of the difference comes from.

The physics underneath

2 courses

The handful of quantities that do not change while everything else does, and what goes wrong in the arithmetic when a calculation loses one of them.

Putting a model into service

11 courses

Getting a trained model in front of people, paying for it, and knowing whether it is any good.

Show the other 5 coursesShow fewer

The systems around the model

7 courses

The machinery a model sits in: storage, speed, doing several things at once, and staying up.

Show the other 1 coursesShow fewer

How a program runs

6 courses

What happens under the code you write, from a pointer to the cache to the scheduler that decides nothing is happening right now.

Where the data lives

7 courses

Storing things so they can be found again: what an index really is, what a planner is deciding, and what a transaction actually promises.

Show the other 1 coursesShow fewer

The network and the web

5 courses

Everything between one machine asking and another answering, including the part that makes it private and the part that draws the page.

Keeping it safe

4 courses

The small number of mistakes behind most breaches, and the handful of habits that prevent them.