How your work gets marked, and where we admit we cannot

A tutor that tells you you are right when you are wrong is worse than no tutor, because you cannot find out. That single sentence decided most of how marking works here, so it is worth setting out what is actually checked, by what, and where the honest answer is that nothing can check it.

Six kinds of practice

A lesson sets whichever of these suits the idea it just taught:

• a number, where there is one right value
• a program, where there is a specification and tests
• a derivation, where the working is the answer
• an argument, such as a proof by induction
• an explanation in your own words, which is the thing worth being able to do in plenty of subjects
• a choice, where the misconceptions are the options

Five of the six are not judged by asking a model whether your answer looks right. Here is what judges them instead.

A program is run, one test at a time

Your code runs in a sandbox against the tests written for that problem, in Python, JavaScript or Java.

The detail that matters is not that it runs, it is what comes back. "Three tests failed" is what makes most automated marking useless. Each test reports the moment it ends, so a failure names the one test it failed, with the hint written for that particular misunderstanding. Code that loops forever is killed, and you still get the results of the tests that finished before it hung.

Working is checked line by line

For a derivation, marking the final answer is close to worthless teaching. "x = 2" is wrong in the same way whether you divided badly on the second line or lost a negative root on the last one, and being told only that the answer is wrong hands the diagnosis back to the person who has just shown they cannot do it.

So the working is read as a chain, and the question asked of each line is whether it follows from the one above. Two readings are kept, because one cannot serve both: ordinary algebra, where letters commute, and matrix algebra, where they do not and the order is often the entire content of the step. Under the scalar reading a step like transposing a product looks fine, and it is the most common mistake in the subject.

Beside that sits a second instrument we call margin numbers. Pin a value to every free symbol and each line becomes a number. Two consecutive lines of valid working give the same number, so the first pair that disagree is the slip, and nothing had to understand the mathematics to find it.

Arguments and explanations: a model, said plainly

A proof by induction, an epsilon and delta argument, a proof by contradiction: none of these is a chain of algebra, so the symbolic marker has nothing to hold, and there is nothing to run. For an explanation in your own words there is no instrument either.

Here a model marks, and we would rather say so than dress it up. Two things keep that survivable. The mark is made blind, from the question and what you wrote and nothing else, so it cannot be led by a reference answer it has already seen. And where a step rests on a definition or a named result that no instrument on our side can reach, the line is labelled unchecked, with a doubt offered as an opinion rather than recorded as a verdict.

The problem itself is marked before you see it

The same check runs one step earlier, and this is the part nobody thinks to ask about. When a problem is written, the reference answer is written with it, and the reference is put through the marker before the problem is allowed onto a page. A generated problem is not a problem until something has answered it correctly.

That is the only thing standing between a written course and a course full of questions with no right answer.

What this is for

Not a score. The record of what you got right and wrong is what the next lesson is built from: more practice where you keep slipping, less time on what you clearly have. Marking that flatters you breaks that chain at the first link.

Reading about it is not the same as watching your own wrong line get named. The first course is free from beginning to end.