When testing isn't enough

Scientific code should be held to higher standards than other software. So it would help to write test cases that check if the outputs of our programs look plausible. But for some people, that’s not enough.

Yaron Minsky, who introduced the exotic programming language OCaml at the financial trading firm Jane Street, explains how they go about writing their sensitive systems:

We do an enormous amount of trading. There’s billions of dollars of nominal value kind of sloshing back and forth in the systems that we build. And what this means is, we are very nervous about technological risk. Because there is no faster way to light yourself on fire than to write a piece of software that makes the same bad decision over and over in a tight loop. (link)

He argues that on such a scale normal software testing isn’t enough, because even the very unlikely strange cases – that you haven’t thought about and written test cases for – might plausibly happen. So you have to understand the code really well and to make it readable, so that other people can check its correct functioning.