Guest Post: The Higgs Boson: How Sure is Pretty Sure?

You'll have heard, unless you particularly like the conditions under rocks, that the folks at CERN's LHC have found some pretty convincing evidence for the existence of what is known as the Higgs boson. At long last they've got something to work with beyond 'it should be there, and it should be a bit like this'. They're pretty sure that what they've found is the Higgs boson and not just something that happens to act a bit like one should act. But how sure is 'pretty sure'? Friend of the blog Colin Beveridge**** answers that question. Apologies for letting through a small instance of bad language. I thought it worked.

“They’re only pretty sure they’ve found the Higgs boson, aren’t they?”

If I had a billion dollars for every time I’d heard that question, I’d have enough to build a medium-sized hadron collider. Technically, that’s right: they’re only pretty sure. But, as with all things scientific, there’s ‘pretty sure’ and there’s ‘pretty sure.’

Luckily, science has a better way of talking about things than just saying ‘we’re pretty sure’, and it’s — surprise! — to put a number on things. You’ll quite often see something described as ‘a three-sigma event’ or — in the case of the Higgs — ‘a five-sigma discovery’. (There’s a business philosophy that, like most business philosophies, had a year or two in the spotlight and then vanished, called ‘six-sigma’ — it’s the same idea.) The higher the number, the more certain you are about the event⁠*.


So what is it this mysterious sigma represents?

Well, it’s to do with hypothesis testing, and it’s based on the mathematical idea of proof by contradiction. The idea is, if you want to show that something is true, you can prove it by showing that the opposite isn’t true. For instance, if you wanted to prove that there was no biggest number, you’d start by saying “let’s assume the opposite! Assume there is a biggest number.” Being a mathematician, you’d probably call it N, but you don’t have to. “Now, add one to that number — you’ve got a bigger number now, so that assumption about having a biggest number? Load of bollocks⁠**. Therefore, there can’t be a biggest number.”

Got it? To prove something is true, you can assume that it isn’t true, and show that reality is inconsistent with that assumption.

That’s sort of how hypothesis testing works, too, but it’s a little bit trickier. When you’re doing nice proofs about real numbers, all the properties are known and you can say for certain (most of the time) whether something is absolutely, definitely true or not. When you’re dealing with the real world, it’s not so simple. As Einstein said, “As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.” You really can’t show that something is definitely untrue in science — only that it’s extremely unlikely.

So that’s what you do! You assume the opposite of what you’re trying to prove (and call it the null hypothesis, if you’re that way inclined). You say what you’re trying to prove (the alternate hypothesis). You work out a threshold of how certain you want to be⁠*** — which is about the same thing as saying how unlikely the null hypothesis needs to be — and then you do the experiment.

If your observations would be absurdly unlikely under the null hypothesis, but quite plausible under the alternate hypothesis, you throw out the null hypothesis as a bad job, and accept the alternate hypothesis. If your observations aren’t unlikely enough, no matter how unlikely they are, you don’t have enough evidence for what you were trying to prove. Sorry.


So, where do the sigmas come in?

They come from the normal distribution, which looks to me like a boa constrictor that’s just eaten an elephant. It’s a shape that comes up repeatedly in science — most measurements are pretty close to the mean, in the middle of the graph, and the further away from the middle you get, the less likely the result. For instance, if you look at the heights of a large group of 25-year old adult males, you’d expect most of them to be about 175cm tall, most people within — I’d guess — 15cm of that, and possibly some extremely tall or extremely short people. The graph of those heights would looks something like a normal distribution.

There are two important bits of information there: the 175cm, which is the mean of the distribution — how tall you expect a random person picked from the crowd to be; and the 15cm, which is the standard deviation. The standard deviation is a bit harder to explain — in very loose terms, it’s the ‘give or take’: if you say “a person from this group will be between 160cm and 190cm tall”, you should be right about 70% of the time. That’s the nature of the normal distribution.

Sigma is just the number of standard deviations an observation is away from the mean. So, if you had a person in the group who was exactly 175cm tall, that would be a zero-sigma event. Someone 160cm tall is 15cm away from the mean, or one standard deviations, making him a 1-sigma event. Someone 205cm tall — a big lad — would be 30cm, or two standard deviations from the mean, making him a 2-sigma event.

The thing about the normal distribution is, it tails off very quickly. A one-sigma event in a given direction shows up about 14% of the time (around one in seven). A two-sigma event, only about 2% (one in 44), and a three-sigma event, barely 0.1% (1 in 740). By the time you get to 4 sigma, you’re talking about one in 32,000, and 5 sigma is one in three and a half million — about the same as winning the lottery if you buy four tickets.


So, the answer is ‘yes, they’re only pretty sure’, but in this context, it means that the alternative is preposterously unlikely. Not impossible, of course, they could just have got lucky and found something Higgs-shaped exactly where they happened to be looking for it — and if that’s the case, we’ll find out more about it in the next few months.

* I’ll get into the details of what the sigma means later.
** Probably not the technical language you’d use in a formal proof.
*** I’m getting to the sigmas, promise!
**** Colin's a mathematician with his finger under many belts, if you'll excuse the mixed metaphor: he's a maths tutor, a maths blogger, a maths writer, and a folk guitarist too.