Medical Statistics: Testing hypotheses

An important use of medical statistics is to draw valid conclusions from observations. For example, we might identify a group of patients with brain tumours and another group of people without brain tumours, ask them about their previous mobile phone use, and conclude whether users of mobile phones are more likely than non-users to have brain tumours.

But we can never be absolutely sure about our conclusions. As we have already discovered in this series, statistics are often used when events are not entirely predictable.1 We usually only collect information on a sample. For example, when we collect information on 500 patients with a particular disease treated with drug A and another 500 patients treated with drug B, we then attempt to draw conclusions on the effectiveness of the drug treatments on all patients with such a disease. By chance, however, our sample of patients given one drug may be less severely ill than the patients given the other drug. This is why the conclusion we draw may depend on how certain we are.

If we find that 300 of the 500 patients taking drug A recovered, compared with 275 out of the 500 patients taking drug B, we might conclude that drug A is more effective than drug B at a particular level of certainty. But we might not be able to draw the same conclusion at a higher level of certainty. So we must decide on this significance level before we can draw any conclusions.

We need an easy method to calculate and measure this level of certainty. It is very complicated to calculate directly the probability of our conclusion being true. On the other hand, …