Data

Prediction interval or confidence interval?

Two intervals that sound alike: one bounds the mean, one bounds a new individual, and size papers use them differently.

By 3 min readData

Guides on Data: Meta-analysis statistics, start to finish, Reading a size study critically, How a size study is built, Reading a nomogram without fooling yourself, The Veale meta-analysis, read properly

A confidence interval says where a population's mean probably lies. A prediction interval says where a new individual's reading probably falls. Because individuals scatter far more than the mean's estimate does, the prediction interval is many times wider, and mixing them up changes a claim about the group into a claim about a person.

The two questions

A confidence interval answers: given this sample, what range probably contains the true population mean? It narrows as the sample grows, because a bigger sample pins the mean down.

A prediction interval answers: if I draw one more man from this population, what range probably contains his reading? It does not shrink to zero as the sample grows, because even a perfectly known mean leaves the individual's scatter intact.

The relationship to the standard deviation is the heart of it. A confidence interval is built on the standard error, the SD divided by the square root of n. A prediction interval is built on the SD itself, with a small correction for the uncertainty in the mean.

A worked comparison

The figures below take the erect length reported in Veale et al. (2015), BJU International: a mean of 13.12 cm and an SD of 1.66 cm, with 692 men contributing. Treating those as if they came from one simple sample is a simplification, and the paper's own pooled intervals use a more elaborate model. The point is the ratio, not the third digit.

Calculation Result
Standard error 1.66 / square root of 692 0.063 cm
95% confidence interval for the mean 13.12 plus or minus 1.96 x 0.063 13.00 to 13.24 cm
95% prediction interval for a new man 13.12 plus or minus 1.96 x 1.66 x 1.0007 9.86 to 16.38 cm

The confidence interval is about a quarter of a centimetre wide. The prediction interval is more than 6.5 cm wide. That is a factor of about 26, and it is just the square root of the sample size, roughly.

Both intervals are correct. They are answers to different questions, and the wrong one in the wrong sentence misleads.

Why papers confuse them

Many papers print a mean with a bracket after it, and the bracket might be an SD, a standard error or a confidence interval. Reading the error column in a published table is the habit that separates them, and confidence interval versus standard deviation sets out the first confusion.

A prediction interval adds a second confusion in meta-analysis. There, IntHout, Ioannidis, Rovers and Goeman (2016), BMJ Open, describe it as the expected range of true effects in similar studies, not the range of individuals. Their point is that a pooled confidence interval hides how much the underlying studies disagree, and a prediction interval exposes it.

So the phrase has two uses in the literature:

  • In a single sample, a range for one more individual.
  • In a meta-analysis, a range for one more study's true mean.

Both are wider than the confidence interval, for the same reason: they carry a variance that the confidence interval leaves out. Check which one a paper means before comparing it to anything.

Which one to use

For a statement about the group, such as "the pooled mean is about 13.1 cm", use the confidence interval. For placing one man's reading against the population, use the spread, which is what a prediction interval and a percentile both encode. What 15,521 men buys you argues that point from the sample-size side.

Neither interval is the uncertainty on your own measurement. That is a property of your technique, found with a repeatability test, and it is a third quantity.

A schematic in words

Picture a number line with the mean at the centre. Around it, a short bracket marks the confidence interval, a tight band a few tenths of a centimetre wide. Around that, a much longer bracket marks the prediction interval, spanning most of the observed readings. The short bracket gets shorter with every additional man, the long one stays roughly where it is.

Where scores differ

None of this carries over to a rating. A score from Rate Cock is a judgement about a photograph and has no population of repeat readings to bracket. How an AI scoring tool reports its own limits is covered at aipenis.com, and how a score is composed is covered at penisrater.com. How a rating site handles money is a question for cockrate.com, and the intervals above stay with measured lengths.

Read next

Full archive