Gear

Least count, resolution, accuracy

Least count is the smallest division, resolution is what you can distinguish, accuracy is closeness to true; an instrument can be fine on two and poor on the third.

3 min readGear

These three words get used interchangeably and they describe three different things. An instrument can be excellent on two of them and useless on the third, and the digital calliper below is the clearest example this site has.

Least count

Least count is the smallest division an instrument's scale actually marks - the gap between one printed line and the next, or one digit on a display and the next. A ruler with millimetre marks has a least count of one millimetre. A digital calliper that displays to 0.01 mm has a least count of one hundredth of a millimetre. Least count is a property of the printed or displayed scale, and it is fixed by manufacture; it tells you nothing on its own about what the instrument can usefully measure.

Resolution

Resolution is what a reader can actually distinguish between two close readings, given the instrument, the eye, and the material being read. On a millimetre ruler, a careful reader can usually interpolate to about half a millimetre by judging where a mark falls between two lines - resolution finer than the least count, in that case, because interpolation adds information the raw marks do not carry alone. On soft, compressible tissue read with a fine-scale instrument, resolution can be coarser than the least count, because the surface itself will not hold still to that precision even if the scale could show it.

Accuracy

Accuracy is how close a reading sits to the true value, and it is the one of the three that has nothing to do with how finely an instrument is marked. A ruler with true millimetre marks and a correctly seated zero is accurate even though its least count is coarse. A beautifully engraved scale on an instrument with an offset zero is inaccurate no matter how fine its divisions are - that offset is a separate fault with its own name, and no amount of resolution corrects it.

Where a digital calliper comes apart

A digital calliper displays two decimal places of a millimetre, which is an extremely fine least count. Closed on skin over the pre-pubic fat pad, or on shaft tissue, that display keeps updating as the jaws compress soft material by an amount that depends on how hard you close them - there is no natural stop the way there is for a rigid ruler pressed to bone. The last two digits of that reading are not describing your body more precisely than a coarser instrument would; they are describing how hard you squeezed. Fine least count, genuine resolution on a hard object, and poor accuracy on tissue - all three sit on the same instrument, disagreeing with each other.

Why the distinction is worth keeping straight

Reporting a figure to a resolution the instrument cannot support - a hundredth of a millimetre from a calliper on soft tissue, or a tenth of a millimetre interpolated by eye on a coarse scale - looks precise and is not, because precision borrowed from a fine least count is not the same thing as accuracy against the true value. The technique-side version of this problem is about not writing down more decimal places than three or five readings can support; the instrument-side version is knowing which of your tools' printed numbers you can trust in the first place. A rigid ruler with a coarse millimetre least count, read carefully and pressed to bone, will usually out-accuracy a calliper with a far finer display, because accuracy tracks the measurement's physical stability, not the number of digits on the readout.

What this has nothing to do with

None of these three words apply to a rating. A score from an image-based tool is not read off a scale with a least count at all - it comes from a model inferring a figure from visual cues, not from an instrument with divisions - and a rating service's output is a position in a distribution of photographs rather than a physical reading with any resolution to speak of, worth reading on its own terms rather than as a measurement. A person's stated opinion carries no least count either; a human judge's impression is honest as an impression and does not become more accurate by being stated to a decimal. Rate Cock's rating sits in that same category - a judgement, not a reading off any scale - which is exactly why questions about least count, resolution and accuracy belong to a ruler and a tape, not to it.

Read next

Full archive