Method
The blind test: how we measure our own estimating accuracy
We don't quote an accuracy percentage — we run a blind test instead. Here's how the estimating accuracy test works, and what the variance actually shows.
Insights Published · 8 min · By Scaleline
For Australian builders of new homes.
An estimating accuracy test usually means someone quoting you a percentage. We do it differently: we run a blind test, on a job you’ve already built, and show you the variance line by line. Here’s how the test works, why it’s more honest than a number, and what the result actually tells you about whether we’re any good.
Why we won’t quote an accuracy percentage
Most estimating pitches lead with a precision figure, and the figure is usually reassuring. The problem is that a single number hides the spread behind it — a wall measured from a dimensioned elevation is not the same strength as a cable run inferred from a room size, and folding both into one percentage tells you nothing about either.
So we don’t quote a blanket accuracy percentage, and we don’t ask you to trust one. We show you the mechanism, then invite you to test it. The mechanism is the part you can inspect: vector and text layers read first, a local estimator checking what the annotations leave out, and every line carrying its source page, dimension and confidence. The test is the part you can run yourself. Between the two, you get something a percentage can’t give you — a number you can check, not a number you have to believe.
Our method page explains the mechanism in full. The blind test is the proof of it, and that’s what the rest of this piece is about.
How the blind test works, step by step
The test is built around one rule: we never see your costs. You pick a job you’ve already finished, send us the drawings, and hold back what it actually cost you. We measure the job independently, in 24–48 hours, and then we compare our quantities against what you already know. That’s it — no order, no cost disclosed, no sales call.
The sequence matters. First you book it through a short form. Then you send one set of drawings from a real build — not a hypothetical, not a tender, something that’s already on the ground. We measure it blind, because the test only works if there’s nothing for us to fish for. Then we sit down with you and walk the variance line by line, showing where we matched and where we didn’t.
The comparison is the whole point. We don’t produce a headline “we were this far out” figure; we show the actual lines, the actual quantities, and where each one diverges from the job you know inside out. You read the whole thing on our blind test page, and book it from there.
What the variance actually shows you
The variance is more useful than a percentage because it’s diagnostic. When a line matches, it tells you the drawing was clear and the measure was right. When a line diverges, it tells you why — a drawing that didn’t pin something down, or a trade we only measure to a summary. That “why” is the part a percentage throws away.
Think of it as the difference between a grade and a marked exam. A grade tells you how you did; a marked exam tells you exactly which lines were right and which weren’t, and why. For a builder deciding whether to trust a take-off on the next job, the marked exam is the only version that’s actually useful. You can decide, line by line, whether the divergence is the kind that matters to you.
That’s also why the blind test uses your real build rather than a synthetic one. A synthetic test can be gamed, consciously or not, because someone picked the drawings knowing they’d be tested. A job you’ve already built is the honest benchmark — the drawings existed before the test did, and the numbers are what actually happened.
How to read the comparison honestly
Reading the comparison is as important as the comparison itself, and there are a few ways to read it wrong. The first is to treat any divergence as a failure. Some lines should be estimates — a cable route or a painting breakdown isn’t on the drawings, so no estimator could measure it exactly, and the honest report says so up front. The question isn’t whether an estimate line is approximate; it’s whether the report told you it was.
The second way to read it wrong is to fixate on the light trades and ignore the deep ones. Walls, slab and framing are where the geometry lives and where a take-off should be close. Electrical and painting are where the drawings carry less and the estimate carries more. The useful signal is how the deep trades performed, and whether the estimate lines were flagged as estimate rather than dressed up as measure.
The third is to expect the same closeness everywhere a percentage might imply. A bill of quantities states an accuracy class per section precisely because the firmness varies. The blind test shows you that variation in the open, which is the only way it can be checked.
What a blind test can’t tell you
A blind test is powerful, but it has limits, and we’d rather name them than oversell. One job is one job. It tells you how we measured that drawing set, not how we’d measure every drawing set you’ll ever send. A slab-heavy single storey and a split-level with a skillion roof make different demands, and a single test only samples one of them.
The test also can’t speak for the trades the drawings don’t pin down. Where a job’s drawings are thin on a trade, the result is an estimate, and the blind test will show you an estimate — it can’t manufacture geometry that isn’t on the sheet. The honest answer there is that a specification tightens those trades up, and the test will show you exactly which ones want a specification before you price them.
None of that diminishes the test. It just frames it. A blind test is evidence, not a guarantee, and evidence is exactly what an accuracy percentage pretends to be and isn’t. If the question is “are you any good,” a single real job measured blind answers it better than any marketing number, provided you read the result for what it is.
Why builders test us before they hire us
The builders who run the test before committing are the ones making the most rational decision, because the test costs almost nothing and replaces a leap of faith. They already know the numbers on their build, so the comparison is instant and the conclusion is theirs, not ours. There’s no incentive in it for us to misrepresent — the job is finished, the numbers are fixed, and either we get close or we don’t.
That’s the whole logic of the estimating accuracy test done honestly: the burden of proof sits on the measurer, and the test puts us in front of it. If you’d rather verify before you spend a dollar, this is the cleanest way to do it. Pick a build you know, send the drawings, and hold back the costs.
Book a blind test → and see the variance on a job you already know, or start with the full mechanism on our method page.
This is Scaleline’s view. We measure residential jobs from drawings — we don’t price them, and we don’t sell software. Where we mention how we work, it’s to be useful, not to oversell.