Sampling & Reliability
~10 min read · Judge sample size, outliers and significance before trusting a statistic.
Every comp set is a sample standing in for a market. Sampling logic — representativeness, size, bias — plus reliability vocabulary (confidence, significance) tells you how far to trust what the sample says. The exam tests the concepts more than the formulas.
Samples and representativeness
The population is every relevant transaction; the appraiser works with samples — and rarely random ones: comps are SELECTED, which is fine (selection by competitiveness is the method) so long as the selection isn't biased toward a desired answer. Representativeness beats size: five true competitors outweigh thirty scattered sales. Named biases: selection bias (cherry-picking high sales), survivorship/availability bias (only MLS-recorded deals), and time-window bias (sampling only the hot quarter).
- Comps = purposive sample of the competitive market
- Representativeness first, count second
- Cherry-picking is bias with a workfile
Size and stability
Larger samples stabilize estimates — the standard error of a mean shrinks with √n — but real markets cap n; the practical rules: more data for heterogeneous markets, fewer needed for tract uniformity; small samples make outliers deadly (one bad sale in five is 20% of the evidence); and conclusions from thin data get stated with humility (wider ranges, more corroboration).
- Precision improves with √n — diminishing returns
- Heterogeneous markets demand more data
- Small n = every observation is load-bearing
Confidence and significance
A confidence interval brackets where the population value plausibly lies (95% convention); statistical significance asks whether an observed effect (the corner-lot premium) plausibly exceeds chance. For appraisal reliability: agreement among independent indications (multiple pairs, multiple approaches) is the working substitute for formal inference — triangulation is the field's confidence interval. Report language should match the evidence's strength: 'supported by three matched pairs' vs 'indicated by limited data.'
Worked example
An appraiser 'supports' a $25,000 view adjustment with one matched pair, selected from a lakefront market where nine other pairs were available — most suggesting $12,000–$16,000. A reviewer challenges the support. Diagnose with sampling vocabulary.
The defect is selection bias: from ten available pairs, the one supporting the largest adjustment was chosen — a sample of one, unrepresentative by construction. Sample-size logic compounds it: n=1 has no stability; the single pair's difference could be noise (that buyer overpaid) as easily as signal. The honest procedure: extract ALL available pairs, examine the distribution (center ~$14,000, spread modest, the $25,000 pair an outlier to investigate), and conclude an adjustment near the cluster with the outlier explained. Reliability in appraisal is the nine-pair answer — representative extraction, stated range, triangulated against any rent or resale evidence. One pair wasn't support; it was a choice.
Common exam pitfalls
Supporting adjustments with the friendliest single pair.
Extract everything available; the distribution — not the favorite observation — is the support.
Equating more data with better data.
Thirty unrepresentative sales lose to five true competitors — relevance gates before count.
Overstating thin evidence.
Match report language to sample strength; small-n conclusions carry ranges and corroboration.
Represent first, count second, doubt the single data point — and let independent evidence agree before you do.
Recap
- Comps are purposive samples; selection must be competitive, not convenient
- Bias catalog: selection, availability, time-window
- Precision grows with √n; heterogeneity demands more
- Confidence and significance frame chance vs signal
- Triangulation = practical reliability
- Report language mirrors evidence strength
Prove it: 10 questions on this topic
Every lesson ends with a ten-question check in the free course — your progress syncs between the web and the EstatePass app.
Studying for the appraiser licensing exam? Track every lesson free — progress syncs with the app.
Start freeMore in Statistical Methods
Study smarter in the free dashboard
- Every lesson tracked, synced with the iOS app
- Ten-question checks after each lesson
- Lesson videos, flashcards and mock exams
No credit card required.