StatisticsHARD4.5% of exam

Which action directly tests a fitted regression’s predictive performance beyond its training observations?

Correct Answer

B) Test predictions on observations not used in fitting

Testing on suitable observations excluded from fitting can assess predictive generalization and reveal overfitting. The validation design must avoid information leakage and reflect the intended use. Arithmetic, input verification, residual checks and applicability are also important, while high training R-squared alone does not establish predictive quality.

Answer Options

Recheck the arithmetic on the training observations

Test predictions on observations not used in fitting

Only confirm that each training sale was recorded

Require training R-squared to exceed an arbitrary threshold

Why This Is the Correct Answer

Option B is correct because evaluating predictions on appropriate observations not used in fitting shows how the model performs on data it has not seen. This kind of out-of-sample testing can assess predictive generalization and reveal overfitting. The validation design should avoid information leakage and reflect the model's intended use.

Why the Other Options Are Wrong

Option A: Recheck the arithmetic on the training observations

Option A, rechecking the arithmetic on the training observations, is a useful quality-control step, but it only confirms the calculations on data the model already saw. It does not test how the model predicts observations outside the training set.

Option C: Only confirm that each training sale was recorded

Option C, only confirming that each training sale was recorded, verifies data entry for the inputs. It says nothing about whether the fitted model predicts new observations well.

Option D: Require training R-squared to exceed an arbitrary threshold

Option D, requiring training R-squared to exceed an arbitrary threshold, measures fit on the training data only. A high training R-squared does not by itself establish predictive quality and can coexist with overfitting.

Test on data unseen

A student who memorizes the practice exam has not proven they can pass the real one. A model proves itself only on observations it did not train on.

How to use: Use this when a question asks how to judge predictive performance rather than in-sample fit.

Exam Tip

When a question asks about prediction beyond the training data, look for the answer that tests on observations excluded from fitting. Training-set statistics describe fit, not out-of-sample performance.

Common Mistakes to Avoid

-Treating a high training R-squared as proof that the model predicts well
-Testing the model on the same sales used to fit it
-Letting holdout data influence model choices, which leaks information into the test

Concept Deep Dive

Analysis

This question tests the difference between how well a regression fits the data it was built on and how well it predicts new cases. Fit statistics computed on the training data can look strong even when the model has memorized noise. The only action listed that directly measures predictive performance is checking predictions against suitable observations that were held out of the fitting process.

Background Knowledge

Overfitting occurs when a model captures random noise in its training data, producing strong in-sample fit but weak predictions for new data. Holdout testing, where some suitable observations are reserved and the model's predictions are compared against them, is a standard way to check generalization. Input verification, residual review and applicability checks remain important alongside validation.

Real-World Application

An appraiser builds a regression model from 150 neighborhood sales to support adjustments. Before relying on it, the appraiser sets aside a group of comparable recent sales not used in fitting and compares the model's predicted prices to their actual prices. Large prediction errors on that holdout group would signal that the model is overfit or not applicable.

regressionoverfittingholdout validationout-of-sample testingR-squared

Was this explanation helpful?

People Also Study

Real Estate Market

13.6% of exam

Property Description

11.8% of exam

Land or Site Valuation

4.5% of exam

Sales Comparison Approach

16.4% of exam

Cost Approach

13.6% of exam

Previous Question

What is the risk of applying a countywide price trend to a specific neighborhood?

Next Question

A data set consists of the values 10, 12 and 15. What is the harmonic mean of these values, to three decimals?

Statistics Questions Practice Interactively