Why does adding more variables to a regression not necessarily improve it?
Correct Answer
D) The model may fit noise specific to the training sample
Additional predictors may fit sample-specific noise without improving predictions on new relevant observations. For nested intercept OLS models on the same data, raw R-squared cannot decrease, but can remain unchanged. Adjusted R-squared penalizes added parameters; independent validation is important for predictive performance. The response need not change when a predictor is added.
Why This Is the Correct Answer
Option D is correct because additional predictors may fit noise specific to the training sample without improving predictions for new, relevant observations. A higher in-sample R-squared does not by itself mean the model fits well or predicts well. Adjusted R-squared and independent validation help detect this problem.
Why the Other Options Are Wrong
Option A: Raw R-squared must fall whenever another predictor is added
Option A claims raw R-squared must fall whenever another predictor is added, which reverses the actual behavior. For nested ordinary least squares models with an intercept on the same data, raw R-squared cannot decrease, though it can stay the same.
Option B: Each added predictor improves predictions for new cases
Option B claims each added predictor improves predictions for new cases, which is the opposite of the overfitting problem. Extra predictors can fit sample noise and make out-of-sample predictions worse.
Option C: Adding a predictor changes which variable is the response
Option C says adding a predictor changes which variable is the response, but the response variable is chosen by the analyst and does not change when a predictor is added. Adding predictors changes only the right-hand side of the model.
More Variables, More Noise Risk
Each extra predictor gives the model another chance to memorize sample noise, much like a student memorizing one practice test instead of learning the material.
How to use: Use this image when a question asks whether adding variables automatically improves a regression.
Exam Tip
Remember that raw R-squared cannot decrease when nested predictors are added, so a rising R-squared alone does not prove a better model.
Common Mistakes to Avoid
- -Believing raw R-squared falls when a predictor is added
- -Equating higher in-sample fit with better predictions
- -Ignoring adjusted R-squared or validation on new data
Concept Deep Dive
Analysis
This question tests the concept of overfitting. Adding predictors can improve fit to the sample used for estimation while capturing random noise that does not carry over to new cases. Recognizing that in-sample fit and predictive accuracy are different leads to the answer.
Background Knowledge
Overfitting occurs when a model becomes complex enough to describe random features of the estimation sample. Raw R-squared does not penalize added parameters, which is why adjusted R-squared and validation on independent data are used to judge whether added variables genuinely improve the model.
Real-World Application
An appraiser adds a dozen minor features to a regression of 60 local sales and sees R-squared climb, yet predictions for recent sales held out of the model get worse. The added variables were fitting quirks of the original sample.
More Statistics Questions
Sale prices are $270,000, $290,000, $300,000, $310,000, $330,000. Treat the listed observations as the complete population and divide by n when computing the standard deviation. Report to the nearest dollar.
A sales sampling frame systematically excludes a relevant part of the target market. What is the principal risk?
An appraiser estimates a market-wide price trend using only sales data from one brokerage that specializes in luxury homes, without checking how other segments are represented. What is the principal concern?
Compared with analyzing one matched sale pair for one feature, what can a suitably specified multiple regression do?
The base-period median sale price is $400,000 and the current-period median is $460,000. Compute the price relative defined as current ÷ base, as a decimal to three places.
The assessment-to-sale ratios are 0.8, 0.85, 0.9, 0.92, 0.95, 0.98, 1.02, 1.05, 1.08, 1.1, 1.15, 1.2. Calculate COD as [average absolute deviation from the median ÷ median]×100. For even n, average the two middle ratios. Report the COD as a percentage to two decimals.
Assessment-to-sale ratios are 0.94, 0.96, 0.98, 1, 1.01. Compute the arithmetic mean. Report as a decimal ratio to three places.
Market data show a first quartile sale price of $405,000 and a third quartile sale price of $495,000. What is the interquartile range, to the nearest dollar?
Assessment-to-sale ratios are 0.96, 0.98, 0.99, 1.01. Compute the arithmetic mean. Report as a decimal ratio to three places.
Given first quartile $400,000, median $440,000 and third quartile $500,000, compute the trimean (Q1+2×median+Q3)/4. Report to the nearest dollar.
People Also Study
Real Estate Market
13.6% of exam
Property Description
11.8% of exam
Land or Site Valuation
4.5% of exam
Sales Comparison Approach
16.4% of exam
Cost Approach
13.6% of exam
Previous Question
An appraiser is summarizing ten verified sales of similar homes, which sold for $210,000, $240,000, $250,000, $260,000, $270,000, $280,000, $290,000, $300,000, $400,000 and $640,000. One colleague calculates a trimmed mean and another a winsorized mean, each treating one observation in each tail. What are the correct results?
Next Question
How should an appraiser handle numerical precision when applying a statistically estimated adjustment?