AP Statistics Topic 5.5: Least-Squares Regression

01 · Start here

The concept in 60 seconds

The least-squares regression line is the line that makes the sum of squared residuals as small as possible for the observed data. Its usual prediction form is:

\hat{y}=a+bx.

The intercept $a$ gives the predicted response when $x=0$ . The slope $b$ tells you how much the predicted response changes for each additional unit of $x$ .

We square each vertical prediction error before adding. This prevents positive and negative residuals from canceling and gives larger misses more weight.

Best for a specific purpose: “Best” here means the smallest sum of squared residuals among straight lines. It does not guarantee that a line is an appropriate model or that every prediction is accurate.

Quick check: does the least-squares line have to pass through every point?

No. It balances the squared misses across the whole data set. When an intercept is included, it does pass through the mean point $(\bar{x},\bar{y})$ .

02 · Continue the delivery story

A real situation: choosing a delivery-time line

A delivery team records distance $x$ in kilometres and time $y$ in minutes for $9$ invented deliveries. These are the same teaching data used in Topics 5.3 and 5.4.

Paired observations: keep each distance with its own delivery time.
Delivery ID	Distance $x$ (km)	Observed time $y$ (min)
A	$5$	$22$
B	$7.5$	$24.5$
C	$10$	$29$
D	$12.5$	$35.5$
E	$15$	$39$
F	$17.5$	$44.5$
G	$20$	$47$
H	$22.5$	$51.5$
I	$25$	$58$

Technology gives the least-squares line:

\hat{y}=12+1.8x.

The means are $\bar{x}=15$ km and $\bar{y}=39$ minutes. The line predicts $12+1.8(15)=39$ minutes at the mean distance, so it passes through $(15,39)$ .

The fitted line passes through the mean point

Circles are observed deliveries. The line predicts time from distance. The highlighted mean point lies on the fitted line, even though it need not be an observed delivery.

These data were constructed to make the arithmetic easy; their exceptionally close fit is not typical of all real delivery data. Use the observed distance range, $5\leq x\leq25$ , when judging whether a prediction or interpretation is supported.

03 · Know what each symbol does

Key ideas and notation

Response and prediction

$y$ is an observed response; $\hat{y}$ is the response predicted at the same $x$ . The hat matters.

Intercept: $a$

The predicted response at $x=0$ . Its units are the response units.

Slope: $b$

The change in predicted response per unit increase in $x$ . Its units are response units per explanatory unit.

Residual: $e$

The signed vertical difference $e=y-\hat{y}$ . Least squares uses $e^2$ to compare lines.

Correlation: $r$

A unitless measure of the direction and strength of linear association. It is not the slope.

Coefficient of determination: $r^2$

The proportion of variation in the response explained by its linear relationship with the explanatory variable.

Watch the output convention: some calculators label slope $a$ and intercept $b$ in the form $y=ax+b$ . Others use $y=a+bx$ . Identify coefficients from the displayed equation rather than memorizing which letter is the slope.

04 · Why this line?

What least squares minimizes

For each observation, calculate the residual, square it, then add all squared residuals. We call this total the sum of squared errors, or $\mathrm{SSE}$ .

\mathrm{SSE}=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2.

The index $i$ simply labels an observation, and $n$ is the number of observations. For the delivery line, the residuals in minutes are:

(1,-1,-1,1,0,1,-1,-1,1).

There are $8$ residuals of magnitude $1$ and one of $0$ , giving $\mathrm{SSE}=8(1^2)+0^2=8$ square minutes. The signed residuals add to $0$ , but the squared errors do not.

Compare candidate lines using the same delivery data. Smaller squared-error total is better by the least-squares criterion.
Candidate equation	$\mathrm{SSE}$ (min $^2$ )	What changes?
$\hat{y}=12+1.8x$	$8$	Least-squares fit
$\hat{y}=13+1.8x$	$17$	Same slope, higher intercept
$\hat{y}=9+2x$	$23$	Steeper line through the same mean point

Same data, different squared-error totals

The fitted line has the smallest squared-error total. The other candidates illustrate why changing either the height or slope can increase the total. Squared-error units are square minutes.

Checking these candidates illustrates the rule; technology has found the minimum over all straight lines with an intercept. The line $\hat{y}=9+2x$ also passes through $(15,39)$ , so passing through the mean point alone does not prove a line is the least-squares line.

Vertical, not perpendicular: errors are measured in the response direction at the same explanatory value. The criterion is not shortest slanted distances, total absolute residuals, or the sum of signed residuals.

With an intercept and nonconstant explanatory values, the fitted line passes through $(\bar{x},\bar{y})$ and its residuals sum to $0$ , apart from rounding. Swapping $x$ and $y$ changes the prediction task and generally produces a different line.

05 · Turn coefficients into meaning

Interpret the slope and intercept

Slope: a change in predicted response

For $\hat{y}=12+1.8x$ , the slope is $b=1.8$ minutes per kilometre.

For each additional kilometre of delivery distance, the model predicts an increase of $1.8$ minutes in delivery time, on average.

This describes the change in the prediction. It does not mean every pair of deliveries differs by exactly that amount, or that distance is the only cause of travel time. Over an increase of $5$ km, the predicted change is $1.8(5)=9$ minutes.

Intercept: the prediction at zero

The intercept is $a=12$ minutes: the model predicts a delivery time of $12$ minutes at a distance of $0$ km.

However, $0$ km is outside the observed range $5\leq x\leq25$ . The intercept helps position the fitted line, but the data do not support interpreting it as an established loading time or a reliable zero-distance delivery time.

Meaningfulness check: Is $x=0$ sensible in the situation? Is it within or reasonably near the observed range? Is the predicted response logically possible? Explain any limitation instead of inventing a story for the intercept.

What if the slope is negative?

Say the predicted response decreases. For a hypothetical battery model $\hat{y}=98-6x$ , where $x$ is hours of use and $y$ is charge percentage, an additional hour predicts a decrease of $6$ percentage points in charge. This is a change in percentage points, not a relative decrease of $6\%$ .

06 · How much variation does the line explain?

Understand the coefficient of determination

In simple least-squares regression with an intercept, $r^2$ is the square of the correlation. It describes the proportion of variation in observed responses, about their mean, explained by the fitted linear relationship.

0\leq r^2\leq1.

Least-Squares Regression

By the end of this lesson, you should be able to:

The concept in 60 seconds

A real situation: choosing a delivery-time line

Key ideas and notation

Response and prediction

Intercept: $a$

Slope: $b$

Residual: $e$

Correlation: $r$

Coefficient of determination: $r^2$

What least squares minimizes

Interpret the slope and intercept

Slope: a change in predicted response

Intercept: the prediction at zero

Understand the coefficient of determination

Main Menu

Quick Links

The concept in 60 seconds

A real situation: choosing a delivery-time line

Key ideas and notation

Response and prediction

Intercept: aa

Slope: bb

Residual: ee

Correlation: rr

Coefficient of determination: r2r^2

What least squares minimizes

Interpret the slope and intercept

Slope: a change in predicted response

Intercept: the prediction at zero

Understand the coefficient of determination

Main Menu

Quick Links​

Intercept: $a$

Slope: $b$

Residual: $e$

Correlation: $r$

Coefficient of determination: $r^2$

Quick Links