Introduction

Written by:

Ylva B Almquist

Linear regression is used when y is continuous (ratio/interval; see Measurement scales).

A linear regression model generally has the aim to predict or “forecast” the value of y, based on the values of one or more x-variables. Linear regression is concerned with finding the best-fitting straight line through the data points.

The regression line has an intercept (or constant) and a slope. The intercept is where the regression line strikes the y-axis when the value of the x-variable(s) is 0. The slope is basically the steepness of the line; i.e. how much y changes when x increases.

The regression model thus gives us predicted values of y across the values of the x-variable(s). Of course, there is generally a difference between what the model predicts and what the individuals’ actual (observed) values are. This difference is called residual and is calculated as the observed value minus the predicted value.

Often, the term error is used instead of residual, and although these terms are closely related, they are not the exact same thing: an error is the difference between the observed value and the population mean (and the population mean is typically unobservable), whereas a residual is the difference between the observed value and the sample mean (and the sample mean is observable).  

The most common method for fitting the linear equation is the method of ordinary least squares (OLS). It minimises the sum of squared differences between the observed and predicted values.

Just a little bit of maths

We promised to not have (almost) any equations in this guide, but here is a very simple expression of the one for linear regression:

y=a+bx+e

  • y (or rather y hat; ŷ) is the predicted value of y.
  • a is the intercept (or constant), i.e. the value of y when x=0.
  • b is the slope (steepness) of the regression line, i.e. how much y changes per unit increase in x.
  • x is the value of x.
  • e is the error term (or residual), i.e. the error in predicting the value of y given the value of x.

Other names for linear regression

Linear regression is often referred to as OLS regression.