Line of Best Fit in Regression Analysis: Method of Least Squares

Picture a professor plotting the marks of forty students against the number of hours they studied. The dots scatter all over the graph, but a rough pattern is visible: more hours generally means higher marks. The question is, how do you draw a single straight line through that mess of points that best represents the trend? This is exactly the problem the line of best fit solves, and the technique behind it, called the method of least squares, is one of the most widely used tools in statistics and data analysis.

What is a line of best fit? 🔗

When you plot two related variables on a scatter diagram, such as advertising spend and sales, or rainfall and crop yield, the points rarely fall in a perfectly straight line. There is almost always some scatter. A line of best fit is the single straight line that comes closest to representing the overall relationship between the two variables, even though it may not pass through every point exactly.

The gap between an actual data point and the line at any given value of X is called a deviation, or sometimes a residual or error. Some points sit above the line, some below it, and the whole exercise of “fitting” a line is really about deciding where to place it so these gaps are as small as possible overall.

The method of least squares explained 🔗

There are infinite straight lines you could draw through a scatter plot. The method of least squares gives you a precise mathematical rule for choosing the one best line: it is the line for which the sum of the squares of the vertical deviations from each point to the line is at its lowest possible value. This is why the line is also called the least squares regression line.

The logic is straightforward once you break it down. For every data point, you measure how far it is from the candidate line (vertically, along the Y-axis). You square that distance, so that positive and negative gaps don’t cancel each other out, and negative values don’t understate the error. Then you add up all these squared distances for every single point. The line that produces the smallest possible total is declared the line of best fit.

Why square the deviations instead of just adding them? 🔗

If you simply added up the raw deviations without squaring them, positive and negative deviations would offset one another, and you could end up with a total close to zero even for a badly fitting line. Squaring solves this problem in two ways. It makes every deviation positive, so they cannot cancel out, and it penalises larger errors more heavily than smaller ones. A point that is very far from the line contributes disproportionately more to the total, which pushes the line to avoid large errors wherever possible. This is confirmed by multiple statistical explanations of the technique, which describe least squares as minimising the total squared error between observed and predicted values.

Once the best line is found, it is usually expressed in the standard regression equation form, Y = a + bX, where a is the intercept (the value of Y when X is zero) and b is the slope. The slope is calculated using the covariance between the two variables and the variance of X, and this pair of values, once calculated, fully defines the straight line that best represents your data, as standard explanations of the least squares equation lay out step by step.

Properties of the least squares regression line 🔗

The least squares line isn’t just an arbitrary “best guess.” It satisfies two mathematical properties that make it genuinely special among all possible lines you could draw.

The sum of deviations is always zero 🔗

If you take the deviation of each actual Y value from its corresponding predicted value on the regression line (written as Y minus Yₑ) and add up all these deviations across every data point, the total will always equal exactly zero. This happens because the line is anchored through the mean of both variables, so the positive deviations above the line and the negative deviations below it balance out perfectly. This is a defining algebraic feature of the least squares fit, not a coincidence for any particular dataset.

The sum of squared deviations is the smallest possible 🔗

This is the property that gives the method its name. Among every conceivable straight line you could draw through the same set of points, the least squares line produces a smaller sum of squared deviations than any other line. In other words, if you tried any alternative line, its total squared error would always be equal to or greater than that of the least squares line. This is precisely what makes it the “best” fit in a mathematical sense, and it’s why the least squares approach remains the standard method used across regression and data fitting problems in statistics, economics, and the sciences.

Reading the direction of the relationship 🔗

Once you have your regression line, its slope tells you something important about the relationship between the two variables, and this is closely tied to the correlation coefficient (r), which measures both the strength and direction of a linear relationship.

Positive correlation and an upward-sloping line 🔗

When r is positive, the regression line slopes upward from left to right. As the value of X increases, Y also tends to increase. Study hours and exam marks, or advertising spend and sales figures, typically show this pattern. The steeper the upward slope, the more strongly one variable rises as the other does.

Negative correlation and a downward-sloping line 🔗

When r is negative, the line slopes downward from left to right, meaning that as X increases, Y tends to decrease. A classic example is the relationship between the price of a product and the quantity demanded. As price rises, demand typically falls, producing a downward-sloping trend line.

It’s worth noting that the sign of r determines the direction of the slope, but the actual steepness of the slope also depends on how spread out the X and Y values are relative to each other, a relationship explored in detail by university-level explanations of correlation and regression.

How well does the line actually fit? 🔗

Not every regression line is equally trustworthy. The closer the actual data points cluster around the fitted line, the more confidence you can have in it as a predictive tool. If the points are scattered widely, even if a best fit line exists, predictions based on it will carry more uncertainty. This is generally where the correlation coefficient becomes useful again. A value close to +1 or -1 indicates the points sit tightly around the line, while a value near zero suggests a weak or negligible linear relationship, in which case the line of best fit may not be a reliable model at all.

Why this matters beyond the classroom 🔗

The line of best fit isn’t just an exam topic. It underpins a huge range of real decisions. In agriculture, researchers regularly use regression models to study how rainfall, area under cultivation, and other environmental factors affect crop yield, helping predict harvests and manage food supply more effectively, as demonstrated in research analysing crop yield through regression techniques. Similar approaches are used to forecast yields using historical weather and production data, a method described in detail in studies on crop yield forecasting that rely on linear regression as a core tool.

Businesses use the same principle to forecast sales based on marketing spend, economists use it to study the relationship between income and consumption, and even sports analysts use it to track player performance trends over a season. Anywhere you have two related numerical variables and want to understand or predict how one moves in response to the other, the least squares regression line is usually the starting point.

A quick recap 🔗

The line of best fit is the straight line that most accurately represents the relationship between two variables on a scatter plot. It is calculated using the method of least squares, which minimises the sum of the squared vertical deviations between the actual data points and the line. This gives the line two defining properties: the deviations from it always sum to zero, and no other line produces a smaller total of squared deviations. The slope of this line, combined with the sign of the correlation coefficient, tells you whether the relationship between the variables is positive or negative, and how strong that relationship really is.

What do you think? If you plotted your own study hours against your test scores over the last semester, do you think the line would slope upward as steeply as you’d expect? And in a field like agriculture, where rainfall depends on so many unpredictable factors, how reliable do you think a simple straight-line model can really be?

References

1. https://www.geeksforgeeks.org/maths/least-square-method/
2. https://www.datacamp.com/tutorial/least-squares-method
3. https://www.cuemath.com/data/least-squares/
4. https://www.monash.edu/student-academic-success/mathematics/linear-regression-and-linear-relations/correlation-and-least-squares-regression-line
5. https://indjst.org/articles/prediction-of-crop-yield-using-regression-analysis
6. https://link.springer.com/article/10.1007/s40003-019-00413-x