Why Is Feature Scaling Important in KNN?

Blog › Machine Learning Explained

· by Julia Berens · 4 min read

If the same bike ride can be recorded in meters or kilometers, why is feature scaling important in KNN? Because k-nearest neighbors compares numbers, not units. A distance calculation does not know that a difference of 500 meters on one feature and 30 meters on another might have very different meanings for the prediction. Change the scale of one column and you can change which example comes first, even though the rides themselves stay the same.

Why is feature scaling important in KNN when units differ?

In k-nearest neighbors, or KNN, a feature is an input used to compare examples. Suppose you want to predict whether a bike route will be labeled easy or hard from two features: route length in meters and average grade in percentage points. You have a few labeled routes and one new route, called the query. With k = 1, the predicted label comes from the single closest labeled route.

One common way to measure closeness is Euclidean distance, the straight-line distance between two points on a graph. Subtract the two length values, subtract the two grade values, square both gaps, add them, and take the square root. If all you need is the ranking, you can skip the square root: the smallest squared distance also gives the smallest distance. For more on how the choice of distance fits into KNN, see the scikit-learn guide to nearest neighbors.

Raw distances favor the route with the closest length

Use this made-up dataset to isolate the effect of scale. The labels are given for the exercise, not meant as general ratings for real bike rides. Rides A, B, and C are the training examples. The query is not part of the training data.

RouteLength in metersAverage grade (%)Label
Ride A5,1005Hard
Ride B5,6002Easy
Ride C7,1006Hard
Query5,0002Unknown

Ride A is only 100 meters away in length, but its average grade differs by 3 percentage points. Ride B is 600 meters away in length and matches the query's grade. Squaring the gaps gives these results:

  • Ride A: 1002 + 32 = 10,009
  • Ride B: 6002 + 02 = 360,000
  • Ride C: 2,1002 + 42 = 4,410,016

Ride A wins and the model predicts Hard. Notice how little the grade gap contributes to A's total: just 9 out of 10,009. That does not show that grade is unimportant for predicting difficulty. It shows that a gap counted in meters is numerically much larger here than a gap counted in percentage points.

Scaling by the training ranges puts Ride B first

Min-max scaling subtracts the training minimum from each value in a column and divides the result by that column's training range. Here the length range is 7,100 − 5,100 = 2,000 meters. The grade range is 6 − 2 = 4 percentage points. After scaling, a difference of 0.25 on either feature means a quarter of that feature's training range.

Apply the same transformation to the query and the three training rides. The query's scaled length is (5,000 − 5,100) / 2,000 = −0.05, and its scaled grade is 0. A negative value is fine here: the query is shorter than the shortest training ride. The training minimum and maximum map to 0 and 1, but new values can fall outside that interval.

RouteScaled lengthScaled gradeSquared distance to query
Ride A00.750.052 + 0.752 = 0.565
Ride B0.2500.302 + 02 = 0.09
Ride C111.052 + 12 = 2.1025

Now Ride B is the closest, so the prediction changes to Easy. Ride A's length advantage is small on the scaled axis: its gap is 0.05 instead of B's 0.30. But A's grade gap is 0.75, while B has no grade gap at all. The routes and labels did not change. The calculation now measures each gap relative to the spread of its own feature.

Changing units is not the same as choosing a useful scale

Try one more calculation: express the raw lengths in kilometers and leave the grade values alone. The squared distance becomes 0.12 + 32 = 9.01 for Ride A and 0.62 + 02 = 0.36 for Ride B. Ride C's squared distance is 20.41, so B wins in this version too. But this is a unit conversion, not min-max scaling. The gap for each length is divided by 1,000 because that is how many meters are in a kilometer, not because of anything about the training data.

By contrast, if you compute the min-max distances using kilometers from the start, you get the same scaled distances as before. Both the length gap and the length range are divided by 1,000, so their ratio stays the same. You can check this with Ride B: 600 / 2,000 in meters is the same as 0.6 / 2 in kilometers.

Test the scale before trusting the prediction

Min-max scaling is a choice, not proof that two features deserve equal influence. One unusually long training ride could stretch the length range and make ordinary length differences look tiny. Another option is standardization, which subtracts the training mean and divides by the training standard deviation for a feature with nonzero spread. Compare choices on validation data rather than deciding that the one prediction you expected must be right. In each comparison, calculate the scaling values on the training split alone and apply them unchanged to the validation examples.

For a paper exercise, keep the grades and query fixed but move Ride C farther away in length. Recalculate the training length range and watch what happens to the scaled distances to A and B, even though neither ride moved. If you want more practice with model calculations, see the IMLC learning resources and training and practice problems.

Frequently asked questions

Does scaling always change the nearest neighbor?
No. Scaling can change the order, but if one example already has the smallest gap on every feature, dividing each feature by a positive scale will not make another example closer.

Do I scale the new example too?
Yes. Use the same training minimum and range, or the same training mean and standard deviation, that you used to scale the training examples.

Does this matter when k is greater than 1?
Yes. Scaling can change which examples make it into the group of k nearest neighbors, which can change the group's vote.

About the International Machine Learning Competition: an online machine learning competition for high school students from around the world. Qualification Round 2026: submissions are open until Sunday, 13 December 2026. How the rounds work

Written by Julia Berens, Social Media Assistant of the IMLC team.

More articles

All articles