OgbonLab

Chapter 7 Quiz: K-Nearest Neighbors

Chapter 5: Distance-Based Learning — K-Nearest Neighbors

Learning objectives

Test your understanding of distance metrics, K selection, feature scaling, and KNN implementation

This quiz covers both the lecture material and lab exercises from Chapter 7.

Key Concepts Review

KNN: Store training data. Find $K$ nearest neighbors and vote (classification) or average (regression).
Distances: Euclidean $d = \sqrt{\sum(x_i - y_i)^2}$ , Manhattan $d = \sum|x_i - y_i|$ .
Feature Scaling: Essential for KNN. Use StandardScaler or MinMaxScaler.
K Selection: $K=1$ overfits; $K=N$ underfits. Cross-validation selects optimal $K$ .
Non-parametric: Stores entire training set. Slow prediction for large data.

References

Hastie, T., Tibshirani, R., Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.), ch. 13.3. Springer.
Murphy, K.P. (2022). Probabilistic Machine Learning: An Introduction, ch. 16.1. MIT Press.
James, G., Witten, D., Hastie, T., Tibshirani, R. (2021). An Introduction to Statistical Learning (2nd ed.), ch. 3 & 4. Springer.