iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

K-Means Clustering

By · Reviewed & last updated · Part of Machine Learning Tutorial

K-Means partitions n points into k clusters by alternating: assign each point to its nearest centroid, then recompute centroids. Fast, simple, and the first thing to try for unsupervised grouping.

sklearn KMeans + elbow + silhouette

EXAMPLE
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
import numpy as np
import matplotlib.pyplot as plt

# 1) Synthetic data
X, y_true = make_blobs(n_samples=500, centers=4, cluster_std=0.8, random_state=42)

# 2) ALWAYS scale before K-Means (distance-based)
X = StandardScaler().fit_transform(X)

# 3) Fit
km = KMeans(n_clusters=4, n_init=10, random_state=42)
km.fit(X)

labels    = km.labels_              # cluster id per point
centroids = km.cluster_centers_     # (k, n_features)
inertia   = km.inertia_             # sum of squared distances to centroid

# 4) Predict on new data
new_labels = km.predict(X[:5])

# 5) Pick K — Elbow method
inertias = []
ks = range(1, 11)
for k in ks:
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
    inertias.append(km.inertia_)

plt.plot(ks, inertias, 'o-')
plt.xlabel('k'); plt.ylabel('inertia')
plt.title('Elbow — pick the bend')

# 6) Pick K — Silhouette score (higher = better, max at 1.0)
for k in range(2, 8):
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
    score = silhouette_score(X, km.labels_)
    print(f'k={k}: silhouette={score:.3f}')

# 7) K-Means++ initialization (default in sklearn) — much better than random
KMeans(n_clusters=4, init='k-means++', n_init=10)

# 8) MiniBatchKMeans for large datasets
from sklearn.cluster import MiniBatchKMeans
mbk = MiniBatchKMeans(n_clusters=8, batch_size=1024, random_state=42)
mbk.fit(X)

# 9) Visualize 2-D clusters
plt.scatter(X[:, 0], X[:, 1], c=km.labels_, cmap='tab10', s=20)
plt.scatter(centroids[:, 0], centroids[:, 1], marker='X', s=200, c='red')

# 10) Real-world: customer segmentation
import pandas as pd
cust = pd.DataFrame({
    'recency_days':  np.random.exponential(30, 1000),
    'frequency':     np.random.poisson(5, 1000),
    'monetary_aud':  np.random.gamma(2, 100, 1000),
})

X = StandardScaler().fit_transform(cust)
km = KMeans(n_clusters=4, n_init=10, random_state=42).fit(X)
cust['segment'] = km.labels_

profile = cust.groupby('segment').mean()
print(profile)
# segment 0: 'whales'  — low recency, high frequency, high monetary
# segment 3: 'lapsed'  — high recency, low everything

# 11) When K-Means is the WRONG tool
#   • Non-spherical clusters → use DBSCAN, Gaussian Mixture
#   • Density varies a lot → DBSCAN, HDBSCAN
#   • Categorical features → K-Modes or one-hot + careful scaling
#   • Hierarchy matters → AgglomerativeClustering
#   • You need probabilities → GaussianMixture

# 12) Limitations and gotchas
#   • Sensitive to initialization → always n_init >= 10
#   • k must be chosen in advance
#   • Assumes equal-variance, isotropic clusters
#   • Outliers pull centroids → consider removing or using K-Medoids
#   • Different scales dominate distance → ALWAYS scale features

Why it matters

K-Means is your first stop for clustering, but it fails on non-spherical or density-varying data — reach for DBSCAN or HDBSCAN when blobs aren’t round. And never skip scaling: an unscaled income feature will swamp every other dimension.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
from sklearn.cluster import KMeans
km = KMeans(n_clusters=4, n_init='auto', random_state=0).fit(X)
print(km.labels_[:10])
print(km.cluster_centers_)
Try it Yourself »

Discussion

Loading…