Clustering Categorical Data with K-Modes in Python

K-mode clustering is an unsupervised machine learning technique designed to group categorical data into distinct clusters. Unlike K-means, which measures similarity using numerical distances, K-modes uses the number of category mismatches to decide similarity between data points.

Example of K-Modes Clustering

Consider the following data points:

• Data point 1: ["red", "small", "round"]
• Data point 2: ["blue", "small", "square"]

These two points have two mismatches (color and shape), indicating they are not very similar.

When to Use K-Modes

K-Modes is particularly useful in scenarios where:

• The dataset comprises categorical variables such as gender, color, or brand.
• There is a need to group customers by product preferences.
• Analyzing survey responses with options like Yes/No or Male/Female is required.

How K-Modes Works

K-Modes requires specifying the number of clusters (K) beforehand. The process involves:

1. Initialization: Randomly select K data points from the dataset to serve as initial cluster centers, known as "modes."
2. Assign Data to Clusters: Compare each data point to these modes based on mismatches and assign it to the most similar cluster.
3. Update Clusters: Determine the most common category values in each cluster and update the modes accordingly.
4. Iterate: Repeat the assignment and update steps until no changes occur in cluster assignments.

The goal is to minimize dissimilarities between data objects and cluster centers, using measures such as the Hamming distance.

Implementing K-Modes in Python

Here's a step-by-step guide to implementing K-Modes using Python with NumPy and Pandas.

Step 1: Prepare Your Data

import numpy as np
import pandas as pd

data = np.array([
    ['A', 'B', 'C'],
    ['B', 'C', 'A'],
    ['C', 'A', 'B'],
    ['A', 'C', 'B'],
    ['A', 'A', 'B']
])

Step 2: Set Number of Clusters

k = 2

Step 3: Pick Starting Points (Modes)

np.random.seed(0)
modes = data[np.random.choice(data.shape[0], k, replace=False)]

Step 4: Assign Data to Clusters

clusters = np.zeros(data.shape[0], dtype=int)

for _ in range(10):  
    for i, point in enumerate(data):
        distances = [np.sum(point != mode) for mode in modes]
        clusters[i] = np.argmin(distances)

Step 5: Update Cluster Modes

for j in range(k):
    if np.any(clusters == j):
        modes[j] = pd.DataFrame(data[clusters == j]).mode().iloc[0].values

Step 6: View Final Results

print("Cluster assignments:", clusters)
print("Cluster modes:", modes)

Cluster with kmodes Library

To find the optimal number of clusters, the Elbow method can be used.

from kmodes.kmodes import KModes
import matplotlib.pyplot as plt

cost = []
K = range(1, 5)
for k in K:
    kmode = KModes(n_clusters=k, init="random", n_init=5, verbose=1)
    kmode.fit_predict(data)
    cost.append(kmode.cost_)

plt.plot(K, cost, 'x-')
plt.xlabel('No. of clusters')
plt.ylabel('Cost')
plt.title('Elbow Curve')
plt.show()

The Elbow method helps determine the optimal number of clusters by identifying points where adding more clusters doesn't significantly reduce cost.

Conclusion

K-Modes is effective for clustering categorical data by minimizing dissimilarities between data points and cluster modes. This approach is ideal for datasets with categorical attributes.