Optimizing SVM Performance with GridSearchCV

Support Vector Machines (SVM) are commonly utilized for classification tasks, but obtaining optimal performance requires selecting the right hyperparameters, such as C and gamma. Identifying the best combination of these parameters can be challenging. GridSearchCV simplifies this process by automatically testing different hyperparameter combinations and choosing the best one based on cross-validation results.

Step 1: Importing Required Libraries

To build and evaluate the model, the libraries Pandas, NumPy, and Scikit-learn are used.

import pandas as pd
import numpy as np
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.datasets import load_breast_cancer
from sklearn.svm import SVC

Step 2: Loading and Displaying the Dataset

The Breast Cancer dataset from Scikit-learn is used in this example. It includes details about cell features and their corresponding cancer diagnosis—either malignant or benign.

cancer = load_breast_cancer()

df_feat = pd.DataFrame(cancer['data'], columns=cancer['feature_names'])
df_target = pd.DataFrame(cancer['target'], columns=['Cancer'])

print("Feature Variables: ")
print(df_feat.info())
print("Dataframe looks like: ")
print(df_feat.head())

Step 3: Splitting the Data

The dataset is divided into training (70%) and testing (30%) sets using train_test_split.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    df_feat, np.ravel(df_target),
    test_size=0.30, random_state=101)

Step 4: Training an SVM Model Without Tuning

Initially, a basic SVM classifier is trained without hyperparameter tuning.

model = SVC()
model.fit(X_train, y_train)

predictions = model.predict(X_test)
print(classification_report(y_test, predictions))

Despite achieving around 92% accuracy, the model's performance can be enhanced by tuning the hyperparameters.

Step 5: Hyperparameter Tuning with GridSearchCV

GridSearchCV is employed to determine the best combination of C, gamma, and kernel hyperparameters for the SVM model. Here's a brief explanation of these parameters:

• C: Balances between a wider margin (low C) and accurately classifying all points (high C).
• gamma: Influences the reach of data points, with high gamma leading to a tight boundary, potentially causing overfitting.
• kernel: The function used to transform data for class separation. The RBF kernel is used here for handling non-linear relationships.

from sklearn.model_selection import GridSearchCV

param_grid = {'C': [0.1, 1, 10, 100, 1000],
              'gamma': [1, 0.1, 0.01, 0.001, 0.0001],
              'kernel': ['rbf']}

grid = GridSearchCV(SVC(), param_grid, refit=True, verbose=3)

grid.fit(X_train, y_train)

Step 6: Obtaining the Best Hyperparameters and Model

Once grid search is complete, the best hyperparameters and the optimized model can be retrieved.

print(grid.best_params_)
print(grid.best_estimator_)

Step 7: Evaluating the Optimized Model

The optimized model's performance can be assessed using the test dataset.

grid_predictions = grid.predict(X_test)

print(classification_report(y_test, grid_predictions))

Following hyperparameter tuning, the model's accuracy increased to 94%, demonstrating improved performance. This approach enhances model accuracy and reliability.