Difference between revisions of "Python-for-Machine-Learning/C3/Support-Vector-Machine/English"

Welcome

Learning Objectives

• Support Vector Machine (SVM)
• Linear SVM and
• Non Linear SVM

System Requirements

• Ubuntu Linux OS version 24.04
• Jupyter Notebook IDE

Prerequisite

• The learner must have basic knowledge of Python.
• For prerequisite Python tutorials, please visit this website.

Code files

• The files used in this tutorial are provided in the Code files link.
• Please download and extract the files.
• Make a copy and then use them while practicing.

SVM

• SVM is a supervised learning algorithm used for classification and regression.
• It finds the best boundary, called a hyperplane, to separate classes.

Hyperplane and Margin

Show margin.png

Narration

• The best hyperplane is the one that leaves the largest gap between classes.
• This gap is called the margin, and a larger margin reduces errors.

Linear SVM

• If a straight line hyperplane can separate the data, we use Linear SVM.
• Linear SVM aims to find the hyperplane that maximizes the margin.

Non-Linear SVM

• When data is not linearly separable, we use Non Linear SVM.
• Non Linear SVM uses the kernel trick to transform the data.
• Kernels help find decision boundaries for data that isn’t linearly separable.

The columns in the dataset helps to classify whether a house price is High or Low.

Type conda activate ml

Press Enter

Activate the machine learning environment as shown

Type cd Downloads

Type jupyter notebook

Press Enter

Please navigate to the directory of your respective code file location.

Then type, jupyter space notebook and press Enter.

Click on SVM.ipynb

Click the SVM dot ipynb file to open it.

Note that each cell will have the output displayed in this file.

import pandas as pd import seaborn as sns from sklearn.decomposition import PCA

Press Shift and Enter

Now, we will implement a Linear SVM model.

Make sure to Press Shift and Enter to execute the code in each cell.

housing_df = pd.read_csv('californiahousing.csv')

housing_df.head()

housing_df.shape

selected_features = ["MedInc", "HouseAge", "AveRooms", "AveBedrms", "Housing Price"]

X

y

X_train, X_test, y_train, y_test =

scaler = MinMaxScaler() X_train_scaled =

To set up a Linear SVM, we use the Linear kernel.

y_train_pred_linear = svc_linear.predict(X_train_scaled)

y_pred_linear =

Accuracy: 0.840

train_sizes, train_scores, test_scores = learning_curve

Hover over training accuracy line and validation accuracy line in the plot.

The blue and red lines show training and validation accuracy respectively.

The learning curve helps to analyze model performance before further tuning.

svc_rbf = SVC(kernel='rbf', C=10,

We set the regularization parameter C to 10 for better separation.

We also use class weighting to handle class imbalance.

y_train_pred_rbf = svc_rbf.predict(X_train_scaled)

y_pred_rbf = svc_rbf.predict(X_test_scaled)

Accuracy: 0.840

pca = PCA(n_components=2) X_train_pca = pca.fit_transform(X_train_scaled) X_test_pca = pca.transform(X_test_scaled)

We now plot the support vectors, which help define the decision boundary.

Each point represents a data sample from the dataset.Red and blue colors indicate two different target classes.Black X marks represent the model's support vectors.Support vectors are the key points defining the decision boundary.

Thus, this is a 2D visualization of an originally 9D dataset.

Summary

• Change the value of C to 5 as shown here
• Observe the change in accuracy.

Assignment Solution

Show Linear.PNG image file

FOSSEE Forum

Thank you

Thanks for joining.