Difference between revisions of "Python-for-Machine-Learning/C2/K-Nearest-Neighbor-Classification/English"

Welcome and Title Slide

Learning Objectives

• The fundamentals of KNN Algorithm
• Implementing KNN for Classification using Iris dataset
• Evaluating the performance of the trained model

System Requirements

• Ubuntu Linux OS version 24.04
• Jupyter Notebook IDE

Pre- requisites

• The learner must have basic knowledge of Python.
• For pre-requisite Python tutorials, please visit this website.

Code Files

• The files used in this tutorial are provided in the Code files link.
• Please download and extract the files.
• Make a copy and then use them while practicing.

KNN

• KNN stands for K Nearest Neighbors.
• Nearest Neighbor algorithm predicts using closest similar training data points.
• K indicates the number of neighboring points to be considered for prediction.

KNN Classification

• Features of K nearest samples are compared to determine similarity.
• The new data point gets the most frequent class among its neighbors.
• KNN is versatile and can effectively handle multi-class classification problems.

Iris Dataset irisflowers.png

Hover over setosa, versicolor and virginica images

Hover over sepal length, sepal width, petal length and petal width

It has 3 distinct flower classes.

Classes are classified by sepal length, sepal width, petal length, petal width.

Iris Dataset iris.png

We will use the four features of the flower to classify the three distinct classes.

A black dot represents a flower in the dataset that lacks a defined class.

Our goal is to predict the black dot’s class based on its nearest neighbors.

The closest points to the black dot are called its neighbors.

These neighbors determine the black dot’s class.

Point to KNN classification.ipynb

KNN classification dot ipynb is the python notebook file for this demonstration.

Type conda activate ml

Press Enter

Activate the machine learning environment by typing

conda space activate space ml

Press Enter.

Press Enter Type jupyter notebook Press Enter

Please navigate to the respective folder of your code file location.

Type, jupyter space notebook and press Enter to open Jupyter Notebook.

Click on KNNClassification.ipynb

Locate the KNN classification dot ipynb file.

Open the file by clicking on it.

Note that each cell will have the output displayed in this file.

Let us see the implementation of the KNN classification model.

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

Please press Shift plus Enter to execute the code in each cell.

iris = load_iris()

The iris dataset is a built-in dataset available in the Scikit- learn library.

Highlight iris.feature_names

Setosa is shown as 0, versicolor as 1 and virginica as 2.

We load the Iris dataset to the dataframe.

It holds the features and target class.

Highlight output

The default is 5 rows, but the value can be changed by specifying the argument.

iris_df.head()

Highlight

sns.pairplot( iris_df, hue='target', diag_kind='kde', palette='colorblind' ) plt.show()

It visualizes relationships between different features.

It creates scatterplots for each pair of features, colored by class labels.

plt dot show is used to display the generated plots.

Scatter plots compare two features and help to identify patterns.

Diagonal KDE plots show the distribution of each feature for different classes.

Different colors represent different target classes.

Clusters indicate which species are overlapping.

Highlight X = iris_df.drop('target', axis=1)

First we remove the target column named target.

Then copy the remaining features into the variable X.

Highlight the output

It is the species of the iris flower.

Highlight train_test_split(X, y, test_size=0.4, random_state=42)

We use the train underscore test underscore split method.

The split ratio is adjustable through the test underscore size parameter.

We set the test underscore size as 0.4.

Here, we use 40 percent of the data for testing and 60 percent for training.

Setting random state equal to 42 ensures the split is reproducible.

It guarantees we get the same result across multiple executions.

X underscore train contains the features of the training data.

It is used for model training.

y underscore train contains the target values for the training data.

X underscore test contains features of the test data.

It is used for making predictions.

y underscore test contains the actual class labels for the test data.

It is used for evaluating the model performance.

Fit method adjusts the model parameters using the training data.

print("Training Accuracy: {training_accuracy:.3f}")

Accuracy is the ratio of correct prediction to the total number of instances.

It helps to measure how well the model is performing.

Training Accuracy: 0.956

print("\nClassification Report:")

print(classification_report(y_train, y_train_pred))

The classification report helps to evaluate how well the model is performing.

Precision tells how many positive predictions made by the model were correct.

F1 Score is the balance between precision and recall.

Recall shows how well the model detects actual positive cases correctly.

Support is the count of true instances of each class in the dataset.

Box for the data

F1-Score shows good overall performance across all classes.

The accuracy is 96 percent.

This means the model made correct predictions for 96 percent of the instances.

macro and weighted average reflect consistent performances across the dataset.

We use the predict method to predict the class label using the trained model.

We store the actual class label in the actual underscore class variable.

Then, we print the predicted and actual class labels of the data sample.

The actual class is also 2, indicating the prediction is correct.

print(f"Testing Accuracy: {accuracy:.3f}")

We can conclude that the model is performing well on unseen data.

Highlight y_test_bin = label_binarize(y_test, classes=[0, 1, 2])

It calculates precision and recall.

It computes average precision and summarizes the precision recall curve into a single score.

plt.ylabel("Precision")

plt.title("Precision-Recall

plt.show()

KNN classifier achieved perfect precision-recall for all classes.

High AP indicates the model performs well in distinguishing all three classes.

print(classification_report(y_test, y_pred))

The report offers a detailed assessment of the model’s performance.

Summary

Assignment

• Use K as 7 and test size as 0.2.
• Evaluate the model performance using the classification report.

Assignment Solution

K=7, train_test_split = 0.2

We will get an accuracy of 96 percent.

FOSSEE Forum

Thank You

Thanks for joining.