Difference between revisions of "Python-for-Machine-Learning/C3/Decision-Tree/English"

Welcome

Learning Objectives

• Decision Tree
• Decision Tree Structure and Nodes

System Requirements

• Ubuntu Linux OS 24.04
• Jupyter Notebook IDE

Prerequisite

To follow this tutorial,

• The learner must have basic knowledge of Python.
• For prerequisite Python tutorials, please visit this website.

Code files

• The files used in this tutorial are provided in the Code files link.
• Please download and extract the files.
• Make a copy and then use them while practicing.

Decision Tree

• A decision tree is a tool used in machine learning that helps make decisions.
• It uses a tree like structure to make predictions.

Working of Decision Tree

Show dt.png img

• The root node starts the decision tree with a question or condition.
• Based on the answer, we follow a branch to another node.
• A branch connects nodes, where each node represents a condition with outcomes.

• This new node poses another question or condition.
• We repeat this process of asking questions and following branches.
• Finally, we reach a leaf node, which gives us the final decision or outcome.

Here, we analyze patient’s data to predict the most suitable drug for them.

drug200 dataset contains Age, Sex, BP, Cholesterol, Na to K ratio and Drug.

Type conda activate ml

Press Enter

Activate the machine learning environment by typing conda space activate space ml Press Enter.

Type cd Downloads Press Enter Type jupyter notebook

Press Enter

Please navigate to the respective folder of your code file location.

Type, jupyter space notebook and press Enter to open Jupyter Notebook.

Click on DecisionTree.ipynb file

Click the DecisionTree dot ipynb file to open it.

Note that each cell will have the output displayed in this file.

import numpy as np import pandas as pd from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeClassifier Press Shift+Enter

Please remember to Execute each cell by pressing Shift and Enter to get output.

df_drug=pd.read_csv("drug200.csv") df_drug.head() Press Shift+Enter

print("Length of Dataset:", len(df_drug)) print("Dataset Shape:", df_drug.shape) Press Shift+Enter

le_sex = LabelEncoder() le_BP = LabelEncoder() Press Shift+Enter

x=df_drug.drop(columns=['Drug']).values y = df_drug['Drug'].values Press Shift+Enter

x

y

print("\nNumber of Duplicate Rows:", df_drug.duplicated().sum()) df = df_drug.drop_duplicates() print("Dataset Shape After Removing Duplicates:", df.shape)

We also print the dataset shape after removing the duplicates.

numerical_columns = df.select_dtypes(include=['int64', 'float64']).columns scaler = StandardScaler()

df.hist(figsize=(10, 8), bins=20, color='skyblue', edgecolor='black')

plt.suptitle("Distribution of Numerical Features")

plt.show()

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=100)

Press Shift+Enter

clf_entropy = DecisionTreeClassifier(criterion="entropy", random_state=5, max_depth=4, min_samples_leaf=5)

Press Shift+Enter

Entropy is the measure of disorder in the dataset. It helps to classify the features into root and branches of the decision tree.

By navigating through the root and branches, we arrive at a decision of classes.

clf_entropy.fit(x_train, y_train) y_pred_train = clf_entropy.predict(x_train)

Once trained, we predict the target values for the training set.

print("Training Accuracy is", accuracy_score(y_train, y_pred_train) * 100 )

y_pred_en

accuracy = round(accuracy_score(y_test, y_pred_en) * 100, 3)

print("Accuracy is", accuracy)

Press Shift+Enter

cm = confusion_matrix(y_test, y_pred_en) plt.figure(figsize=(10, 7)) Press Shift+Enter

It shows how well the model is correctly classifying the instances.

report = classification_report(y_test, y_pred_en,zero_division=0)

print("Classification Report:")

Press Shift+Enter

This report gives precision, recall, f1-score, and support for each class.

classes = np.unique(y) y_test_bin = label_binarize(y_test, classes=classes) y_score = clf_entropy.predict_proba(x_test)

Next, we binarize y underscore test for multi class ROC plotting.

We binarize to handle multi class ROC as it needs binary format.

Then, we predict class probabilities using the model.

fpr = dict() tpr = dict() roc_auc = dict() n_classes = len(classes) for i in range(n_classes): plt.legend(loc="lower right") plt.grid(True) plt.show()

The ROC curve shows how well the model distinguishes between classes.

Let us see about the AUC score that is area under the curve.

DrugA, DrugB, DrugC have an AUC score of 1, indicating perfect classification.

DrugX and DrugY have AUC scores of 0.96 and 0.98, which are very high.

The closer the curve is to the top left, the better the model performs.

Here, all curves are close to the top left corner of the plot.

So, our classifier performs very well for all classes.

feature_names = df_drug.columns[:-1]

plt.figure(figsize=(29, 10))

Next, we set the figure size and plot the decision tree.

Then we save and display the tree visualization as a PNG image file.

It first checks the Na to K value, then splits further using BP, Age, and Cholesterol.

Each colored box shows sample count and predicted drug type.

The tree splits until leaves mostly contain one drug, showing zero entropy.

The model showed high accuracy, indicating strong predictive performance.

Summary

In this tutorial, we have learnt about

• Decision Tree
• Decision Tree Structure and Nodes

Assignment

As an assignment, please do the following

Replace the max underscore depth as shown here.

Observe the change in Testing accuracy.

Assignment Solution

Show x img

FOSSEE Forum

Thank you

Thanks for joining