Difference between revisions of "Python-for-Machine-Learning/C2/Linear-Regression/English"

Welcome

Learning Objectives

• Linear Regression
• Simple Linear Regression
• Multiple Linear Regression
• Evaluation Metrics

System Requirements

• Ubuntu Linux operating system 24.04
• Jupyter Notebook IDE

Prerequisite

• The learner must have basic knowledge of Python.
• For prerequisite Python tutorials, please visit this website.

Code files

• The files used in this tutorial are provided in the Code files link.
• Please download and extract the files.
• Make a copy and then use them while practicing.

Linear Regression

• Linear regression is a predictive technique used in machine learning.
• It builds the relationship between a dependent and independent variable.
• Linear regression is categorized into Simple and Multiple linear regression.

Simple Linear Regression

• Simple Linear Regression is a way to find relationships between two variables.
• It studies how one independent variable affects one dependent variable.

Multiple Linear Regression

• Multiple linear Regression is an extension of simple linear regression.
• It examines how multiple factors influence a single outcome.

Evaluation Metrics

• To assess the model’s performance, we use evaluation metrics.
• These metrics indicate how well the regression model fits the data.
• The two common metrics are Mean Absolute Error and R squared score.

Open the file salaries_mlr.csv and point to the fields as per narration.

This dataset contains salaries based on years of experience.

We use salaries underscore mlr dot csv dataset for Multiple Linear Regression.

This dataset contains multiple columns as shown.

Type conda activate ml

Press Enter

First, we need to activate the machine learning environment.

Run the command conda space activate space ml.

Press Enter.

Type cd Downloads

Press Enter

Type jupyter notebook

Press Enter

Please navigate to the respective folder of your code file location.

Then type, jupyter space notebook and press Enter.

Click onLinearRegression.ipynb file

Click the LinearRegression dot ipynb file to open it.

Note that each cell will have the output displayed in this file.

import numpy as np

import pandas as pd

Press Shift+Enter

Make sure to Press Shift and Enter to execute the code in each cell.

df_salary=pd.read_csv("salaries.csv")

df_salary.head()

df_salary.describe()

sns.heatmap(df_salary.corr(), annot=True, cmap="coolwarm")

plt.show()

Correlation measures the relationship between two variables

The correlation values range from -1 to 1.

Let us understand the correlation value ranges.

Correlation Matrix

• A value of 1 means a perfect positive correlation.
• A value of -1 means a perfect negative correlation.
• A value of 0 means no correlation

plt.figure(figsize=(6,4))

Outliers are extreme values that differ significantly from other data points.

They are the small circles on the right side of the boxplot.

Here, incomes around 60,000 to 65,000 are considered as outliers.

The line inside the box is the median.

Q1 = df_salary[['experience', 'income']].quantile(0.25)  

Q3 = df_salary[['experience', 'income']].quantile(0.75)  

IQR = Q3 - Q1

We calculate first quartile Q1 and third quartile Q3 for experience and income.

Then, we compute the IQR and remove the outliers.

plt.figure(figsize=(6,4))

x=df_salary['experience']

y=df_salary['income']

x_train=np.array(x_train).reshape(-1,1)

x_test=np.array(x_test).reshape(-1,1)

The same is done for x underscore test for compatibility.

lr=LinearRegression()

lr.fit(x_train,y_train)

print("Intercept (W0):", lr.intercept_)

print("Coefficient (W1):", lr.coef_)

These define the model’s slope and relationship between experience and income.

y_pred_train = lr.predict(x_train)

y_pred_train = y_pred_train.round().astype(int)

y_pred_train

We round the predictions to whole numbers for better readability.

Then, we display the rounded predictions.

mae_train = mean_absolute_error(y_train, y_pred_train)

print("MAE (Training):", mae_train)

Mean Absolute Error measures prediction accuracy.

r2_score(y_pred_train, y_train)

R squared score measures how well the model explains the variance in the data.

A value closer to 1 indicates a stronger fit.

y_pred_test = lr.predict(x_test)

y_pred_test = y_pred_test.round().astype(int)

y_pred_test

plt.scatter(x_test,y_test)

It shows a positive correlation.

mean_absolute_error(y_test,y_pred_test)

r2_score(y_pred_test, y_test)

The R-squared score of 0.87 shows the model explains most of the variance.

Overall, the model performs well but has some prediction errors.

df_salaries = pd.read_csv(r"salaries_mlr.csv")

df_salaries.tail()

df_salaries.dtypes

df_salaries.isnull().sum()

df_salaries['gender'] = df_salaries['gender'].map({'m': 1, 'f': 0})

X = df_salaries.drop(columns='income')

y = df_salaries['income']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

model = LinearRegression()

model.fit(X_train, y_train)

coefficients = pd.DataFrame({'Feature': X.columns, 'Coefficient': model.coef_})

y_train_pred = model.predict(X_train)

y_train_pred = y_train_pred.round().astype(int)

y_train_pred

mae_train = mean_absolute_error(y_train, y_train_pred) print(f'Training data MAE: {mae_train}')

r2_train = r2_score(y_train, y_train_pred)

n_train = len(y_train

After that, we compute and print the adjusted R squared score.

y_test_pred = model.predict(X_test)

y_test_pred = y_test_pred.round().astype(int)

plt.scatter(y_test, y_test_pred, color='red', label='Predicted')

plt.scatter(y_test, y_test, color='blue', alpha=0.5, label='Actual')

mae_test = mean_absolute_error(y_test, y_test_pred) print(f'Testing data MAE: {mae_test}')

r2_test = r2_score(y_test, y_test_pred)

n_test = len(y_test)

k_test = X_test.shape[1]

It indicates the model explains 92.1 percent of income variance.

residuals = y_test - y_test_pred

plt.show()

We create a scatter plot of predicted values versus residuals.

The red dashed line represents zero residual.

Points above the line mean predictions are lower than actual values.

Points below the line mean predictions are higher than actual values.

Most residuals are close to zero, meaning predictions are fairly accurate.

Summary

In this tutorial, we have learnt about

• Linear Regression
• Simple Linear Regression
• Multiple Linear Regression
• Evaluation Metrics

Assignment

In Multiple Linear Regression code,

• Replace the test_size parameter as shown here.

• Replace the test_size parameter as shown here.
• Observe the change in MAE and Adjusted R squared score.

Assignment Solution

Show s1 img file

FOSSEE Forum

Thank you

Thanks for joining.