Python-for-Automation/C3/Building-and-Training-a-Chatbot/English - Revision history

Madhurig at 09:51, 18 November 2024

2024-11-18T09:51:53Z

Madhurig at 09:22, 18 November 2024

2024-11-18T09:22:08Z

Nirmala Venkat: Created page with " {| border="1" |- || '''Visual Cue''' || '''Narration''' |- |- style="border:1pt solid #000000;padding:0.176cm;" || Show Slide: '''Welcome''' || Hello and welcome to the Spok..."

2024-10-28T11:22:52Z

Created page with " {| border="1" |- || '''Visual Cue''' || '''Narration''' |- |- style="border:1pt solid #000000;padding:0.176cm;" || Show Slide: '''Welcome''' || Hello and welcome to the Spok..."

New page

{| border="1"
|-
|| '''Visual Cue'''
|| '''Narration'''
|-
|- style="border:1pt solid #000000;padding:0.176cm;"
|| Show Slide:

'''Welcome'''
|| Hello and welcome to the Spoken Tutorial on '''“Building and Training a ChatBot”'''
|- style="border:1pt solid #000000;padding:0.176cm;"
|| Show Slide:

'''Learning Objectives'''
|| In this tutorial, we will learn to
* <div style="margin-left:1.27cm;margin-right:0cm;">Create an auto-updatable Q n A database for a chatbot</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Compare input questions to database and find similarities and</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Teach the chatbot to learn an answer that it does not know</div>
|-
|| '''Show slide:'''

System Requirements
|| To record this tutorial, I am using
*<div style="margin-left:1.27cm;margin-right:0cm;">'''Ubuntu Linux OS version 22.04'''</div>
*<div style="margin-left:1.27cm;margin-right:0cm;">'''Python 3.12.3'''</div>

|-
|| '''Show slide:'''

Prerequisite

[https://www.spoken-tutorial.org/ https://spoken-tutorial.org]
|| To follow this tutorial:
* <div style="margin-left:1.27cm;margin-right:0cm;">You must have basic knowledge of using Linux Terminal and Python</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">For pre-requisite Linux and Python Tutorials, please visit this website</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Python libraries required for automation must be installed</div>
|-
|| '''Show slide:'''

Code files
||
* <div style="margin-left:1.27cm;margin-right:0cm;">The files used in this tutorial are provided in the '''Code files''' link.</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Please download and extract the files.</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Make a copy and then use them while practicing.</div>
|-
|| '''Show Slide:'''

About Chatbot
||
* <div style="margin-left:1.27cm;margin-right:0cm;">A chatbot is a software application designed to simulate conversations</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Typically they interact with human users through text or speech</div>
|-
|| '''Show Slide:'''

Building and Training a Chatbot
|| In this tutorial we will build a Chatbot and an auto-updatable database.
* <div style="margin-left:1.27cm;margin-right:0cm;">When the user asks a question, the bot will search for an answer in the database</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">If the question is not found, the user’s answer is saved to the database</div>
|-
|| '''Show Slide:'''

Chatbot - Libraries
|| The Python libraries required for this tutorial are:
* <div style="margin-left:1.27cm;margin-right:0cm;">'''Json '''module''' '''which parses '''JSON '''strings with key/value pairs</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">'''difflib '''module is used to compare sequences</div>
|-
|| '''Narration'''
|| Do you know how a chatbot works?

Let me show you a small demo.
|-
|| '''Show a demo on terminal'''

'''Type python3 chatbot.py'''
|| Let me type “Hello” and press Enter.

Can you see a response?

Again let me type “How are you?” and press Enter

You can see the chatbot is giving appropriate replies.
|-
|| '''Narration'''
|| Let us see how it has been implemented in Python.
|-
|| '''Open downloads folder'''
|| I have created the source files required for this demonstration.
|-
|| '''Open QnA_base.json file'''
|| '''QnA_base.json''' file acts as the database for the ChatBot.

It has question-answer pairs stored in a dictionary format.
|-
|| '''Switch tab to Downloads folder'''

'''Point to chatbot.py and QnA_base.json'''
|| <span style="color:#252525;">'''chatbot</span>.py '''is the '''python '''code which is linked to the '''json '''file to fetch data.

Note that the '''json''' file and '''python '''code file should be saved in the same location.
|-
|| '''Highlight the lines:'''

from __future__ import annotations

import json

|| Now, we will go through the '''python''' code in the text editor.

<span style="background-color:#ffffff;">It allows the use of new </span><span style="background-color:#ffffff;">'''Python '''</span><span style="background-color:#ffffff;">features in modules if there are transitions</span> between versions.

First we need to import the '''json '''module for encoding and decoding '''JSON '''data.
|-
|| '''Highlight the lines:'''

from difflib import get_close_matches
|| '''Get <span style="color:#252525;">underscore </span>close <span style="color:#252525;">underscore </span>matches''' is a function from the '''difflib '''library.

It is used to find matches for a string in a list of strings based on similarity.
|-
|| '''Highlight:'''

def loadData(file_path: str) -> dict:
|| '''loadData''' loads the '''JSON '''file.
|-
|| '''Highlight:'''

with open(file_path, 'r') as file:
|| Next we open the '''JSON '''file in '''read '''mode.
|-
|| '''Highlight:'''

return json.load(file)
|| '''json dot load''' function is used to read the '''JSON '''data in the file.
|-
|| '''Highlight: '''

def saveData(file_path: str, data: dict):
|| '''saveData''' function writes new data into our Chatbot’s database.
|-
|| '''Highlight:'''

with open(file_path, 'w') as file:
|| Next we open the '''JSON''' file in write mode.
|-
|| '''Highlight:'''

json.dump(data, file, indent=2)
|| '''Json dot dump''' writes the content of the '''JSON '''file''' '''into the dictionary '''data'''.
|-
|| '''Highlight:'''

indent=2
|| We can also add an '''indent''' value of '''2''' for readability in the '''JSON''' file.
|-
|| '''Highlight:'''
|| '''findClosestMatch''' function finds the best match for a user’s question in a list of questions.
|-
|| '''Highlight:'''

query
|| Here, '''query''' is the variable that stores the user’s question.
|-
|| '''Highlight:'''

questionsp
|| '''questions '''is a list of all questions from the''' JSON '''file.
|-
|| '''Highlight:'''
|| '''Get_close_matches''' function is used to find a match to '''query''' in the list '''questions.'''
|-
|| '''Highlight:'''

cutoff=0.6
|| Here, '''cutoff''' is used to find the percentage of similarity that is acceptable.

We set it as '''0.6'''.

Now the Bot answers the question with a 60 percent match in the '''JSON '''file<span style="color:#ff0000;">.</span>
|-
|| '''Highlight:'''

return matches[0] if matches else None
|| Then, we return the best match.

If no match is found then '''None''' is returned.
|-
|| '''Highlight:'''

def fetchAnswer(query: str, chatbotDatabase: dict) -> str | None:
|| '''fetchAnswer '''function retrieves the answer for a given question from the '''JSON '''file'''.'''
|-
|| '''Highlight:'''

query
|| Here, '''query '''is the question asked by the user for which the answer is retrieved.
|-
|| '''Highlight:'''

database
|| '''database''' is the dictionary containing the Bot’s database.
|-
|| '''Highlight:'''

|| We iterate through each question-answer pair in''' database.'''

If the question matches the input then we return the corresponding answer.
|-
|| '''Highlight:'''

def runChatbot():
|| '''runChatbot''' function acts as the main function to implement the chatbot.
|-
|| '''Highlight:'''

database = loadData(r"QnA_base.json")
|| We first call the '''loadData '''function to load the '''JSON '''file<span style="color:#ff0000;">.</span>
|-
|| '''Highlight:'''
|| We print a greeting, then start an infinite loop to interact with the user.
|-
|| '''Highlight:'''

lower()
|| '''lower''' function converts the user input to lowercase.

This prevents the case sensitivity of strings while typing the answer by the user.
|-
|| '''Highlight:'''
|| The loop will only break if the user enters '''quit.'''
|-
|| '''Highlight:'''
|| '''findClosestMatch''' is used to find a match for the user’s input in the database.
|-
|| '''Highlight:'''
|| If a match is found, we can call '''fetchAnswer''' and print the answer.
|-
|| '''Highlight:'''
|| If no match is found, we prompt the user to give an answer to the question.
|-
|| '''Highlight:'''
|| The answer that the user enters as input is saved in a variable named '''newResponse'''.
|-
|| '''Highlight:'''

if newResponse.lower() != "skip":

chatbotDatabase["questions"].append({"questions": userInput, "answer": newResponse})
| style="border-top:none;border-bottom:1pt solid #000000;border-left:none;border-right:1pt solid #000000;padding-top:0cm;padding-bottom:0cm;padding-left:0.176cm;padding-right:0.176cm;" | '''newResponse''' is then appended to the '''JSON '''file using the append function.

If the user enters '''skip''', we move on to the next iteration of the while loop.

Now the bot will prompt the user for a question again.
|-
|| '''Highlight:'''

saveData(r"QnA_base.json", chatbotDatabase)

print("Great! I've learned something new.")
|| We call '''saveData''' to overwrite the '''JSON '''file<span style="color:#ff0000;">''' '''</span>with the new answer.

The bot displays a message after successful updation in the database.
|-
|| '''Highlight:'''

runChatbot()
|| Finally, we call the '''runChatbot '''function.
|-
|| '''Press ctrl+s'''
|| Save the code as '''chatbot.py '''in the '''Downloads''' folder.
|-
|| '''Running the code:'''

Open terminal -

Press Ctrl + Alt + T
||

Open the '''terminal''' by pressing '''Control, Alt and T '''keys simultaneously.
|-
|| '''Type in terminal:'''

source Automation/bin/activate
|| We will open the '''virtual environment''' we created for the '''Automation '''series.

Type '''source space Automation forward slash bin forward slash activate'''.

Then press '''enter'''.
|-
|| '''Type in terminal:'''

> cd Downloads

> Python3 chatbot.py
|| In the terminal, type '''cd Downloads '''and press enter'''.'''

Next type '''python3 chatbot.py''' and press enter.
|-
|| '''Type in terminal:'''

Hello
|| The chatbot prints a greeting message and prompts the user.

Here I am going to type '''Hello '''and press '''Enter'''
|-
|| '''Highlight:'''

Hey there
|| The bot answers: '''Hey there.'''

|-
|| '''Type in terminal:'''

What is the capital of Denmark?
|| Let us ask: '''What is the capital of Denmark?'''
|-
|| '''Highlight:'''

Copenhagen

'''Show the JSON file and highlight the Q and A'''
|| The bot answers '''Copenhagen''' because this answer is already in its database,

We will check the''' JSON''' file now. It is working as we expected.
|-
|| '''Type in terminal:'''

In what year was the first international modern Olympiad held?
|| Now type: '''In what year was the first modern Olympiad held?'''
|-
|| '''Highlight:'''

Hmm, I don't have an answer for that. Can you update me?
|| The bot does not know the answer to this and hence it asks '''Can you teach me?'''

The chatbot is waiting for our input.
|-
|| '''Type in terminal:'''

1896
|| Let us enter the answer: '''1896'''

The bot responds with a message to confirm that the answer has been added.
|-
|| '''Show:'''

QnA_base.json file
|| Check the '''json '''file.

We have received an alert: '''This file has been changed by another program'''
|-
|| '''Click on reload'''
|| Let us reload this file.
|-
|| '''Highlight:'''
|| Now we can see that the new question and answer have been appended to the file.

Switch back to the terminal again.
|-
|| '''Show terminal and type:'''

What is the capital of India?
|| Now let us ask: '''What is the capital of India?'''
|-
|| '''Highlight:'''

Copenhagen
|| The Bot gives us the wrong answer. Why does this happen?
|-
|| '''Type in terminal:'''

quit
|| First let us terminate the program by typing '''quit'''.
|-
|| '''Show file:'''

chatbot.py

|| Switch to the source code and check the function '''get_close_matches'''.

We set the cutoff value or the similarity percentage to 60 percent.
|-
|| '''Highlight in QnA_base.json:'''

what is the capital of
|| The program matches the words ''''what is the capital of''''.

This forms a 60 percent match with an existing question in the database.

The accuracy rate is significantly low, as is evident.
|-
|| '''Type:'''

Change 0.6 to 1.0
|| This problem can be fixed by changing the '''cutoff '''value to something higher.

Let us change it to 1 point 0.

Now the bot will only match questions with 100 percent similarity in the database.
|-
|| '''Save chatbot.py:'''

'''Type in terminal:'''

python3 chatbot.py
|| Save the source code and switch to the '''terminal'''.

Rerun the code by typing the command '''python3 chatbot.py'''
|-
|| '''Type in terminal:'''

What is the capital of India?
|| We will type the question: '''What is the capital of India? '''again.
|-
|| '''Highlight:'''

Hmm, I don't have an answer for that. Can you teach me?
|| This time the bot does not know the answer.

This means that the Chatbot is treating the question as a new one.

Enter the answer '''New Delhi'''

This will automatically be saved in our '''QnA_base.json''' file
|-
||
|| In this way, we can build a chatbot for our specific purpose.
|-
|| '''Type in terminal:'''

quit

deactivate
|| Type '''quit '''to terminate the program.

Then, type '''deactivate''' and press '''Enter.'''This will allow you to exit the virtual environment.
|-
|| '''Show slide:'''

Summary
|| This brings us to the end of the tutorial. Let us summarize.

In this tutorial, we have learnt to
* <div style="margin-left:1.27cm;margin-right:0cm;">Create an auto-updatable QnA database for a chatbot</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Compare input questions to the database and find similarities</div>
* <div style="margin-left:1.27cm;margin-right:0cm;">Teach the chatbot to learn an answer that it doesn't know</div>
|-
|| '''Show slide:'''

Assignment
|| As an assignment, please do the following:
# <div style="margin-left:1.27cm;margin-right:0cm;">Consider a product of your choice, for example a Samsung phone.</div>
# <div style="margin-left:1.27cm;margin-right:0cm;">Create a Customer Service chatbot that can answer customer queries.</div>
|-
|| '''Assignment- '''Customer Service chatbot

assignment_chatbot.jpg
|| Try to create a Customer Service chatbot as shown.
|-
|| '''Show slide:'''

About the Spoken Tutorial Project
|| The video at the following link summarizes the '''Spoken Tutorial Project.'''Please download and watch it
|-
|| '''Show Slide:'''

<div style="color:#252525;">Spoken Tutorial Workshops</div>
|| The '''Spoken Tutorial Project''' team conducts workshops and gives certificates.

For more details, please write to us.
|-
|| <span style="color:#252525;">'''Show Slide: '''</span><span style="color:#252525;">Answers for THIS Spoken Tutorial</span>
|| Please post your timed queries in this forum.
|-
|| <div style="color:#252525;">'''Show Slide:'''</div>

<div style="color:#252525;">FOSSEE Forum</div>
|| For any general or technical questions on''' Python for Automation''', visit the '''FOSSEE forum''' and post your question.
|-
|| '''Show slide:'''

Acknowledgement
|| '''The Spoken Tutorial Project''' was established by the '''Ministry of Education, Government of India.'''
|-
|| '''Show slide:'''

Thank You
|| This is '''Jasmine Tresa Jose''', a '''FOSSEE Semester Long Intern 2024, IIT Bombay''' signing off.

Thanks for joining.
|-
|}

@@ Line 14: / Line 14: @@
 '''Learning Objectives'''
-|| In this tutorial, we will learn to
+|| In this tutorial, we will learn to,
 * Create an auto-updatable Q n A database for a chatbot
 * Compare input questions to database and find similarities and
@@ Line 31: / Line 31: @@
 Prerequisite
-[https://www.spoken-tutorial.org/ https://spoken-tutorial.org]
+[https://www.spoken-tutorial.org/https://www.spoken-tutorial.org/]
 || To follow this tutorial:
 * You must have basic knowledge of using Linux Terminal and Python
@@ Line 63: / Line 63: @@
 Chatbot - Libraries
 || The Python libraries required for this tutorial are:
-* '''Json '''module''' '''which parses '''JSON '''strings with key/value pairs
+* '''Json '''module which parses '''JSON '''strings with key/value pairs
 * '''difflib '''module is used to compare sequences
 |-
 || '''Narration'''
-|| Do you know how a chatbot works?
+|| Do you know how a '''chatbot''' works?
 Let me show you a small demo.
@@ Line 147: / Line 147: @@
 json.dump(data, file, indent=2)
-|| '''Json dot dump''' writes the content of the '''JSON '''file''' '''into the dictionary '''data'''.
+|| '''Json dot dump''' writes the content of the '''JSON '''file into the dictionary '''data'''.
 |-
 || '''Highlight:'''
 indent=2
-|| We can also add an '''indent''' value of '''2''' for readability in the '''JSON''' file.
+|| We can also add an '''indent''' value of 2  for readability in the '''JSON''' file.
 |-
 || '''Highlight:'''
@@ Line 165: / Line 165: @@
 questionsp
-|| '''questions '''is a list of all questions from the''' JSON '''file.
+|| '''questions ''' is a list of all questions from the''' JSON '''file.
 |-
 || '''Highlight:'''
-|| '''Get_close_matches''' function is used to find a match to '''query''' in the list '''questions.'''
+|| '''Get_close_matches''' function is used to find a match to '''query''' in the list '''questions'''.
 |-
 || '''Highlight:'''
@@ Line 188: / Line 188: @@
 || '''Highlight:'''
-def fetchAnswer(query: str, chatbotDatabase: dict) -> str | None:
+def fetchAnswer(query: str, chatbotDatabase: dict)
-|| '''fetchAnswer '''function retrieves the answer for a given question from the '''JSON '''file'''.'''
 |-
 || '''Highlight:'''
@@ Line 203: / Line 205: @@
 || '''Highlight:'''
-|| We iterate through each question-answer pair in''' database.'''
+|| We iterate through each question-answer pair in''' database'''.
 If the question matches the input then we return the corresponding answer.
@@ Line 210: / Line 212: @@
 def runChatbot():
-|| '''runChatbot''' function acts as the main function to implement the chatbot.
+|| '''runChatbot''' function acts as the main function to implement the '''chatbot'''.
 |-
 || '''Highlight:'''
 database = loadData(r"QnA_base.json")
-|| We first call the '''loadData '''function to load the '''JSON '''file<span style="color:#ff0000;">.
+|| We first call the '''loadData '''function to load the '''JSON '''file.
 |-
 || '''Highlight:'''
@@ Line 246: / Line 248: @@
 if newResponse.lower() != "skip":
-chatbotDatabase["questions"].append({"questions": userInput, "answer": newResponse})
+chatbotDatabase["questions"]
 || '''newResponse''' is then appended to the '''JSON '''file using the append function.
 If the user enters '''skip''', we move on to the next iteration of the while loop.
-Now the bot will prompt the user for a question again.
+Now the '''bot''' will prompt the user for a question again.
 |-
 || '''Highlight:'''
@@ Line 258: / Line 262: @@
 print("Great! I've learned something new.")
-|| We call '''saveData''' to overwrite the '''JSON '''file<span style="color:#ff0000;">''' '''with the new answer.
+|| We call '''saveData''' to overwrite the '''JSON '''file with the new answer.
-The bot displays a message after successful updation in the database.
+The '''bot''' displays a message after successful updation in the database.
 |-
 || '''Highlight:'''
@@ Line 293: / Line 297: @@
 > Python3 chatbot.py
-|| In the terminal, type '''cd Downloads '''and press enter'''.'''
+|| In the terminal, type '''cd Downloads '''and press '''Enter'''.
 Next type '''python3 chatbot.py''' and press enter.
@@ Line 307: / Line 311: @@
 Hey there
-|| The bot answers: '''Hey there.'''
+|| The bot answers: '''Hey there'''.
 |-
@@ Line 322: / Line 326: @@
 || The bot answers '''Copenhagen''' because this answer is already in its database,
-We will check the''' JSON''' file now. It is working as we expected.
+We will check the''' JSON''' file now.
 |-
 || '''Type in terminal:'''
@@ Line 434: / Line 440: @@
 || Type '''quit '''to terminate the program.
-Then, type '''deactivate''' and press '''Enter.'''This will allow you to exit the virtual environment.
+Then, type '''deactivate''' and press '''Enter.
 |-
 || '''Show slide:'''
 Summary
-|| This brings us to the end of the tutorial. Let us summarize.
+|| This brings us to the end of the tutorial.
 In this tutorial, we have learnt to
@@ Line 461: / Line 471: @@
 About the Spoken Tutorial Project
-|| The video at the following link summarizes the '''Spoken Tutorial Project.'''Please download and watch it
+|| The video at the following link summarizes the '''Spoken Tutorial Project.'''
 |-
 || '''Show Slide:'''
@@ Line 476: / Line 488: @@
 FOSSEE Forum
-|| For any general or technical questions on''' Python for Automation''', visit the '''FOSSEE forum''' and post your question.
+|| For any general or technical questions on''' Python for Automation''',
 |-
 || '''Show slide:'''