Difference between revisions of "Python for Automation"

Revision as of 18:16, 7 March 2025

Python's versatility and automation capabilities empower developers to create and maintain sophisticated platforms with unparalleled efficiency, innovation, and precision, all while significantly reducing the time required for routine tasks. By leveraging Python's automation potential, developers can streamline processes, minimize manual errors, and establish highly efficient workflows with minimal effort and maximum output. Whether you're safeguarding your data through automated file backup and compression, efficiently managing and organizing large directories, or extracting critical information from websites through web scraping, Python offers a robust suite of tools and libraries to accomplish these tasks with remarkable ease. Python also excels in automating login procedures, ensuring seamless and secure access to various platforms without the burden of repetitive manual input.

Beyond these capabilities, Python extends its power to parsing and analyzing log files, enabling developers to swiftly identify issues, monitor system performance, and derive actionable insights from data. This versatility makes Python an indispensable tool for modern development, empowering you to automate repetitive tasks and focus on what truly matters—building and innovating. In this Python Automation series, we will explore a comprehensive range of techniques and libraries designed to optimize your workflows, boost productivity, and elevate your development practices. Whether you're a seasoned developer or just starting, this series will equip you with the knowledge and tools to harness Python's full potential, ensuring your projects are efficient, effective, and at the cutting edge of innovation.

In this Python Automation series, we explore various techniques and libraries in Python to automate repetitive tasks.

The contributors who helped to create the outline, transcribe, code and record the tutorials are Dhaarani Pushpam S, Jasmine Tresa Jose and Sai Sathwik Matury under the guidance of Dr.T. Subbulakshmi from VIT Chennai. The Spoken Tutorial Effort for Python Automation is being contributed by Ms. Nirmala Venkat and Ms. Madhuri Ganapathifrom Spoken Tutorial project, IIT Bombay.

Basic Level

1. Overview of Python for Automation

• About Python for Automation
• About Libraries required for Automation
• Glimpse of Basic level Spoken Tutorials available on Python for Automation series
• Glimpse of Intermediate level Spoken Tutorials available on Python for Automation series
• About Spoken Tutorial

2. Setup Python environment for Automation

• Introduction to Automation
• Download package.txt file which contains library names used for automation series
• Create a Python virtual environment in Ubuntu
• Install all libraries in the packages.txt file in Ubuntu
• Verify installation of packages
• Deactivate the Virtual environment
• Setup Python environment if you have python version lower than 3.12.3
• Installation of python 3.12.3. in Windows
• Install all libraries in the package.txt file in windows

3. File Management

• File Management using python automation
• Libraries used for file management in python
• Obtain the list of files
• Separate the filename and its extension
• List the unique extension files
• Create new directory based on extension
• Move files to desired directory based on extension
• Organize the files in the sub folders
• Error and exception handling.
• Display organized files

4. File Backup and Compression

• Import necessary modules: Include os, shutil, tarfile, and datetime
• Implement compression logic for zip, tar.gz, and tar.bz2 formats
• Set up the overall backup process
• Ensure the backup directory exists or create it
• Delete any existing current backup directory
• Establish a fresh current backup directory for the new backup
• Traverse the source directory to mirror its structure
• Copy files from the source to the backup, only if they are new or modified.
• If compression is enabled, call compress_backup and delete the temporary backup directory
• Print a message indicating the backup's success and location
• Schedule the backup process to run at a fixed time every day

5. File Encryption and Decryption

• About Encryption and Decryption
• About symmetric Encryption
• Import the necessary libraries for file operations and encryption
• Read file data and encrypt it using a provided key
• Save the encrypted data with an ‘.encrypted’ extension
• Iterate through all files in a specified folder and its subdirectories and encrypt each file
• Read encrypted file data and decrypt it using a provided key
• Save the decrypted data by removing the ‘.encrypted’ extension
• Securely store the encryption key, as it is required for decryption
• Handle file paths to ensure files are located and saved in the intended directories during encryption and decryption

6. File Conversion

• About file conversion such as PDF to audio, JPG to PNG images, PDF to Docx using Python
• About PyPDF2, pdfplumber, pyttsx3, pdf2docx, docx2pdf and PIL/Pillow libraries
• Steps to open, manipulate, and save images using PIL, focusing on JPG to PNG conversion
• Testing and demonstrating the conversion of a JPG image to PNG format
• Details on initializing pyttsx3, selecting a PDF, extracting text, and converting it to speech
• About espeak package and how to install it
• Testing and demonstrating the conversion of a PDF document to an audio file
• Play and check the generated audio file
• Demonstrating the conversion of a PDF file to a word document using the converter class of the pdf2docx library
• Check the converted docx file by editing it

Intermediate Level

1. Web Scraping

• About Web scraping
• About the libraries used for web scraping process
• Scrape data from spoken tutorial website
• Python program to implement the web scrapping process
• Import the necessary libraries for making web requests, parsing HTML content, handling data, and *creating visualizations.
• Define various functions to scrape data from a specified URL
• Organize the collected data into a structured table and remove any duplicate entries
• Extract it to a CSV file
• Perform basic data analysis
• Generate a pie chart, bar chart to visualize the data
• Save the analyzed data into an Excel file, with separate sheets for raw data

2. Spelling and Grammar Checker

• About Spelling and Grammar Checker
• Python libraries used to implement the spelling and grammar checker
• Java and tkinter installation commands
• Import necessary modules and download NLTK 'words' corpus for spell-checking
• Initialize LanguageTool for US English grammar checking
• Define checkSpelling (text) function that uses NLTK’s words corpus
• Define checkGrammar(text) function that uses LanguageTool’s check function
• Define checkText() function to combine checks and print the results
• Define uploadFile() function that allows users to upload a text, Word, or PDF file
• Configure the result area to show errors or "None" if no issues
• Create a tkinter window (root) and set its title
• Add scrollable text area (textArea) for user input
• Implement "Check" button triggering checkText() function
• Include scrollable result area for displaying results

3. Building and Training a ChatBot

• About chatBot
• Building and training a chatbot
• Libraries required for building a chatbot
• Create an auto-updatable Q n A database for a chatbot
• Compare input questions to database and find similarities
• User input is processed to find the question in the database
• If the question is not found, the user is prompted to enter the answer
• The chatbot learns by adding new QnA pairs to its database
• Interaction continues until the user types "quit", terminating the program
• Different cases are shown and errors are handled

4. Log Monitoring

• About log monitor
• Types of Logs
• Libraries used to log monitor
• Analyze log file
• About Syslog files
• Python program to Monitor logs continuously
• Produce log summary
• Plot the log summary
• Report of abnormalities
• Print most abnormal entity and the period