Difference between revisions of "Linux AWK"

From Script | Spoken-Tutorial
Jump to: navigation, search
(Created page with "Linux AWK is a language for processing text files. AWK is typically used as a data extraction and reporting tool. It is a standard feature of most Unix-like operating systems....")
 
 
(2 intermediate revisions by 2 users not shown)
Line 7: Line 7:
  
 
The Spoken Tutorial Effort for Linux is being contributed by Antara De, Praveen S and Nancy Varkey.
 
The Spoken Tutorial Effort for Linux is being contributed by Antara De, Praveen S and Nancy Varkey.
 +
 +
'''Learners''': Polytechnic/UG/PG computer hardware students and system administrators can learn how to use the wide variety of AWK commands of Linux, to handle files, directories, processes etc.
  
 
__TOC__
 
__TOC__
  
==Introduction to Linux - Novice==
+
#Overview of Linux AWK
# Ubuntu desktop    
+
#* About awk commands
#* Ubuntu Desktop
+
#* AWK Process
#* Main Menu
+
#* Glimpse of Spoken Tutorials available on AWK
#* System Tray
+
#** Basics of AWK
#* Trash Bin icon (RHS corner)
+
#** Variables Operators
#* Desktop icon (LHS corner), pen-drive
+
#** Conditional Statements
# Synaptic package manager
+
#** Loops
#*Synaptic Package Manager
+
#** Single and Multi Dimensional Arrays
#*How to install packages
+
#** Built-in Functions and User Defined functions
# Ubuntu software system
+
#Basics of awk
#*Ubuntu-Software-Center
+
#*Installing softwares through Ubuntu Software Center
+
# Basic Commands    
+
#* Commands with example
+
#* Command interpreter
+
#* Shell
+
#* Using man
+
#* Apropos
+
#* Whatis
+
#* Using --help option
+
# General Purpose Utilities in Linux    
+
#* echo
+
#* uname
+
#* who
+
#* passwd
+
#* date
+
#* cal
+
#* Brief overview on Files and directories
+
#* pwd
+
#* ls
+
#* cat
+
# File System    
+
#* File
+
#* Directory
+
#* File Inode
+
#* Types of Files
+
#* Home directory and Current directory
+
#* Change Directory(cd)
+
#* mkdir,rmdir
+
# Working with Regular Files    
+
#* cat
+
#* rm
+
#* cp
+
#* mv
+
#* cmp
+
#* wc
+
# File Attributes    
+
#* chown, chmod, chmod -R, displaying files with ls -l
+
#* chmod u+, chmod a-w, chmod g+w, chmod -r, chgrp
+
#* inode, hard link, symbolic link
+
# Redirection Pipes    
+
#* Input,output and error stream
+
#* Redirection : > and >>
+
#* Pipes : |
+
# Working with Linux Process    
+
#* Process
+
#* Shell process
+
#* Process spawning - parent and child process
+
#* Process attributes - pid, ppid
+
#* Init Process
+
#* User process & System process
+
#* ps with options
+
# The Linux Environment    
+
#* Environment variable vs Local variables
+
#* set command
+
#* env command
+
#* SHELL, HOME, PATH, LOGNAME, PS1, PS2
+
#* history
+
#* ! and ~
+
#* alias
+
#  Basics of System Administration    
+
#* Root login-su
+
#* User management -  UID, GID, useradd, usermod, userdel
+
#* Discs – Du, df
+
# Simple filters     
+
#* Head
+
#* tail
+
#* sort
+
#* cut
+
#* paste
+
 
+
==Introduction to Linux - Intermediate==
+
# The grep command    
+
#* To see the content of a file
+
#* To list the entries of a particular stream
+
#* To ignore cases
+
#* Lines that do not match the pattern
+
#* To list the line numbers with the entries
+
#* To store the result in another file
+
#* To know the count
+
#  More on grep command    
+
#* Search using grep
+
#* To match more than one pattern
+
#* To check a word that has different spelling
+
#* Character class
+
#* The use of *
+
#* To match any one character using dot
+
#* To match a pattern at the beginning of the file
+
#* To match a pattern at the end of the file
+
# The sed command    
+
#* sed
+
#* To print using sed
+
#* Line Addressing
+
#* Context Addressing
+
#More on sed command    
+
#* substitute
+
#* insert
+
#* delete
+
#Basics of awk    
+
 
#* Awk Preliminaries
 
#* Awk Preliminaries
 
#* Selection criteria
 
#* Selection criteria
#* action
+
#* Action
 
#* Formatted printing - printf
 
#* Formatted printing - printf
 
#* Fields and -F option
 
#* Fields and -F option
 
#* Regular expressions
 
#* Regular expressions
 
#* NR - number of records
 
#* NR - number of records
#* Variables
+
#* Variables  
# Networking tools    
+
#Variables and Operators in Awk
#* Ping
+
#* User defined variables
#* Telnet
+
#* Variable Initialisation
#* ftp
+
#* Operators
#* ssh
+
#* String Concatenation
#* scp & sftp
+
#* String matching operator
# Linux Process    
+
#* Regular expressions
#* Fork
+
#* Relational Operators
#* Exec
+
#* BEGIN and END statement
#* Wait
+
#Built-in variables in Awk
#* Nice
+
#* Record separator rs
#* Kill with options
+
#* Output record separator ors
# More about Linux Process    
+
#* Number of records nr
#* Cron, crontab
+
#* Number of fields nf
 +
#* Field separator fs
 +
#* Output field separator ofs
 +
#* Filename
 +
#* Awk script
 +
#Conditional statements in awk:
 +
#* What are conditional statements
 +
#* Syntax of conditional statements
 +
#* Defining the rules or conditions to be checked and corresponding actions to be taken
 +
#* Execution of if
 +
#* Execution of else
 +
#* Execution of else if
 +
#* Changing the rules and re-executing the code
 +
#Loops in awk:
 +
#* Conditional loops in awk
 +
#** for loop
 +
#** while loop
 +
#** do-while loop
 +
#** Search pattern using awk
 +
#* Process data in single and multiple files using
 +
#** next
 +
#** nextfile
 +
#Basics of Single Dimensional Array in awk
 +
#* What are Arrays in awk
 +
#* How it is different from arrays in other programming languages
 +
#* Refer the elements of an array
 +
#* Syntax of assigning an array element
 +
#* The index in awk arrays
 +
#* Advantage of associative array
 +
#* Check whether any element exists in an array at a certain index
 +
#* Wrong and right ways to check the presence of an element
 +
#More on Single Dimensional Array in awk
 +
#* Using awk array with file
 +
#* Example: To calculate the HRA for all students as per a formula
 +
#* Scan the elements of an array
 +
#* New variation of "for loop"
 +
#* Delete statement
 +
#** Delete array element (single record)
 +
#** Delete whole array
 +
#* Show the values of ARGC and ARGV
 +
#* Example: ARGC = number of command line arguments supplied +1
 +
#* ARGV array and ENVIRON array
 +
#MultiDimensional Array in awk
 +
#* Definition of multidimensional array in awk
 +
#* Element is identified by a sequence of multiple indices
 +
#* Concatenated into a single string, with a separator between them
 +
#* Create 2 by 2 multidimensional array in awk
 +
#* Create the transpose of a 2 by 2 matrix
 +
#* Scan the multidimensional array
 +
#* Combine the for loop with the split function
 +
#* Syntax of the for loop
 +
#* Syntax of the split function
 +
#Built-in Functions in awk
 +
#* awk Built-in functions:
 +
#* Arithmetic functions
 +
#** sqrt(x) function
 +
#** int(x) function
 +
#** exp(x) function
 +
#** log(x) natural logarithm function
 +
#** sin(x) and cos(x) functions
 +
#* Random functions.
 +
#** rand()
 +
#** srand()
 +
#* String functions
 +
#** length() function
 +
#** index() function
 +
#** substr() function
 +
#* Input/Output functions and
 +
#** system() function
 +
#* Timestamp functions
 +
#** system("date")
 +
#User Defined Functions in awk
 +
#* Syntax of user defined function
 +
#* Syntax of function call
 +
#** Example to print stipend
 +
#** Explanation of if-else code
 +
#** Explanation of how to call the function
 +
#* Syntax of return statement
 +
#** Example to calculate average of an array
 +
#** Explanation of local variables
 +
#** Explanation of return value of the function call
 +
#* Explanation of reverse function

Latest revision as of 12:42, 16 January 2020

Linux AWK is a language for processing text files. AWK is typically used as a data extraction and reporting tool. It is a standard feature of most Unix-like operating systems. It consists of a set of actions to be taken against streams of textual data for purposes of extracting or transforming text, for eg: producing formatted reports. The language uses the string datatype, associative arrays, and regular expressions.

AWK was created at Bell Labs in the 1970s and its name is an acronym derived from the surnames of its authors—Alfred Aho, Peter Weinberger, and Brian Kernighan.


In this series, we would mainly concentrate on how to use the wide variety of AWK commands of Linux, to handle files, directories, processes etc. These tutorials are created using Ubuntu version 16.04 and above. Please see the associated text box of individual spoken tutorials on the website to decide the versions of Linux OS to which it is applicable.

The Spoken Tutorial Effort for Linux is being contributed by Antara De, Praveen S and Nancy Varkey.

Learners: Polytechnic/UG/PG computer hardware students and system administrators can learn how to use the wide variety of AWK commands of Linux, to handle files, directories, processes etc.


  1. Overview of Linux AWK
    • About awk commands
    • AWK Process
    • Glimpse of Spoken Tutorials available on AWK
      • Basics of AWK
      • Variables Operators
      • Conditional Statements
      • Loops
      • Single and Multi Dimensional Arrays
      • Built-in Functions and User Defined functions
  2. Basics of awk
    • Awk Preliminaries
    • Selection criteria
    • Action
    • Formatted printing - printf
    • Fields and -F option
    • Regular expressions
    • NR - number of records
    • Variables
  3. Variables and Operators in Awk
    • User defined variables
    • Variable Initialisation
    • Operators
    • String Concatenation
    • String matching operator
    • Regular expressions
    • Relational Operators
    • BEGIN and END statement
  4. Built-in variables in Awk
    • Record separator rs
    • Output record separator ors
    • Number of records nr
    • Number of fields nf
    • Field separator fs
    • Output field separator ofs
    • Filename
    • Awk script
  5. Conditional statements in awk:
    • What are conditional statements
    • Syntax of conditional statements
    • Defining the rules or conditions to be checked and corresponding actions to be taken
    • Execution of if
    • Execution of else
    • Execution of else if
    • Changing the rules and re-executing the code
  6. Loops in awk:
    • Conditional loops in awk
      • for loop
      • while loop
      • do-while loop
      • Search pattern using awk
    • Process data in single and multiple files using
      • next
      • nextfile
  7. Basics of Single Dimensional Array in awk
    • What are Arrays in awk
    • How it is different from arrays in other programming languages
    • Refer the elements of an array
    • Syntax of assigning an array element
    • The index in awk arrays
    • Advantage of associative array
    • Check whether any element exists in an array at a certain index
    • Wrong and right ways to check the presence of an element
  8. More on Single Dimensional Array in awk
    • Using awk array with file
    • Example: To calculate the HRA for all students as per a formula
    • Scan the elements of an array
    • New variation of "for loop"
    • Delete statement
      • Delete array element (single record)
      • Delete whole array
    • Show the values of ARGC and ARGV
    • Example: ARGC = number of command line arguments supplied +1
    • ARGV array and ENVIRON array
  9. MultiDimensional Array in awk
    • Definition of multidimensional array in awk
    • Element is identified by a sequence of multiple indices
    • Concatenated into a single string, with a separator between them
    • Create 2 by 2 multidimensional array in awk
    • Create the transpose of a 2 by 2 matrix
    • Scan the multidimensional array
    • Combine the for loop with the split function
    • Syntax of the for loop
    • Syntax of the split function
  10. Built-in Functions in awk
    • awk Built-in functions:
    • Arithmetic functions
      • sqrt(x) function
      • int(x) function
      • exp(x) function
      • log(x) natural logarithm function
      • sin(x) and cos(x) functions
    • Random functions.
      • rand()
      • srand()
    • String functions
      • length() function
      • index() function
      • substr() function
    • Input/Output functions and
      • system() function
    • Timestamp functions
      • system("date")
  11. User Defined Functions in awk
    • Syntax of user defined function
    • Syntax of function call
      • Example to print stipend
      • Explanation of if-else code
      • Explanation of how to call the function
    • Syntax of return statement
      • Example to calculate average of an array
      • Explanation of local variables
      • Explanation of return value of the function call
    • Explanation of reverse function

Contributors and Content Editors

Nancyvarkey, PoojaMoolya, Pravin1389