Difference between revisions of "Linux-AWK/C2/Loops-in-awk/English"

From Script | Spoken-Tutorial
Jump to: navigation, search
Line 1: Line 1:
'''Title of script:''' Built-in variables and awk Script
+
'''Title of script: Loops in Awk'''
  
'''Author:''' Antara Roy Choudhury
+
'''Author: Antara Roy Choudhury'''
  
'''Keywords:''' Built-in variables, RS, ORS, NR, NF, FS, OFS, FILENAME
+
'''Keywords: for, while, do-while, next, nextfile'''
  
  
Line 13: Line 13:
 
|-
 
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 1: Introduction
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 1: Introduction
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Welcome to the spoken tutorial on '''awk built-in variables''' and '''awk script.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Welcome to this spoken tutorial on '''Loops''' in '''awk'''.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 2: Learning Objectives
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 2: Learning Objective
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In this tutorial we will learn about
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In this tutorial we will learn about-
  
  
* '''Built-in variables '''
+
* '''while'''
* '''awk script'''
+
* '''do-while'''
 +
* '''for '''
 +
* and more '''looping constructs '''in''' awk'''
  
 
We will do this through some examples.
 
We will do this through some examples.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 3a: System requirement
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 3: System requirement
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| To record this tutorial, I am using  
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| To record this tutorial, I am using  
  
 
* '''Ubuntu Linux 16.04 OS '''and''' '''
 
* '''Ubuntu Linux 16.04 OS '''and''' '''
* '''gedit text editor version 3.20.1'''
+
* '''gedit text editor 3.20.1'''
 +
 
 +
* You can use any text editor of your choice.
  
  
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 3b: Code Files
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 4: Pre-requisite
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The files used in this tutorial are available in the '''Code Files''' link on this tutorial page.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| * To practice this tutorial, you should have gone through the previous '''awk''' tutorials on our website.
 +
* You should have familiarity with any '''programming language''' like C or C++
 +
* If not, then please go through the corresponding tutorials on our website.
  
  
Please download and use them.
 
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 4: Prerequisite
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 5: Code Files
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| To practice this tutorial, you should have gone through the earlier '''awk tutorials''' on this website.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The files used in this tutorial are available in the '''Code Files''' link on this tutorial page.
  
  
If not, then please go through the corresponding tutorials on this website.
+
Please download and extract them.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 5: awk built-in variables
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 6: Loops
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| First, let us see some of the '''built-in variables '''in '''awk'''.  
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| A '''loop''' allows us to perform one or more actions repeatedly.
  
  
* Capital '''RS''' specifies the '''record separator''' in an '''input''' file. By default, it is '''newline'''.
+
'''while, do-while '''and''' for '''are the available '''loops''' in '''awk'''.  
* Capital '''FS''' specifies the '''field separator '''in an '''input''' file.
+
 
+
By default, the value of '''FS''' is a '''whitespace'''.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 5: awk built-in variables
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 7: While loop
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| * Capital '''ORS''' defines the '''output record separator'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The syntax of '''while''' '''loop''' can be seen here.
  
By default, it is '''newline'''.
 
  
* Capital '''OFS''' defines the '''output field separator'''.
+
'''While loop '''first checks whether the specified '''condition''' is true.  
  
By default, it is '''whitespace.'''
+
If yes, then it executes the code within the '''body.'''
  
  
Let us understand the meaning of each of these.
+
This '''loop '''will be repeated as long as the specified '''while condition '''is '''true'''.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show awkdemo.txt in Gedit
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show awkdemo.txt file in gedit
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us have a look at the '''awkdemo''' file now.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We will use the same '''awkdemo.txt '''file, that we have used earlier.  
 
+
 
+
When we are processing this '''awkdemo''' file with '''awk''' command, this becomes our '''input '''file.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight appropriately
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| show while_loop.awk in Gedit
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Observe that all the records are separated from each other by a '''newline character'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| I have already written a script named '''while_loop.awk'''
  
  
'''newline''' is the default value for '''record separator RS variable. '''
+
The same is available in the '''Code Files''' link of this tutorial.
  
 
+
|-
So, there is no need to do anything else.
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Point to the pipe symbol
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here we have set the '''field separator''' as '''Pipe symbol.'''
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight vertical bar | character
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight i = 1
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Notice that all the fields are separated by the '''pipe symbol'''.  
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Initially, we have to the set the value of the '''loop variable i '''as 1.  
  
  
How can we inform '''awk '''about it?
 
  
 
Let us see.
 
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 6: How to reset value of FS variable?
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight f = 1
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| By default, any number of '''spaces''' or a '''tabs''' separate the fields.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here, we have taken one more variable '''f''' and initialized it to 1.
  
  
We can reset this with the help of '''hyphen capital F''' option as learnt in our earlier tutorials.
+
Variable '''f''' represents the '''field counter '''or the position of the '''fields''' for each record.
  
 
+
|-
Or else, we can reset this in the '''BEGIN section '''with the use of '''FS''' '''variable'''.  
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <nowiki>Highlight while (i<=3)</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now, in the '''while condition''', we check if '''i''' is less than or equal to 3.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight Print statement
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us do this through an example.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| If yes, then it will print the value in the '''f'''<sup>th</sup>''' field, '''for that record in '''awkdemo.txt '''file.
 
+
 
+
Suppose, I want to find out the name of students who are getting a stipend of more than Rs.5000.
+
  
 
|-
 
|-
| style="border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Open the terminal
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight f++
| style="border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Open the '''terminal''' by pressing '''CTRL, ALT''' and '''T''' keys.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Then we will increase the '''field counter f''' by 1.
  
 
|-
 
|-
| style="border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <nowiki>cd /<saved folder></nowiki>
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight i++
| style="border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Go to the folder in which you downloaded and extracted the '''Code Files''' using '''cd command.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| After that, we will also increment the value of '''loop''' variable '''i''' by 1.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| awk 'BEGIN{FS="|"} $6>5000 {print $2,$6}' awkdemo.txt
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight printf("\n")
| style="border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type the command as shown here.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This '''printf''' is for printing a '''newline character''' at the end of each row.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight {FS="|"} area
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here in the '''BEGIN''' section, we have assigned the value of '''FS''' as a '''pipe symbol.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This '''loop''' will be executed for all the records in the '''awkdemo.txt''' file.
  
  
Similarly, we can modify '''RS variable.'''
+
Which means the first 3 fields will get printed for each record.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <nowiki>[Enter]</nowiki>
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press '''Enter''' to execute the command.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let’s execute this code now.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Open the terminal
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The output shows the list of students who are receiving more than Rs.5000 as a stipend.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Open the '''terminal''' by pressing '''Ctrl, Alt''' and '''T''' keys.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight appropriately
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <nowiki>cd /<saved folder></nowiki>
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here the '''name '''field and the '''stipend '''field is separated by a blank '''space'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Go to the folder in which you downloaded and extracted the '''Code Files''' using '''cd command'''
 
+
 
+
Also, all the records are separated by a '''newline character.'''
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type in terminal:
  
 +
awk -f while_loop.awk awkdemo.txt
  
 +
<nowiki>[Enter]</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now type:
  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Suppose we want '''colon '''as the '''output field separator.'''
+
'''awk space minus small f while_loop.awk space awkdemo.txt'''
  
  
And double '''newline '''as '''output record separator'''.
+
Press '''Enter'''
  
 
+
|-
How can we do this? Let us see.
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Observe that we get the first three '''fields''' of all the rows in the output.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In terminal
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In the '''terminal''', press the '''up arrow '''key to get the previously executed command.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us do the same with the '''do-while loop'''.  
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the previous command
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 8: Do While Loop
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The syntax of '''do-while loop''' can be seen here.
  
  
awk 'BEGIN{FS="|";OFS=":"; ORS="\n\n"} $6>5000 {print $2, $6}' awkdemo.txt
+
The&nbsp;'''do-while&nbsp;loop''' always executes the&nbsp;code inside the '''body''' once.
  
<nowiki>[Enter]</nowiki>
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the command as shown here.
 
  
 +
Then it checks the specified '''condition.'''
  
And then press '''Enter.'''
 
  
|-
+
And repeats the&nbsp;code inside the '''body''',&nbsp;as long as the specified&nbsp;'''condition'''&nbsp;is '''true'''.
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We get the output in the desired format.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show sample.txt
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| show do_loop.awk in Gedit
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| I have already written a script and named it as '''do_loop.awk'''
  
  
Point or highlight
+
The same is available in the '''Code Files '''link.  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now, suppose our new input file is '''sample.txt.'''
+
 
+
 
+
Observe that the '''field separator '''here is '''newline '''and '''record separator '''is double '''newline.'''
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the contents and highlight appropriately
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| How can we extract the roll no. and name information from this file?
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In this code, these are the '''statements '''within the '''do loop, '''which will be executed first.
  
  
Yes, you have guessed correctly.
+
This is the '''condition '''that will be checked.
  
  
We have to modify both '''FS''' and '''RS''' '''variables'''.
+
After that, the '''statements '''inside the '''loop '''will be executed repeatedly as long as the '''condition '''is '''true'''.
 
+
 
+
Pause this tutorial and do this as an assignment.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 7:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Next, let us see other '''built-in variables.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This '''loop''' will iterate for all the records in the '''awkdemo.txt''' file
  
  
* Capital '''NR''' gives us the '''Number of Records''' processed by '''awk'''
+
Which means the first 3 fields will get printed for all the records.
* Capital '''NF''' gives the '''Number of Fields '''in the current record
+
  
 +
|-
 +
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type:
  
 +
awk -f do_loop.awk awkdemo.txt
  
|-
+
<nowiki>[Enter]</nowiki>
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let’s switch to the '''terminal. '''Let me clear the''' terminal.'''
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us see one example of this.  
+
  
 +
Now type:
  
Suppose, we want to find incomplete lines in the file.
 
  
 +
'''awk space -f do_loop.awk awkdemo.txt'''
  
Here, incomplete line means it has less than the normal 6 fields.
+
and press '''Enter'''
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the terminal and clear it
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Scroll up and show outputs of while loop and do loop
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the '''terminal'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We get the same output.
  
  
Let me clear the terminal using '''Ctrl''' and '''L''' keys
+
Then why do we have both '''while '''and '''do-while loops'''?
 +
 
 +
Let us understand the difference.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In while_loop.awk, assign i=4
 
+
awk 'BEGIN{FS="|"} NF !=6 {print NR, $0}' awkdemo.txt
+
 
+
<nowiki>[Enter]</nowiki>
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type the command as shown.
+
  
 +
<nowiki>[code uploaded as while_loop_mod.awk]</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the file '''while_loop.awk'''
  
  
 +
Now, change the value of '''loop counter i '''from 1 to 4.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the command and highlight appropriate areas as per narration
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| As the fields are separated by '''pipe '''symbol, set the '''FS''' value to '''pipe''' symbol in the '''BEGIN section.'''
+
  
 +
<nowiki>(i<=3)</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This will make the specified '''condition false''' from the beginning.
  
Next we have written '''NF!=6'''.
 
  
 +
So this means, we should not get any output.
  
This checks whether the number of fields in the current line, is not equal to 6.
+
|-
 +
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press Ctrl+S
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Save the file and switch to '''terminal'''.
  
 +
|-
 +
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| awk -f while_loop.awk awkdemo.txt
  
If true, then
+
<nowiki>[Enter]</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press up key until you get the command for executing the '''while loop.'''
  
* '''print section '''will print the record’s line number '''NR''',
 
* along with the entire line denoted by '''$0'''.
 
  
Press '''Enter'''.
+
Now press '''Enter.'''
  
 
|-
 
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In the output, we can see that record number 16 is the incomplete record.  
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| See, we are not getting any output apart from blank lines.
  
  
It has only 5 '''fields '''instead of 6.
+
For each record in the '''awkdemo.txt''' file, blank lines are getting printed in the output.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Retain the same screen
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us see one more example.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now, let us make the same changes in the '''do loop '''file.
 
+
 
+
How can we print the first and last '''field '''for each student regardless of how many '''fields''' there are?
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In do_loop.awk, assign i=4
  
 +
<nowiki>[code uploaded as do_loop_mod.awk]</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the file '''do_loop.awk'''
  
awk -F"|" '{print $1,$NF}' awkdemo.txt
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type the command as shown here on the '''terminal'''.
 
  
 +
Change the value of '''i '''from 1 to 4
  
  
 +
Save the file and switch to the '''terminal.'''
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight -F
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| awk -f do_loop.awk awkdemo.txt
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here we have used '''hyphen capital F''' option instead of setting '''FS''' '''variable'''.
+
  
 +
<nowiki>[Enter]</nowiki>
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press the Up arrow key until you get the command for '''do loop.'''
  
Press '''Enter'''.
 
  
|-
+
Now press '''Enter.'''
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the Output
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We get only the first and the last '''fields''' for each record in the file.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 8:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let’s try something else now.
+
  
  
Suppose, the student records are distributed across two files '''demo1.txt''' and '''demo2.txt'''.
 
  
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In the output, only the first field for each row is printed.
  
We want to print the first 3 lines from each of these two files.
 
  
 
+
What is the reason?
We can do this using '''NR variable'''.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show demo1.txt & demo2.txt in gedit
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight in code file
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here are the contents of the two files.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| For each row, '''awk''' first prints the value at the first''' field, '''because value of variable''' f '''is initialized to '''1.'''
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type:
 
  
<nowiki>awk 'NR<=3 {print NR, $0}' demo1.txt demo2.txt</nowiki>
+
Then the '''condition '''is checked.
  
  
<nowiki>[Enter]</nowiki>
+
Since the value of the '''loop counter i '''is 4, the '''condition '''is '''false'''.
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now to display the first 3 lines from each file, type the following command on the '''terminal.'''
+
  
  
Press '''Enter.'''
+
Hence, the '''loop '''is terminated there only, for that record.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight in code file
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The output shows only the first 3 records of '''demo1.txt''' file.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This '''loop''' will iterate for all the records in the '''awkdemo.txt''' file.
  
  
How can we print the same for the second file also?
+
Which means the first '''field''' for each record will get printed.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 8:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight output
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The solution is to use '''FNR''' instead of '''NR'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We are getting the output at least once for each record.
  
  
'''FNR''' is the '''current record number '''in the current file.  
+
Use the '''do-while loop, '''for a job to be executed at least once irrespective of any other '''condition'''.  
  
 
+
|-
'''FNR''' is incremented each time a new record is read.
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Retain same screen
 
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We can do the same with the '''for loop''' also.
 
+
It is reinitialized to zero each time a new input file is started.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 9:
+
| style="border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 9: for loop
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| But '''NR''' is the number of input records '''awk''' has processed since the starting of the program's execution.
+
| style="border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The syntax of '''for loop '''can be seen here.
  
  
It does to reset to zero with a new file.
+
The&nbsp;'''for&nbsp;statement''' starts by executing&nbsp;'''''initialization'''''.  
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In Terminal
 
  
 +
Then, as long as the'''&nbsp;''condition'''''&nbsp;is true, it repeatedly executes&nbsp;the '''''statements''' ''within&nbsp;and then&nbsp;'''''increments''.&nbsp;'''
  
Press up key
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to '''terminal.'''
 
  
 
+
Assuming your familiarity with a language like '''C''' or '''C++''', I am not explaining the syntax in detail.
Press the '''up arrow''' key to get the previous command.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <nowiki>awk ‘FNR<=3 {print NR,FNR, $0}’ demo1.txt demo2.txt</nowiki>
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| show for_loop.awk in Gedit
 
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This is how the '''for loop '''for this '''condition''' looks like.
<nowiki>[Enter]</nowiki>
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the previous command as follows.
+
  
  
Type '''FNR''' instead of '''NR.'''
+
Here, '''initialization, condition''' checking and variable incrementation are done in the same line.
  
  
In the '''Print section,''' next to '''NR,''' type '''FNR''' and press '''Enter.'''
+
Try this out by yourself.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output and highlight appropriately
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 10: looping constructs
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| See, we get the correct output now.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| There are some more''' looping constructs '''
  
 +
* '''break'''
 +
* '''continue'''
 +
* '''exit'''
  
'''FNR''' is set to zero with new file but '''NR''' keeps on increasing.
+
We will see some relevant examples on these in further tutorials.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 10:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show awkdemo_mod.txt
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us now look at some other '''built-in variables.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We may have a single and multiline '''comments''' in our file.
  
  
'''FILENAME''' variable gives the name of the file being read.
+
Here notice that the single line '''comments''' are declared with single '''hash '''(#) symbol'''.'''
  
  
'''ARGC''' specifies the number of '''arguments''' provided at the '''command line'''.
+
The multiline '''comments''' are declared with the help of double '''hash '''(##) symbol.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 11
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''ARGV''' represents an '''array '''that stores the '''command line arguments.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now, there is no point of checking and printing these '''comments''' in the output.  
  
  
'''ENVIRON''' specifies the '''array '''of the '''shell environment variables '''and corresponding values.
+
We have to skip the lines starting with '''hash '''(##) symbol.
  
  
As '''ARGV''' and '''ENVIRON''' use '''array''' in '''awk''', we will look at those in subsequent tutorials.
+
How can we do this?
  
 
|-
 
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Let us have a look at the variable '''FILENAME''' now.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Recall the case of giving 50% increment in the stipend for those who are getting more than 8000.
  
  
How can we print the name of the current file being processed?
+
We will use the same example for skipping the '''comments'''.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| show next.awk
  
awk '{print "We are processing input file " FILENAME}' awkdemo.txt
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the '''terminal''' and type the command as shown.
 
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight the space here
 
  
awk '{print "We are processing input file " FILENAME}' awkdemo.txt
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| I have created a file named''' next.awk '''as shown here for this execution.
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Here we have used '''space '''as a '''string concatenation operator.'''
+
  
 
+
|-
Press '''Enter''' to execute the command.
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Highlight''' $0~/^#/ {next;}'''
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Now, what does this command mean?
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output.
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 11: next
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The output shows the '''input filename '''multiple times.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''awk''' will search for the pattern, '''caret sign hash symbol(^#) '''at the beginning of each line.
  
  
This is because, this command prints the filename once for each row in the '''awkdemo.txt '''file.
+
If the pattern is found, the keyword '''next '''tells '''awk''' to skip the current line immediately.
  
  
How can we print this only once?
+
Then '''awk''' will start processing from the next line in the file.
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press Up arrow key
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Clear the '''terminal'''
 
  
 
+
This will save the processing time.
Press the '''up arrow '''key to get the previously executed command.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the command to become:
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type
  
awk 'END{print "We are processing input file " FILENAME}' awkdemo.txt
+
awk -F "|" -f next.awk awkdemo_mod.txt
  
 
<nowiki>[Enter]</nowiki>
 
<nowiki>[Enter]</nowiki>
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the previous command as shown here.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the '''terminal''' and type the command as shown and
  
 
+
press '''Enter.'''
Press '''Enter.'''
+
  
 
|-
 
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We get the filename only once.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We get the output without any '''comments.'''
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Retain same screen
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| There are some other '''built-in variables '''in '''awk'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Suppose, we have the students’ records in multiple files with the same format.
  
  
Please browse the internet to know more on them.
+
Say in '''awkdemo_mod.txt''' and '''awkdemo2.txt'''
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 12
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show awkdemo2.txt
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| See, it is similar to our previous file.
  
  
 +
It also has '''comments''' preceeded with '''hash <nowiki>#</nowiki> '''sysmbol.
  
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Suppose, we want to
 
  
* find the students who have passed and have stipend more than Rs.8000
+
And it has large text at the end with double '''hash #<nowiki>#</nowiki>''' symbol.
* use '''comma '''as the '''output field separator'''
+
* and print “'''The data is shown for file'''” and the name of file in the '''footer section'''.  
+
 
+
How can we do this?
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| awk 'BEGIN{FS="|"; OFS=","} $5=="Pass" && $6>8000 {print NR, $2, $5, $6} END{print "The data is shown for file " FILENAME }' awkdemo.txt
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 12: nextfile
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| So our data is in two different files.
  
  
<nowiki>[Enter]</nowiki>
+
So, '''awk''' should process both the files to give an increment to all the students.
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In the '''terminal''' type the following command
+
  
  
Press '''Enter'''.
+
Once we reach double '''hash'''(##) symbol of the first file,
  
|-
+
'''awk''' should stop processing that file entirely.
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output and highlight appropriately
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We can see that only one student has passed and gets stipend more than Rs.8000.
+
  
  
And the record number is 2.
+
Then it has to start the execution from the next file.
  
  
We can also see the name of the file in the '''footer''', as desired.
+
This will save the processing time.
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 13
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type '''$0~/^#/ {next;} '''after the begin statement.
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We can use '''awk''' for more and more complex tasks.
+
  
  
In that case, it becomes more difficult to write the '''commands''' every time on the '''terminal'''.
+
Highlight '''$0~/^#/ {nextfile;}'''
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Modify the '''next.awk '''as shown here.
  
  
We can instead write the '''awk''' program in a separate file.
+
I have added '''dollar zero tilde slash caret symbol double hash slash within curly brackets nextfile semicolon '''below the '''begin statement'''.
  
  
To be executable, that file should have the '''dot awk '''extension.
+
This will search for double '''hash #''' symbol at the beginning of a line.
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 14
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| While executing, we can just specify this '''awk''' program filename with the''' awk command.'''
 
  
 
+
If found, '''awk''' will skip the current file to process the next file.
For doing so, we need to use '''hyphen small f''' option.
+
 
+
 
+
Let us see an example.  
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show prog1.awk in gedit
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Press Ctrl+s
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| I have already written an '''awk '''program and saved it as '''prog1 dot awk.'''
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Save this file.
 
+
 
+
This code is also available in the '''Code Files''' link.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In the terminal show the last executed command
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type
  
 +
awk -F "|" -f next.awk awkdemo_mod.txt awkdemo2.txt
  
Highlight the portion in the last executed command and also in this program
+
<nowiki>[Enter]</nowiki>
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the '''terminal'''.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Switch to the the '''terminal '''and type the following command.
  
  
See what have we written inside '''single quotes''' of the '''command '''last executed?
+
'''awk space -F "|" space -f space next.awk space awkdemo_mod.txt space awkdemo2.txt'''
  
  
Content of '''prog1.awk''' file is exactly the same.
+
Press''' Enter'''
 
+
 
+
The only difference is that in the '''awk''' file, we have not written inside the '''single quotes.'''
+
 
+
|-
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Type:
+
 
+
<nowiki>awk -f prog1.awk awkdemo.txt [enter]</nowiki>
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| To execute the file, type the following on the '''terminal-'''
+
 
+
 
+
'''awk space hyphen small f space prog1.awk space awkdemo.txt '''and pres'''s Enter'''
+
  
 
|-
 
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Show the output
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| We are getting exactly the same output as we have seen before.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| See, we are getting the output from both the files.
 
+
 
+
So, this way you can write '''awk''' programs and use it multiple times.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 13: Summary
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This brings us to the end of this tutorial.
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| This brings us to the end of this tutorial.
  
Line 564: Line 504:
 
Let us summarize.
 
Let us summarize.
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 15
 
  
Summary
+
In this tutorial we learnt about-
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| In this tutorial we learnt about-
+
  
* '''Built-in variables'''
+
* '''while'''
* '''awk script'''
+
* '''do… while'''
 +
* '''for'''
 +
* '''next'''
 +
* '''nextfile '''in''' awk'''
  
using various examples.
 
  
|-
 
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 16
 
 
Assignment 1
 
 
 
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| As an assignment-
 
 
 
1. Write an '''awk''' script to print the last field of the 5th line in '''awkdemo.txt '''file.
 
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 17
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 14: Assignment
 +
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| As an assignment for the student records of '''awkdemo2.txt''',
  
Assignment 2
+
* print only even '''fields'''(i.e. '''field''' 2, '''field''' 4 etc.
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 1. Open the system file '''/etc/passwd '''on the '''terminal. '''
+
* irrespective of how many '''fields''' are there in the '''input file'''.
  
  
2. Identify all the '''separators '''therein.
 
 
 
3. Now write a '''script '''to process the file from the 20th line onwards.
 
 
 
4. That too, only for the lines that contain more than 6 fields.
 
 
 
5. You should print the '''line number''', entire line and count of '''fields '''in that particular line.
 
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 18
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 13:
  
 
About Spoken Tutorial project
 
About Spoken Tutorial project
Line 615: Line 534:
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 19
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 14:
  
 
Spoken Tutorial workshops
 
Spoken Tutorial workshops
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The&nbsp;Spoken Tutorial Project&nbsp;team conducts workshops using spoken tutorials and gives certificates.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The&nbsp;Spoken Tutorial Project&nbsp;team conducts workshops using spoken tutorials and gives certificates on passing online tests.
  
  
Line 624: Line 543:
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 20
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 15:
  
 
Forum for specific questions:
 
Forum for specific questions:
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Pls post your timed queries in this Forum.
+
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Do you have questions in THIS Spoken Tutorial?
  
|-
+
*
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 21
+
  
Acknowledgement
+
Please visit this site
 +
 
 +
|-
 +
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 16: Acknowledgement
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.
  
 +
*
  
More information on this mission is available at
+
More information on this mission is available at this link.
 
+
this link.
+
  
 
|-
 
|-
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  
+
| style="background-color:#ffffff;border-top:0.035cm solid #000001;border-bottom:0.035cm solid #000001;border-left:0.035cm solid #000001;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slide 17: Thanks
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The script has been contributed by Antara.
 
| style="background-color:#ffffff;border:0.035cm solid #000001;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| The script has been contributed by Antara.
  

Revision as of 19:28, 31 January 2018

Title of script: Loops in Awk

Author: Antara Roy Choudhury

Keywords: for, while, do-while, next, nextfile


Visual Cue
Narration
Slide 1: Introduction Welcome to this spoken tutorial on Loops in awk.
Slide 2: Learning Objective In this tutorial we will learn about-


  • while
  • do-while
  • for
  • and more looping constructs in awk

We will do this through some examples.

Slide 3: System requirement To record this tutorial, I am using
  • Ubuntu Linux 16.04 OS and
  • gedit text editor 3.20.1
  • You can use any text editor of your choice.


Slide 4: Pre-requisite * To practice this tutorial, you should have gone through the previous awk tutorials on our website.
  • You should have familiarity with any programming language like C or C++
  • If not, then please go through the corresponding tutorials on our website.


Slide 5: Code Files The files used in this tutorial are available in the Code Files link on this tutorial page.


Please download and extract them.

Slide 6: Loops A loop allows us to perform one or more actions repeatedly.


while, do-while and for are the available loops in awk.

Slide 7: While loop The syntax of while loop can be seen here.


While loop first checks whether the specified condition is true.

If yes, then it executes the code within the body.


This loop will be repeated as long as the specified while condition is true.

Show awkdemo.txt file in gedit We will use the same awkdemo.txt file, that we have used earlier.
show while_loop.awk in Gedit I have already written a script named while_loop.awk


The same is available in the Code Files link of this tutorial.

Point to the pipe symbol Here we have set the field separator as Pipe symbol.
Highlight i = 1 Initially, we have to the set the value of the loop variable i as 1.



Highlight f = 1 Here, we have taken one more variable f and initialized it to 1.


Variable f represents the field counter or the position of the fields for each record.

Highlight while (i<=3) Now, in the while condition, we check if i is less than or equal to 3.
Highlight Print statement If yes, then it will print the value in the fth field, for that record in awkdemo.txt file.
Highlight f++ Then we will increase the field counter f by 1.
Highlight i++ After that, we will also increment the value of loop variable i by 1.
Highlight printf("\n") This printf is for printing a newline character at the end of each row.
This loop will be executed for all the records in the awkdemo.txt file.


Which means the first 3 fields will get printed for each record.

Let’s execute this code now.
Open the terminal Open the terminal by pressing Ctrl, Alt and T keys.
cd /<saved folder> Go to the folder in which you downloaded and extracted the Code Files using cd command
Type in terminal:

awk -f while_loop.awk awkdemo.txt

[Enter]

Now type:

awk space minus small f while_loop.awk space awkdemo.txt


Press Enter

Show the output Observe that we get the first three fields of all the rows in the output.
Let us do the same with the do-while loop.
Slide 8: Do While Loop The syntax of do-while loop can be seen here.


The do-while loop always executes the code inside the body once.


Then it checks the specified condition.


And repeats the code inside the body, as long as the specified condition is true.

show do_loop.awk in Gedit I have already written a script and named it as do_loop.awk


The same is available in the Code Files link.

Show the contents and highlight appropriately In this code, these are the statements within the do loop, which will be executed first.


This is the condition that will be checked.


After that, the statements inside the loop will be executed repeatedly as long as the condition is true.

This loop will iterate for all the records in the awkdemo.txt file


Which means the first 3 fields will get printed for all the records.

Type:

awk -f do_loop.awk awkdemo.txt

[Enter]

Let’s switch to the terminal. Let me clear the terminal.

Now type:


awk space -f do_loop.awk awkdemo.txt

and press Enter

Scroll up and show outputs of while loop and do loop We get the same output.


Then why do we have both while and do-while loops?

Let us understand the difference.

In while_loop.awk, assign i=4

[code uploaded as while_loop_mod.awk]

Switch to the file while_loop.awk


Now, change the value of loop counter i from 1 to 4.

Highlight

(i<=3)

This will make the specified condition false from the beginning.


So this means, we should not get any output.

Press Ctrl+S Save the file and switch to terminal.
awk -f while_loop.awk awkdemo.txt

[Enter]

Press up key until you get the command for executing the while loop.


Now press Enter.

Show the output See, we are not getting any output apart from blank lines.


For each record in the awkdemo.txt file, blank lines are getting printed in the output.

Now, let us make the same changes in the do loop file.
In do_loop.awk, assign i=4

[code uploaded as do_loop_mod.awk]

Switch to the file do_loop.awk


Change the value of i from 1 to 4


Save the file and switch to the terminal.

awk -f do_loop.awk awkdemo.txt

[Enter]

Press the Up arrow key until you get the command for do loop.


Now press Enter.

Show the output


In the output, only the first field for each row is printed.


What is the reason?

Highlight in code file For each row, awk first prints the value at the first field, because value of variable f is initialized to 1.


Then the condition is checked.


Since the value of the loop counter i is 4, the condition is false.


Hence, the loop is terminated there only, for that record.

Highlight in code file This loop will iterate for all the records in the awkdemo.txt file.


Which means the first field for each record will get printed.

Highlight output We are getting the output at least once for each record.


Use the do-while loop, for a job to be executed at least once irrespective of any other condition.

Retain same screen We can do the same with the for loop also.
Slide 9: for loop The syntax of for loop can be seen here.


The for statement starts by executing initialization.


Then, as long as the condition is true, it repeatedly executes the statements within and then increments


Assuming your familiarity with a language like C or C++, I am not explaining the syntax in detail.

show for_loop.awk in Gedit This is how the for loop for this condition looks like.


Here, initialization, condition checking and variable incrementation are done in the same line.


Try this out by yourself.

Slide 10: looping constructs There are some more looping constructs
  • break
  • continue
  • exit

We will see some relevant examples on these in further tutorials.

Show awkdemo_mod.txt We may have a single and multiline comments in our file.


Here notice that the single line comments are declared with single hash (#) symbol.


The multiline comments are declared with the help of double hash (##) symbol.

Highlight Now, there is no point of checking and printing these comments in the output.


We have to skip the lines starting with hash (##) symbol.


How can we do this?

Recall the case of giving 50% increment in the stipend for those who are getting more than 8000.


We will use the same example for skipping the comments.

show next.awk


I have created a file named next.awk as shown here for this execution.
Highlight $0~/^#/ {next;} Now, what does this command mean?
Slide 11: next awk will search for the pattern, caret sign hash symbol(^#) at the beginning of each line.


If the pattern is found, the keyword next tells awk to skip the current line immediately.


Then awk will start processing from the next line in the file.


This will save the processing time.

Type

awk -F "|" -f next.awk awkdemo_mod.txt

[Enter]

Switch to the terminal and type the command as shown and

press Enter.

Show the output We get the output without any comments.
Suppose, we have the students’ records in multiple files with the same format.


Say in awkdemo_mod.txt and awkdemo2.txt

Show awkdemo2.txt See, it is similar to our previous file.


It also has comments preceeded with hash # sysmbol.


And it has large text at the end with double hash ## symbol.

Slide 12: nextfile So our data is in two different files.


So, awk should process both the files to give an increment to all the students.


Once we reach double hash(##) symbol of the first file,

awk should stop processing that file entirely.


Then it has to start the execution from the next file.


This will save the processing time.

Type $0~/^#/ {next;} after the begin statement.


Highlight $0~/^#/ {nextfile;}

Modify the next.awk as shown here.


I have added dollar zero tilde slash caret symbol double hash slash within curly brackets nextfile semicolon below the begin statement.


This will search for double hash # symbol at the beginning of a line.


If found, awk will skip the current file to process the next file.

Press Ctrl+s Save this file.
Type

awk -F "|" -f next.awk awkdemo_mod.txt awkdemo2.txt

[Enter]

Switch to the the terminal and type the following command.


awk space -F "|" space -f space next.awk space awkdemo_mod.txt space awkdemo2.txt


Press Enter

Show the output See, we are getting the output from both the files.
Slide 13: Summary This brings us to the end of this tutorial.


Let us summarize.


In this tutorial we learnt about-

  • while
  • do… while
  • for
  • next
  • nextfile in awk


Slide 14: Assignment As an assignment for the student records of awkdemo2.txt,
  • print only even fields(i.e. field 2, field 4 etc.
  • irrespective of how many fields are there in the input file.


Slide 13:

About Spoken Tutorial project

The video at the following link summarises the Spoken Tutorial project.


Please download and watch it.

Slide 14:

Spoken Tutorial workshops

The Spoken Tutorial Project team conducts workshops using spoken tutorials and gives certificates on passing online tests.


For more details, please write to us.

Slide 15:

Forum for specific questions:

Do you have questions in THIS Spoken Tutorial?

Please visit this site

Slide 16: Acknowledgement Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.

More information on this mission is available at this link.

Slide 17: Thanks The script has been contributed by Antara.


And this is Praveen from IIT Bombay signing off.


Thanks for joining.

Contributors and Content Editors

Antarade, Nancyvarkey