Linux-AWK/C2/Built-in-Variables-in-awk/English
Title of script: Built-in variables and awk Script
Author: Antara Roy Choudhury
Keywords: Built-in variables, RS, ORS, NR, NF, FS, OFS, FILENAME
|
|
Slide 1: Introduction | Welcome to the spoken tutorial on awk built-in variables and awk script. |
Slide 2: Learning Objectives | In this tutorial we will learn about
We will do this through some examples. |
Slide 3a: System requirement | To record this tutorial, I am using
|
Slide 3b: Code Files | The files used in this tutorial are available in the Code Files link on this tutorial page.
|
Slide 4: Prerequisite | To practice this tutorial, you should have gone through the earlier awk tutorials on this website.
|
Slide 5: awk built-in variables | First, let us see some of the built-in variables in awk.
By default, the value of FS is a whitespace. |
Slide 5: awk built-in variables |
By default, it is newline.
By default, it is whitespace.
|
Show awkdemo.txt in Gedit | Let us have a look at the awkdemo file now.
|
Highlight appropriately | Observe that all the records are separated from each other by a newline character.
|
Highlight vertical bar | character | Notice that all the fields are separated by the pipe symbol.
|
Slide 6: How to reset value of FS variable? | By default, any number of spaces or a tabs separate the fields.
|
Let us do this through an example.
| |
Open the terminal | Open the terminal by pressing CTRL, ALT and T keys. |
cd /<saved folder> | Go to the folder in which you downloaded and extracted the Code Files using cd command. |
awk 'BEGIN{FS="|"} $6>5000 {print $2,$6}' awkdemo.txt | Type the command as shown here. |
Highlight {FS="|"} area | Here in the BEGIN section, we have assigned the value of FS as a pipe symbol.
|
[Enter] | Press Enter to execute the command. |
Show the output | The output shows the list of students who are receiving more than Rs.5000 as a stipend. |
Highlight appropriately | Here the name field and the stipend field is separated by a blank space.
|
Slide
|
Suppose we want colon as the output field separator.
|
In terminal | In the terminal, press the up arrow key to get the previously executed command. |
Modify the previous command
[Enter] |
Modify the command as shown here.
|
Show the output | We get the output in the desired format. |
Show sample.txt
|
Now, suppose our new input file is sample.txt.
|
How can we extract the roll no. and name information from this file?
| |
Slide 7: | Next, let us see other built-in variables.
|
Slide | Let us see one example of this.
|
Switch to the terminal and clear it | Switch to the terminal.
|
Type
awk 'BEGIN{FS="|"} NF !=6 {print NR, $0}' awkdemo.txt [Enter] |
Type the command as shown.
|
Show the command and highlight appropriate areas as per narration | As the fields are separated by pipe symbol, set the FS value to pipe symbol in the BEGIN section.
Press Enter. |
Show the output | In the output, we can see that record number 16 is the incomplete record.
|
Retain the same screen | Let us see one more example.
|
Type:
|
Type the command as shown here on the terminal.
|
Highlight -F | Here we have used hyphen capital F option instead of setting FS variable.
|
Show the Output | We get only the first and the last fields for each record in the file. |
Slide 8: | Let’s try something else now.
|
Show demo1.txt & demo2.txt in gedit | Here are the contents of the two files. |
Type:
awk 'NR<=3 {print NR, $0}' demo1.txt demo2.txt
|
Now to display the first 3 lines from each file, type the following command on the terminal.
|
Show the output | The output shows only the first 3 records of demo1.txt file.
|
Slide 8: | The solution is to use FNR instead of NR.
|
Slide 9: | But NR is the number of input records awk has processed since the starting of the program's execution.
|
In Terminal
|
Switch to terminal.
|
awk ‘FNR<=3 {print NR,FNR, $0}’ demo1.txt demo2.txt
[Enter] |
Modify the previous command as follows.
|
Show the output and highlight appropriately | See, we get the correct output now.
|
Slide 10: | Let us now look at some other built-in variables.
|
Slide 11 | ARGV represents an array that stores the command line arguments.
|
Let us have a look at the variable FILENAME now.
| |
Type:
awk '{print "We are processing input file " FILENAME}' awkdemo.txt |
Switch to the terminal and type the command as shown. |
Highlight the space here
awk '{print "We are processing input file " FILENAME}' awkdemo.txt |
Here we have used space as a string concatenation operator.
|
Show the output. | The output shows the input filename multiple times.
|
Press Up arrow key | Clear the terminal
|
Modify the command to become:
awk 'END{print "We are processing input file " FILENAME}' awkdemo.txt [Enter] |
Modify the previous command as shown here.
|
Show the output | We get the filename only once. |
Retain same screen | There are some other built-in variables in awk.
|
Slide 12
|
Suppose, we want to
How can we do this? |
awk 'BEGIN{FS="|"; OFS=","} $5=="Pass" && $6>8000 {print NR, $2, $5, $6} END{print "The data is shown for file " FILENAME }' awkdemo.txt
|
In the terminal type the following command
|
Show the output and highlight appropriately | We can see that only one student has passed and gets stipend more than Rs.8000.
|
Slide 13 | We can use awk for more and more complex tasks.
|
Slide 14 | While executing, we can just specify this awk program filename with the awk command.
|
Show prog1.awk in gedit | I have already written an awk program and saved it as prog1 dot awk.
|
In the terminal show the last executed command
|
Switch to the terminal.
|
Type:
awk -f prog1.awk awkdemo.txt [enter] |
To execute the file, type the following on the terminal-
|
Show the output | We are getting exactly the same output as we have seen before.
|
This brings us to the end of this tutorial.
| |
Slide 15
Summary |
In this tutorial we learnt about-
using various examples. |
Slide 16
Assignment 1
|
As an assignment-
|
Slide 17
Assignment 2 |
1. Open the system file /etc/passwd on the terminal.
|
Slide 18
About Spoken Tutorial project |
The video at the following link summarises the Spoken Tutorial project.
|
Slide 19
Spoken Tutorial workshops |
The Spoken Tutorial Project team conducts workshops using spoken tutorials and gives certificates.
|
Slide 20
Forum for specific questions: |
Pls post your timed queries in this Forum. |
Slide 21
Acknowledgement |
Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.
this link. |
The script has been contributed by Antara.
|