Difference between revisions of "Linux-AWK/C2/Loops-in-awk/English"

• Built-in variables
• awk script

We will do this through some examples.

• Ubuntu Linux 16.04 OS and
• gedit text editor version 3.20.1

Please download and use them.

If not, then please go through the corresponding tutorials on this website.

• Capital RS specifies the record separator in an input file. By default, it is newline.
• Capital FS specifies the field separator in an input file.

By default, the value of FS is a whitespace.

By default, it is newline.

• Capital OFS defines the output field separator.

By default, it is whitespace.

Let us understand the meaning of each of these.

When we are processing this awkdemo file with awk command, this becomes our input file.

newline is the default value for record separator RS variable.

So, there is no need to do anything else.

How can we inform awk about it?

Let us see.

We can reset this with the help of hyphen capital F option as learnt in our earlier tutorials.

Or else, we can reset this in the BEGIN section with the use of FS variable.

Suppose, I want to find out the name of students who are getting a stipend of more than Rs.5000.

Similarly, we can modify RS variable.

Also, all the records are separated by a newline character.

And double newline as output record separator.

How can we do this? Let us see.

awk 'BEGIN{FS="|";OFS=":"; ORS="\n\n"} $6>5000 {print $2, $6}' awkdemo.txt

[Enter]

And then press Enter.

Point or highlight

Observe that the field separator here is newline and record separator is double newline.

Yes, you have guessed correctly.

We have to modify both FS and RS variables.

Pause this tutorial and do this as an assignment.

• Capital NR gives us the Number of Records processed by awk
• Capital NF gives the Number of Fields in the current record

Suppose, we want to find incomplete lines in the file.

Here, incomplete line means it has less than the normal 6 fields.

Let me clear the terminal using Ctrl and L keys

awk 'BEGIN{FS="|"} NF !=6 {print NR, $0}' awkdemo.txt

[Enter]

Next we have written NF!=6.

This checks whether the number of fields in the current line, is not equal to 6.

If true, then

• print section will print the record’s line number NR,
• along with the entire line denoted by $0.

Press Enter.

It has only 5 fields instead of 6.

How can we print the first and last field for each student regardless of how many fields there are?

awk -F"|" '{print $1,$NF}' awkdemo.txt

Press Enter.

Suppose, the student records are distributed across two files demo1.txt and demo2.txt.

We want to print the first 3 lines from each of these two files.

We can do this using NR variable.

awk 'NR<=3 {print NR, $0}' demo1.txt demo2.txt

[Enter]

Press Enter.

How can we print the same for the second file also?

FNR is the current record number in the current file.

FNR is incremented each time a new record is read.

It is reinitialized to zero each time a new input file is started.

It does to reset to zero with a new file.

Press up key

Press the up arrow key to get the previous command.

[Enter]

Type FNR instead of NR.

In the Print section, next to NR, type FNR and press Enter.

FNR is set to zero with new file but NR keeps on increasing.

FILENAME variable gives the name of the file being read.

ARGC specifies the number of arguments provided at the command line.

ENVIRON specifies the array of the shell environment variables and corresponding values.

As ARGV and ENVIRON use array in awk, we will look at those in subsequent tutorials.

How can we print the name of the current file being processed?

awk '{print "We are processing input file " FILENAME}' awkdemo.txt

awk '{print "We are processing input file " FILENAME}' awkdemo.txt

Press Enter to execute the command.

This is because, this command prints the filename once for each row in the awkdemo.txt file.

How can we print this only once?

Press the up arrow key to get the previously executed command.

awk 'END{print "We are processing input file " FILENAME}' awkdemo.txt

[Enter]

Press Enter.

Please browse the internet to know more on them.

• find the students who have passed and have stipend more than Rs.8000
• use comma as the output field separator
• and print “The data is shown for file” and the name of file in the footer section.

How can we do this?

[Enter]

Press Enter.

And the record number is 2.

We can also see the name of the file in the footer, as desired.

In that case, it becomes more difficult to write the commands every time on the terminal.

We can instead write the awk program in a separate file.

To be executable, that file should have the dot awk extension.

For doing so, we need to use hyphen small f option.

Let us see an example.

This code is also available in the Code Files link.

Highlight the portion in the last executed command and also in this program

See what have we written inside single quotes of the command last executed?

Content of prog1.awk file is exactly the same.

The only difference is that in the awk file, we have not written inside the single quotes.

awk -f prog1.awk awkdemo.txt [enter]

awk space hyphen small f space prog1.awk space awkdemo.txt and press Enter

So, this way you can write awk programs and use it multiple times.

Let us summarize.

Summary

• Built-in variables
• awk script

using various examples.

Assignment 1

1. Write an awk script to print the last field of the 5th line in awkdemo.txt file.

Assignment 2

2. Identify all the separators therein.

3. Now write a script to process the file from the 20th line onwards.

4. That too, only for the lines that contain more than 6 fields.

5. You should print the line number, entire line and count of fields in that particular line.

About Spoken Tutorial project

Please download and watch it.

Spoken Tutorial workshops

For more details, please write to us.

Forum for specific questions:

Acknowledgement

More information on this mission is available at

this link.

And this is Praveen from IIT Bombay signing off.

Thanks for joining.