Difference between revisions of "Linux-AWK/C2/Variables-and-Operators-in-awk/English-timed"

From Script | Spoken-Tutorial
Jump to: navigation, search
Line 175: Line 175:
 
|-
 
|-
 
| 04:00
 
| 04:00
−
| Similarly, when we '''concatenate''' small '''x,''' which is a number and '''string b''', '''x''' is auto-converted into '''string'''.  
+
| Similarly, when we '''concatenate''' small '''x''' which is a number and '''string b''', '''x''' is auto-converted into '''string'''.  
−
 
+
And the concatenated output becomes '''1tutorial. '''
−
And the '''concatenated''' output becomes '''1tutorial. '''
+
  
 
|-
 
|-
Line 185: Line 184:
 
|-
 
|-
 
| 04:16
 
| 04:16
−
| That's because '''awk '''finds a '''string concatenation operator space '''here between '''x''' and '''b'''.
+
| That's because '''awk '''finds a '''string concatenation operator space '''here, between '''x''' and '''b'''.
  
 
|-
 
|-
 
| 04:25
 
| 04:25
−
| Now, look at the output of small '''x plus '''capital '''X'''.  
+
| Now, look at the output of small '''x''' plus capital '''X'''.  
  
−
Here, we have the '''arithmetic operator plus. '''
+
Here, we have the arithmetic operator '''plus'''.
  
 
|-
 
|-
Line 227: Line 226:
 
|-
 
|-
 
| 05:14
 
| 05:14
−
| Only exception is the '''string matching operator, '''which may be new to you.  
+
| Only exception is the '''string matching operator '''which may be new to you.  
  
 
Let us understand this with an example.
 
Let us understand this with an example.
Line 239: Line 238:
 
|-
 
|-
 
| 05:31
 
| 05:31
−
| Switch to the '''terminal'''
+
| Switch to the '''terminal'''.
  
 
Let us end the previous process by pressing '''Ctrl''' and '''D''' keys.
 
Let us end the previous process by pressing '''Ctrl''' and '''D''' keys.
Line 249: Line 248:
 
|-
 
|-
 
| 05:41
 
| 05:41
−
| Now go to the folder in which you saved the '''awkdemo.txt''' file using the '''cd''' command.
+
| Now, go to the '''folder''' in which you have saved the '''awkdemo.txt''' file using the '''cd''' command.
  
 
|-
 
|-
Line 261: Line 260:
 
|-
 
|-
 
| 05:58
 
| 05:58
−
| In this case, we need to compare two different fields.  
+
| In this case, we need to compare two different '''field'''s.  
  
 
|-
 
|-
Line 269: Line 268:
 
|-
 
|-
 
| 06:07
 
| 06:07
−
| These '''operators''' can compare both '''strings''' and '''numbers'''.
+
| These '''operators''' can compare both '''strings''' and numbers.
  
 
|-
 
|-
 
| 06:12
 
| 06:12
−
| So, on the '''terminal''' type
+
| So, on the '''terminal''', type:
  
−
'''''awk space hyphen capital F within double quotes vertical bar space Within single quotes dollar 5 equal to equal to within double quotes Pass space ampersand ampersand space dollar 4 less than 80 space within curly braces print space plus plus x coma dollar 2 coma dollar 4 coma dollar 5 space awkdemo.txt and press Enter.'''''
+
'''''awk space hyphen capital F within double quotes vertical bar space within single quotes dollar 5 equal to equal to within double quotes Pass space ampersand ampersand space dollar 4 less than 80 space within curly braces print space plus plus x comma dollar 2 comma dollar 4 comma dollar 5 space awkdemo.txt''''' and press '''Enter'''.
  
 
|-
 
|-
 
| 06:54
 
| 06:54
−
| This command shows a number of things.  
+
| This '''command''' shows a number of things.  
  
−
One, we compare a '''string '''with the fifth field.
+
One, we compare a '''string '''with the fifth '''field'''.
  
 
|-
 
|-
Line 301: Line 300:
 
|-
 
|-
 
| 07:27
 
| 07:27
−
| Now suppose, we want to find computer science students who have passed.  
+
| Now suppose, we want to find '''computer science''' students who have passed.  
  
 
|-
 
|-
Line 309: Line 308:
 
|-
 
|-
 
| 07:40
 
| 07:40
−
| We would type
+
| We would type:
  
−
'''''awk space hyphen capital F within double quotes pipe symbol space within single quote dollar 5 equal to equal to within double quotes Pass ampersand ampersand space dollar 3 tilde slash within square brackets small c capital C computers slash space within curly braces print space plus plus small x comma dollar 2 comma dollar 3 coma dollar 5 space awkdemo.txt and press Enter.'''''
+
'''''awk space hyphen capital F within double quotes pipe symbol space within single quote dollar 5 equal to equal to within double quotes Pass ampersand ampersand space dollar 3 tilde slash within square brackets small c capital C computers slash space within curly braces print space plus plus small x comma dollar 2 comma dollar 3 comma dollar 5 space awkdemo.txt''''' and press '''Enter'''.
  
 
|-
 
|-
Line 319: Line 318:
 
|-
 
|-
 
| 08:30
 
| 08:30
−
| Say now we want a list of all non-computer students who have passed.  
+
| Say, now we want a list of all non-computer students who have passed.  
  
 
|-
 
|-
 
| 08:35
 
| 08:35
−
| Use the up arrow to get the previous command.
+
| Use the up-arrow to get the previous command.
  
 
|-
 
|-
 
| 08:39
 
| 08:39
−
| Next to '''dollar 3''' add '''exclamation symbol''' and press '''Enter.'''
+
| Next to '''dollar 3''' add exclamation symbol and press '''Enter.'''
  
 
|-
 
|-
Line 341: Line 340:
 
|-
 
|-
 
| 09:00
 
| 09:00
−
| Now to count the number of empty lines using '''awk''', type
+
| Now to count the number of empty lines using '''awk''', type:
  
 
'''''awk space within single quote within front slash caret symbol dollar space within curly braces x equal to x plus 1 semicolon space print x space awkdemo.txt'' '''
 
'''''awk space within single quote within front slash caret symbol dollar space within curly braces x equal to x plus 1 semicolon space print x space awkdemo.txt'' '''
Line 353: Line 352:
 
|-
 
|-
 
| 09:30
 
| 09:30
−
| The caret sign signifies start of a line while dollar signifies the end of a line.  
+
| The '''caret''' sign signifies start of a line while dollar signifies the end of a line.  
  
 
|-
 
|-
 
| 09:37
 
| 09:37
−
| Hence an empty line would be matched by the '''regular expression caret-dollar.'''
+
| Hence, an empty line would be matched by the '''regular expression caret-dollar.'''
  
 
|-
 
|-
Line 369: Line 368:
 
| This command gives us the running count of blank lines.
 
| This command gives us the running count of blank lines.
  
−
This is because every time a blank line is found, '''x '''would be incremented and then printed.
+
This is because, every time a blank line is found, '''x '''would be incremented and then printed.
  
 
|-
 
|-
Line 375: Line 374:
 
| In our last command, we have seen running count of blank lines.
 
| In our last command, we have seen running count of blank lines.
  
−
But say we only want to print the total number of blank lines.
+
But, say we only want to print the total number of blank lines.
  
 
|-
 
|-
Line 387: Line 386:
 
|-
 
|-
 
| 10:25
 
| 10:25
−
| For such requirements '''awk '''provides the '''BEGIN '''and the '''END '''sections.
+
| For such requirements, '''awk '''provides the '''BEGIN '''and the '''END '''sections.
  
 
|-
 
|-
Line 395: Line 394:
 
|-
 
|-
 
| 10:34
 
| 10:34
−
| This section is executed before the '''main input loop '''is executed.  
+
| This section is '''execute'''d before the '''main input loop '''is executed.  
  
 
|-
 
|-
Line 411: Line 410:
 
| Let's learn how to do this.  
 
| Let's learn how to do this.  
  
−
In the terminal type
+
In the terminal, type:
  
−
'''''awk space opening single quote BEGIN incaps within curly brace print space within double quotes The number of empty lines in awkdemo are ''''' press''' Enter.'''
+
'''''awk space opening single quote BEGIN in caps within curly brace print space within double quotes The number of empty lines in awkdemo are ''''' press''' Enter.'''
  
 
|-
 
|-
Line 423: Line 422:
 
|-
 
|-
 
| 11:26
 
| 11:26
−
| '''''end space within curly braces print space x close single quote space awkdemo.txt and press Enter.'''''
+
| '''''end space within curly braces print space x close single quote space awkdemo.txt''''' and press '''Enter'''.
  
 
|-
 
|-
Line 429: Line 428:
 
| See, we did not get the desired output!  
 
| See, we did not get the desired output!  
  
−
We should get the output as 3, because we have 3 blank lines in the file.  
+
We should get the output as 3 because we have 3 blank lines in the file.  
  
 
|-
 
|-
Line 439: Line 438:
 
|-
 
|-
 
| 11:54
 
| 11:54
−
| So, let us modify the command.  
+
| So, let us modify the '''command'''.  
  
 
|-
 
|-
 
| 11:57
 
| 11:57
−
| Press the up arrow key to get the previous executed command on the '''terminal'''.
+
| Press the up-arrow key to get the previous executed command on the '''terminal'''.
  
 
|-
 
|-
 
| 12:03
 
| 12:03
−
| Now modify lower case '''end '''to upper case '''END. '''
+
| Now, modify lower case '''end '''to upper case '''END. '''
  
 
And press''' Enter.'''
 
And press''' Enter.'''
Line 473: Line 472:
 
|-
 
|-
 
| 12:40
 
| 12:40
−
| In this tutorial we learnt about
+
| In this tutorial, we learnt about:
  
−
'''User defined variables '''in '''awk'''
+
'''User defined variables '''in '''awk''',
  
 
|-
 
|-
 
| 12:45
 
| 12:45
−
|  '''Operators '''
+
|  '''Operators, '''
  
−
'''BEGIN and END statements'''
+
'''BEGIN''' and '''END statement'''s.
  
 
|-
 
|-
 
| 12:49
 
| 12:49
−
| As an assignment print every line where the value of the last field is more than 5000  
+
| As an assignment, print every line where the value of the last '''field''' is more than 5000.
  
−
And the student belongs to '''Electrical department'''.
+
And, the student belongs to '''Electrical department'''.
  
 
|-
 
|-
Line 495: Line 494:
 
|-
 
|-
 
| 13:07
 
| 13:07
−
| The video at the following link summarises the Spoken Tutorial project.
+
| The video at the following link summarises the '''Spoken Tutorial''' project.
  
 
Please download and watch it.
 
Please download and watch it.
Line 539: Line 538:
 
|-
 
|-
 
| 13:59
 
| 13:59
−
| Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.
+
| Spoken Tutorial Project is funded by '''NMEICT, MHRD''', Government of India.
  
 
More information on this mission is available at this link.
 
More information on this mission is available at this link.
Line 545: Line 544:
 
|-
 
|-
 
|14:10
 
|14:10
−
| The script has been contributed by Antara and this is Praveen from IIT Bombay signing off.
+
| The script has been contributed by Antara and this is Praveen from '''IIT Bombay''', signing off.
  
 
Thank you for joining.
 
Thank you for joining.
  
 
|}
 
|}

Revision as of 20:39, 9 July 2019

Time
Narration
00:01 Hello and welcome to this spoken tutorial on Variables and operators in awk command.
00:07 In this tutorial, we will learn about-

User defined variables,

00:12 Operators,

BEGIN and END statements.

00:17 We will do this through some examples.
00:20 To record this tutorial, I am using Ubuntu Linux 16.04
00:26 To practice this tutorial, you should have gone through the earlier Linux tutorials on this website.
00:33 You should be familiar with basic operators used in general programming languages like C or C++.
00:41 If not, then please go through the corresponding tutorials on our website.
00:47 awk combines the power of a filter and a programming language.
00:52 So, it supports variables, constants, operators etc.
00:58 Let’s see what a variable in awk is.
01:02 A variable is an identifier that references a value.
01:07 Awk supports both user-defined variables and built-in variables.
01:12 We will learn about user-defined variables in this tutorial.
01:17 For user-defined variables, variable declaration is not required.
01:22 Variables do not have to be initialized explicitly.
01:26 Awk automatically initializes them to zero or null string.
01:32 A variable must begin with a letter and continue with letters, digits and underscores.

Variables are case-sensitive.

01:43 So, Salary with capital “S” and salary with small “s” are two different variables.
01:50 Let us look at some examples now.
01:53 Open the terminal by pressing CTRL + ALT and T keys.
01:58 On the terminal, type- awk space opening single quote opening curly brace small x equal to 1 semicolon capital X equal to within double quotes capital A semicolon small a equal to within double quotes awk semicolon small b equal to within double quotes tutorial.

Press Enter.

02:25 Type print x, press Enter.
02:29 print capital X, press Enter.
02:34 print a, press Enter.
02:37 print b, press Enter.
02:40 print a space b, press Enter.
02:44 print small x space b, press Enter.
02:49 print small x plus capital X closing curly brace closing single quote and press Enter.
02:57 Since we have not given a filename, awk would need some input from standard input.
03:03 And hence, we can type any letter, say a and then press Enter.
03:10 This example shows a couple of things.

Variables can be initialized with a number.

03:18 It can also be initialized with value as a single character or a string.
03:23 If the value is a character or a string, variable is initialized with value within double quotes.
03:31 We can see the values of the variables.
03:35 Observe that small x and capital X are treated as different variables.
03:41 This proves that variables are case sensitive.
03:45 Also, it shows how two strings can be concatenated.
03:50 Here variables small a and small b are concatenated.
03:55 So, string concatenation operator is simply a space.
04:00 Similarly, when we concatenate small x which is a number and string b, x is auto-converted into string.

And the concatenated output becomes 1tutorial.

04:13 Why does the auto-conversion to string happen?
04:16 That's because awk finds a string concatenation operator space here, between x and b.
04:25 Now, look at the output of small x plus capital X.

Here, we have the arithmetic operator plus.

04:33 So, X is auto-converted to numeric zero.

And the addition output becomes numeric 1.

04:42 Until now, we have seen a couple of operators.

Let’s look at what other operators we can use.

04:49 A variety of operators can be used in expressions.
04:53 Please pause the video here and take a look at all the operators mentioned here.
04:58 I assume you are familiar with these basic operators.
05:02 If not, then kindly visit our website for tutorials on operators in C and C++ series.
05:09 I am not going to discuss the working of all these operators in detail.
05:14 Only exception is the string matching operator which may be new to you.

Let us understand this with an example.

05:23 A file named awkdemo.txt has been provided in the Code files link.

Please download it on your computer.

05:31 Switch to the terminal.

Let us end the previous process by pressing Ctrl and D keys.

05:38 Let me clear the terminal.
05:41 Now, go to the folder in which you have saved the awkdemo.txt file using the cd command.
05:48 Let us have a look at this file now.
05:52 Let's say we want to find the students who have passed but have marks less than 80.
05:58 In this case, we need to compare two different fields.
06:02 For such situations, we can use awk's relational operators.
06:07 These operators can compare both strings and numbers.
06:12 So, on the terminal, type:

awk space hyphen capital F within double quotes vertical bar space within single quotes dollar 5 equal to equal to within double quotes Pass space ampersand ampersand space dollar 4 less than 80 space within curly braces print space plus plus x comma dollar 2 comma dollar 4 comma dollar 5 space awkdemo.txt and press Enter.

06:54 This command shows a number of things.

One, we compare a string with the fifth field.

07:01 Second, we only compare the fourth field with a number.
07:06 Third, we see that we can join two or more comparisons using ampersand operator.
07:13 Instead of specific numbers or strings, we can also compare regular expressions.
07:19 As we have seen in the slide, we have the tilde and the exclamation tilde operators for this purpose.
07:27 Now suppose, we want to find computer science students who have passed.
07:32 Since computers can have both a small and capital C, we would use a regular expression.
07:40 We would type:

awk space hyphen capital F within double quotes pipe symbol space within single quote dollar 5 equal to equal to within double quotes Pass ampersand ampersand space dollar 3 tilde slash within square brackets small c capital C computers slash space within curly braces print space plus plus small x comma dollar 2 comma dollar 3 comma dollar 5 space awkdemo.txt and press Enter.

08:24 If we want to negate the comparison, we can do so using the exclamation tilde operator.
08:30 Say, now we want a list of all non-computer students who have passed.
08:35 Use the up-arrow to get the previous command.
08:39 Next to dollar 3 add exclamation symbol and press Enter.
08:47 Next, let's count the number of blank lines in the same file.
08:52 Open the file and check how many blank lines are there are.

So, it has 3 blank lines.

09:00 Now to count the number of empty lines using awk, type:

awk space within single quote within front slash caret symbol dollar space within curly braces x equal to x plus 1 semicolon space print x space awkdemo.txt

Press Enter.

09:26 We got 3 as our final answer.
09:30 The caret sign signifies start of a line while dollar signifies the end of a line.
09:37 Hence, an empty line would be matched by the regular expression caret-dollar.
09:43 Note, we have not initialized the value of x.

Awk has initialized x to the initial value zero.

09:51 This command gives us the running count of blank lines.

This is because, every time a blank line is found, x would be incremented and then printed.

10:02 In our last command, we have seen running count of blank lines.

But, say we only want to print the total number of blank lines.

10:12 Then we need to print x only once, after the entire file has been traversed.
10:19 We may also want to give a heading saying what the output means.
10:25 For such requirements, awk provides the BEGIN and the END sections.
10:30 The BEGIN section contains procedures for pre-processing.
10:34 This section is executed before the main input loop is executed.
10:40 The END section can contain procedures for post-processing.
10:45 This section is executed after the main input loop has terminated.

The BEGIN and END procedures are optional.

10:55 Let's learn how to do this.

In the terminal, type:

awk space opening single quote BEGIN in caps within curly brace print space within double quotes The number of empty lines in awkdemo are press Enter.

11:14 within front slash caret symbol dollar symbol space within curly braces x equal to x plus 1

press Enter.

11:26 end space within curly braces print space x close single quote space awkdemo.txt and press Enter.
11:39 See, we did not get the desired output!

We should get the output as 3 because we have 3 blank lines in the file.

11:48 What do you think happened?

Actually, we should have written end as upper case END.

11:54 So, let us modify the command.
11:57 Press the up-arrow key to get the previous executed command on the terminal.
12:03 Now, modify lower case end to upper case END.

And press Enter.

12:11 Now the total number of empty lines is displayed in the output.
12:16 Next, let's find the average salary of all the students that we found in the awkdemo.txt file.
12:24 To get that, type the command as shown in the terminal

And press Enter.

And we get the desired output.

12:35 This brings us to the end of this tutorial. Let us summarize.
12:40 In this tutorial, we learnt about:

User defined variables in awk,

12:45 Operators,

BEGIN and END statements.

12:49 As an assignment, print every line where the value of the last field is more than 5000.

And, the student belongs to Electrical department.

13:00 Print the average marks of all the students with the heading “Average marks” in the output.
13:07 The video at the following link summarises the Spoken Tutorial project.

Please download and watch it.

13:14 The Spoken Tutorial Project team conducts workshops using spoken tutorials

and gives certificates on passing online test.

13:23 For more details, please write to us.
13:27 Do you have questions in THIS Spoken Tutorial?

Please visit this site.

13:32 Choose the minute and second where you have the question.

Explain your question briefly.

Someone from our team will answer them.

13:42 The Spoken Tutorial forum is for specific questions on this tutorial.
13:47 Please do not post unrelated and general questions on them.
13:51 This will help reduce the clutter.

With less clutter, we can use these discussion as instructional material.

13:59 Spoken Tutorial Project is funded by NMEICT, MHRD, Government of India.

More information on this mission is available at this link.

14:10 The script has been contributed by Antara and this is Praveen from IIT Bombay, signing off.

Thank you for joining.

Contributors and Content Editors

PoojaMoolya, Sandhya.np14

Retrieved from "https://script.spoken-tutorial.org/index.php?title=Linux-AWK/C2/Variables-and-Operators-in-awk/English-timed&oldid=48092"