Difference between revisions of "BOSS-Linux/C3/Basics-of-awk/English-timed"

From Script | Spoken-Tutorial
Jump to: navigation, search
Line 18: Line 18:
 
| 00:12
 
| 00:12
 
| To record this tutorial, I am using
 
| To record this tutorial, I am using
−
* Linux Operating system
+
*Ubuntu Linux 12.04 OS
 
*GNU BASH v. 4.2.24
 
*GNU BASH v. 4.2.24
  
 
|-
 
|-
−
| 00:21
+
| 00:23
 
| Please note, GNU Bash version 4 or above is recommended to practice this tutorial.
 
| Please note, GNU Bash version 4 or above is recommended to practice this tutorial.
  
Line 30: Line 30:
  
 
|-
 
|-
−
| 00:32
+
| 00:33
 
| The''' awk''' command is a very powerful text manipulation tool.
 
| The''' awk''' command is a very powerful text manipulation tool.
  
 
|-
 
|-
−
| 00:36
+
| 00:38
 
| It is named after its authors, '''Aho, Weinberger''' and '''Kernighan'''.
 
| It is named after its authors, '''Aho, Weinberger''' and '''Kernighan'''.
  
 
|-
 
|-
−
| 00:42
+
| 00:44
 
| It can perform several functions.  
 
| It can perform several functions.  
  
Line 46: Line 46:
  
 
|-
 
|-
−
| 00:49
+
| 00:51
 
| So, it can easily access and edit the individual fields of the record.
 
| So, it can easily access and edit the individual fields of the record.
  
Line 54: Line 54:
  
 
|-
 
|-
−
| 00:58
+
| 00:59
 
| For demonstration purpose, we use the''' awkdemo.txt '''file.  
 
| For demonstration purpose, we use the''' awkdemo.txt '''file.  
  
 
|-
 
|-
−
| 01:03
+
| 01:04
 
| Let us see the contents of '''awkdemo.txt''' file.
 
| Let us see the contents of '''awkdemo.txt''' file.
  
 
|-
 
|-
−
| 01:07
+
| 01:09
−
| Now open the '''terminal''' window
+
| Now open the '''terminal''' by pressing
 +
'''CTRL + ALT''' and '''T''' keys simultaneosuly on your keyboard.
  
 
|-
 
|-
−
| 01:10
+
| 01:17
 
| Let us see how to print using '''awk command.'''  
 
| Let us see how to print using '''awk command.'''  
  
 
|-
 
|-
−
| 01:15
+
| 01:22
−
| Type: '''awk space''' (within single quote) (front slash) ‘'''/Pass '''(front slash)'''/'''(opening curly bracket) '''{print (closing curly bracket)}''' (after the  quotes) '''space awkdemo.txt'''  
+
| Type:
 +
'''awk space''' (within single quote) (front slash) ‘'''/Pass '''(front slash)'''/'''(opening curly bracket) '''{print (closing curly bracket)}''' (after the  quotes) '''space awkdemo.txt'''  
  
 
|-
 
|-
−
| 01:32
+
| 01:38
 
| Press '''Enter'''
 
| Press '''Enter'''
  
 
|-
 
|-
−
| 01:34
+
| 01:40
−
|Here '''Pass''' is the selection criteria.  
+
|Here '''Pass''' is the selection criterion.  
  
 
|-
 
|-
−
| 01:38
+
| 01:44
 
| All the lines of the '''awkdemo''' where '''Pass''' occurs are printed.  
 
| All the lines of the '''awkdemo''' where '''Pass''' occurs are printed.  
  
 
|-
 
|-
−
| 01:43
+
| 01:49
 
| The action here is print.  
 
| The action here is print.  
  
 
|-
 
|-
−
| 01:46
+
| 01:52
 
| We can also use '''regular expressions''' in '''awk'''
 
| We can also use '''regular expressions''' in '''awk'''
  
 
|-
 
|-
−
| 01:50
+
| 01:56
 
| Say we want to print records of students with name '''Mira.'''
 
| Say we want to print records of students with name '''Mira.'''
  
 
|-
 
|-
−
| 01:55
+
| 02:01
−
| We would type: '''awk space '/M opening square bracket ei closing square bracket asterisk sign ra asterisk sign front slash opening curly bracket  print closing curly bracket  after the quoates space awkdemo.txt'''
+
| We would type:
 +
'''awk space '/M opening square bracket [ ei closing square bracket ]*ra */{ print}' space awkdemo.txt'''
  
 
|-
 
|-
−
| 02:20
+
| 02:27
 
| Press '''Enter.'''
 
| Press '''Enter.'''
  
 
|-
 
|-
−
| 02:22
+
| 02:29
−
| '''"*(asterisk)"'''  will give one or more occurrences of previous character.
+
| '''"*"'''  will give one or more occurrences of previous character.
  
 
|-
 
|-
−
| 02:27
+
| 02:33
 
| Thus entries with more than one occurrence for '''i, e''' and '''a''' will be listed.
 
| Thus entries with more than one occurrence for '''i, e''' and '''a''' will be listed.
  
 
|-
 
|-
−
| 02:34
+
| 02:40
 
| For example  
 
| For example  
  
 
|-
 
|-
−
| 02:35
+
| 02:42
−
| Mira (with 'M'  'I'  'R'  'A')
+
| *Mira
  
 
|-
 
|-
−
|02:38
+
|02:45
−
| Meera(with 'M'  'E'  'E'  'R'  'A')
+
| *Meera
  
 
|-
 
|-
−
| 02:41
+
| 02:47
−
| *Meeraa(with M E E R A A )
+
| *Meeraa
  
 
|-
 
|-
−
| 02:45
+
| 02:52
 
| '''awk''' supports the '''extended regular expressions (ERE)'''.
 
| '''awk''' supports the '''extended regular expressions (ERE)'''.
  
 
|-
 
|-
−
| 02:51
+
| 02:58
 
| Which means we can match multiple patterns separated by a '''PIPE'''.
 
| Which means we can match multiple patterns separated by a '''PIPE'''.
−
 
 
|-
 
|-
−
|02:56
+
|03:03
 
|Let me clear the promt
 
|Let me clear the promt
  
 
|-
 
|-
−
|02:59
+
|03:05
 
|Now type:
 
|Now type:
  
 
|-
 
|-
−
| 03:00
+
| 03:06
 
| '''awk space (in single quotes)(front slash) ‘/civil(PIPE) '''|electrical '''(front slash)'''space (open curly brackets)/{print}(close curly brackets) after the quotes space'''awkdemo.txt'''
 
| '''awk space (in single quotes)(front slash) ‘/civil(PIPE) '''|electrical '''(front slash)'''space (open curly brackets)/{print}(close curly brackets) after the quotes space'''awkdemo.txt'''
  
 
|-
 
|-
−
| 03:18
+
| 03:23
 
| Press '''Enter.'''
 
| Press '''Enter.'''
  
 
|-
 
|-
−
| 03:19
+
| 03:26
 
| Now entries for both '''civil''' and '''electrical''' are given.
 
| Now entries for both '''civil''' and '''electrical''' are given.
  
 
|-
 
|-
−
| 03:24
+
| 03:31
 
|  Let us go back to our slides.  
 
|  Let us go back to our slides.  
  
 
|-
 
|-
−
| 03:28
+
| 03:34
 
| Parameters '''awk''' has some special parameters to identify individual fields of a line.  
 
| Parameters '''awk''' has some special parameters to identify individual fields of a line.  
  
 
|-
 
|-
−
| 03:36
+
| 03:41
 
| '''$1(Dollar 1)''' would indicate the first field.  
 
| '''$1(Dollar 1)''' would indicate the first field.  
  
 
|-
 
|-
−
| 03:40
+
| 03:45
 
| Similarly we can have '''$2, $3''' and so on for respective fields.  
 
| Similarly we can have '''$2, $3''' and so on for respective fields.  
  
 
|-
 
|-
−
| 03:47
+
| 03:53
 
| '''$0''' represents the entire line.
 
| '''$0''' represents the entire line.
  
 
|-
 
|-
−
|03:50
+
|03:56
 
|come back to our terminal
 
|come back to our terminal
  
 
|-
 
|-
−
| 03:52
+
| 03:59
 
|Note that each word is separated by '''PIPE''' in the file '''awkdemo.txt.'''
 
|Note that each word is separated by '''PIPE''' in the file '''awkdemo.txt.'''
  
 
|-
 
|-
−
| 03:59
+
| 04:05
 
| In this case '''PIPE '''is called a '''delimiter.'''
 
| In this case '''PIPE '''is called a '''delimiter.'''
  
 
|-
 
|-
−
| 04:03
+
| 04:09
 
| A '''delimiter''' separates words from each other.
 
| A '''delimiter''' separates words from each other.
  
 
|-
 
|-
−
| 04:06
+
| 04:13
 
| A '''delimiter''' can also be a single '''whitespace. '''
 
| A '''delimiter''' can also be a single '''whitespace. '''
  
 
|-
 
|-
−
| 04:11
+
| 04:16
 
| To specify a '''delimiter''', we have to give '''- capital F '''flag followed by a '''delimiter'''.
 
| To specify a '''delimiter''', we have to give '''- capital F '''flag followed by a '''delimiter'''.
  
 
|-
 
|-
−
|04:18
+
|04:24
 
| Let us see  
 
| Let us see  
  
 
|-
 
|-
−
| 04:19
+
| 04:25
−
| Type '''awk space minus capital F space''' within double quotes''' PIPE space''' within single quote '''front slash civil PIPE electrical front slash ''' opening curly bracket''' print space dollar0 closing curly braket  after the quotes space awkdemo.txt'''
+
| Type '''awk space minus capital F space''' within double quotes''' PIPE space''' within single quote '''front slash civil PIPE electrical front slash ''' within curly braces''' print space dollar0 space awkdemo.txt'''
  
 
|-
 
|-
−
| 04:44
+
| 04:51
 
| Press '''Enter'''
 
| Press '''Enter'''
  
 
|-
 
|-
−
| 04:46
+
| 04:53
 
| This print the entire line since we have used '''$0.'''
 
| This print the entire line since we have used '''$0.'''
  
 
|-
 
|-
−
| 04:52
+
| 04:58
 
| Notice that, '''names''' and '''stream of students''' are the second and third fields.
 
| Notice that, '''names''' and '''stream of students''' are the second and third fields.
  
 
|-
 
|-
−
| 04:58
+
| 05:04
 
|Say we only want to print two fields.
 
|Say we only want to print two fields.
  
 
|-
 
|-
−
| 05:01
+
| 05:08
 
| We will replace '''$0''' with '''$2 and$3''' in the above command.
 
| We will replace '''$0''' with '''$2 and$3''' in the above command.
  
 
|-
 
|-
−
| 05:09
+
| 05:15
 
| Press '''Enter '''
 
| Press '''Enter '''
  
 
|-
 
|-
−
| 05:11
+
| 05:18
 
| Only two fields are shown.
 
| Only two fields are shown.
  
 
|-
 
|-
−
| 05:14
+
| 05:21
 
| Though it gives the right result, the display is all jagged and unformatted.  
 
| Though it gives the right result, the display is all jagged and unformatted.  
  
 
|-
 
|-
−
| 05:20
+
| 05:26
 
|  We can provide formatted output by using the '''C''' style '''printf '''statement.  
 
|  We can provide formatted output by using the '''C''' style '''printf '''statement.  
  
 
|-
 
|-
−
| 05:26
+
| 05:32
 
| We can also provide a serial number by using a builtin variable '''NR'''.  
 
| We can also provide a serial number by using a builtin variable '''NR'''.  
  
 
|-
 
|-
−
| 05:33
+
| 05:40
 
| We will see more about builtin variables later.
 
| We will see more about builtin variables later.
  
 
|-
 
|-
−
| 05:37
+
| 05:44
−
| Now Type '''awk''' space '''minus capital F within double quotes (Pipe)”''' after the double quotes space ''''front slash Pass front slash opening curly bracket printf within double quotes  percentage  sign 4d space percentage  sign -25s space percentage  sign  minus 15s space backslash n”,''' after the double quotes  '''NR,$2,$3 closing curly bracket'''' after the single quote space '''awkdemo.txt'''  
+
| Now Type
 +
'''awk''' space '''-F”|”''' space ''''/Pass/{printf “%4d %-25s %-15s \n”,''' '''NR,$2,$3 }'''' space '''awkdemo.txt'''  
  
 
|-
 
|-
−
| 06:25
+
| 06:33
−
| Press '''Enter.'''
+
| '''Press Enter.'''
  
 
|-
 
|-
−
| 06:28
+
| 06:34
 
| We see the difference.
 
| We see the difference.
  
 
|-
 
|-
−
| 06:30
+
| 06:37
 
| Here the '''NR''' stands for number of records.  
 
| Here the '''NR''' stands for number of records.  
  
 
|-
 
|-
−
| 06:33
+
| 06:41
 
| Records are '''integers''', hence we have written''' %d'''.  
 
| Records are '''integers''', hence we have written''' %d'''.  
  
 
|-
 
|-
−
| 06:37
+
| 06:45
 
| '''Name''' and '''Stream''' are '''strings'''. So we have used '''%s'''.  
 
| '''Name''' and '''Stream''' are '''strings'''. So we have used '''%s'''.  
  
 
|-
 
|-
−
| 06:43
+
| 06:50
 
| Here '''25s''' will reserve 25 spaces for '''Name''' field.
 
| Here '''25s''' will reserve 25 spaces for '''Name''' field.
  
 
|-
 
|-
−
| 06:48
+
| 06:55
 
| '''15s''' will reserve 15 spaces for '''Stream''' field.
 
| '''15s''' will reserve 15 spaces for '''Stream''' field.
  
 
|-
 
|-
−
| 06:54
+
| 07:01
 
| The '''minus sign''' is used to left justify the output.
 
| The '''minus sign''' is used to left justify the output.
  
 
|-
 
|-
−
| 06:58
+
| 07:05
 
| This brings us to the end of this tutorial.
 
| This brings us to the end of this tutorial.
  
 
|-
 
|-
−
| 07:01
+
| 07:08
 
| Let us move back to our slides.
 
| Let us move back to our slides.
  
 
|-
 
|-
−
| 07:04
+
| 07:10
 
| Let us summarize.
 
| Let us summarize.
  
 
|-
 
|-
−
| 07:05
+
| 07:11
−
| In this tutorial we learnt To print using '''awk'''
+
| In this tutorial we learnt To print using awk
  
 
|-
 
|-
−
| 07:10
+
| 07:16
−
| Regular expression in '''awk'''. To list the enteries for a paritcular stream
+
| Regular expression in awk To list the enteries for a paritcular stream
  
 
|-
 
|-
−
| 07:15
+
| 07:21
 
| To list only the second and the third fileds
 
| To list only the second and the third fileds
  
 
|-
 
|-
−
| 07:18
+
| 07:24
 
| To display a formatted output
 
| To display a formatted output
  
 
|-
 
|-
−
| 07:19
+
| 07:28
 
| As an assignment  
 
| As an assignment  
  
 
|-
 
|-
−
|07:22
+
|07:29
 
| Display roll no., stream and marks of Ankti Saraf
 
| Display roll no., stream and marks of Ankti Saraf
  
 
|-
 
|-
−
| 07:27
+
| 07:34
 
|  Watch the video available at the link shown below
 
|  Watch the video available at the link shown below
  
 
|-
 
|-
−
| 07:30
+
| 07:37
 
| It summarises the Spoken Tutorial project
 
| It summarises the Spoken Tutorial project
  
 
|-
 
|-
−
| 07:33
+
| 07:40
 
| If you do not have good bandwidth, you can download and watch it
 
| If you do not have good bandwidth, you can download and watch it
  
 
|-
 
|-
−
|07:37
+
|07:45
 
|The Spoken Tutorial Project Team Conducts workshops using spoken tutorials
 
|The Spoken Tutorial Project Team Conducts workshops using spoken tutorials
  
 
|-
 
|-
−
| 07:42
+
| 07:48
 
| Gives certificates to those who pass an online test
 
| Gives certificates to those who pass an online test
  
 
|-
 
|-
−
| 07:45
+
| 07:52
 
| For more details, please write to contact@spoken-tutorial.org
 
| For more details, please write to contact@spoken-tutorial.org
  
 
|-
 
|-
−
| 07:51
+
| 07:58
 
|Spoken Tutorial Project is a part of the Talk to a Teacher project
 
|Spoken Tutorial Project is a part of the Talk to a Teacher project
  
 
|-
 
|-
−
| 07:55
+
| 08:01
 
| It is supported by the National Mission on Education through ICT,MHRD,Government of India
 
| It is supported by the National Mission on Education through ICT,MHRD,Government of India
  
 
|-
 
|-
−
| 08:02
+
| 08:07
 
| More information on this Mission is available at:  [http://spokentutorial.org\NMEICT-Intro]
 
| More information on this Mission is available at:  [http://spokentutorial.org\NMEICT-Intro]
  
 
|-
 
|-
−
| 08:07
+
| 08:12
 
|This is Ashwini Patil from IIT Bombay signning off. Thank you for joining.
 
|This is Ashwini Patil from IIT Bombay signning off. Thank you for joining.
  
 
|}
 
|}

Revision as of 13:07, 12 February 2015

Time Narration
00:01 Welcome to the spoken tutorial on the awk command.
00:05 In this tutorial we will learn, awk command.
00:09 We will do this through some examples.
00:12 To record this tutorial, I am using
  • Ubuntu Linux 12.04 OS
  • GNU BASH v. 4.2.24
00:23 Please note, GNU Bash version 4 or above is recommended to practice this tutorial.
00:29 Let us start with an introduction to awk.
00:33 The awk command is a very powerful text manipulation tool.
00:38 It is named after its authors, Aho, Weinberger and Kernighan.
00:44 It can perform several functions.
00:46 It operates at the field level of a record.
00:51 So, it can easily access and edit the individual fields of the record.
00:56 Let us see some examples.
00:59 For demonstration purpose, we use the awkdemo.txt file.
01:04 Let us see the contents of awkdemo.txt file.
01:09 Now open the terminal by pressing

CTRL + ALT and T keys simultaneosuly on your keyboard.

01:17 Let us see how to print using awk command.
01:22 Type:

awk space (within single quote) (front slash) ‘/Pass (front slash)/(opening curly bracket) {print (closing curly bracket)} (after the quotes) space awkdemo.txt

01:38 Press Enter
01:40 Here Pass is the selection criterion.
01:44 All the lines of the awkdemo where Pass occurs are printed.
01:49 The action here is print.
01:52 We can also use regular expressions in awk
01:56 Say we want to print records of students with name Mira.
02:01 We would type:

awk space '/M opening square bracket [ ei closing square bracket ]*ra */{ print}' space awkdemo.txt

02:27 Press Enter.
02:29 "*" will give one or more occurrences of previous character.
02:33 Thus entries with more than one occurrence for i, e and a will be listed.
02:40 For example
02:42 *Mira
02:45 *Meera
02:47 *Meeraa
02:52 awk supports the extended regular expressions (ERE).
02:58 Which means we can match multiple patterns separated by a PIPE.
03:03 Let me clear the promt
03:05 Now type:
03:06 electrical (front slash)space (open curly brackets)/{print}(close curly brackets) after the quotes spaceawkdemo.txt
03:23 Press Enter.
03:26 Now entries for both civil and electrical are given.
03:31 Let us go back to our slides.
03:34 Parameters awk has some special parameters to identify individual fields of a line.
03:41 $1(Dollar 1) would indicate the first field.
03:45 Similarly we can have $2, $3 and so on for respective fields.
03:53 $0 represents the entire line.
03:56 come back to our terminal
03:59 Note that each word is separated by PIPE in the file awkdemo.txt.
04:05 In this case PIPE is called a delimiter.
04:09 A delimiter separates words from each other.
04:13 A delimiter can also be a single whitespace.
04:16 To specify a delimiter, we have to give - capital F flag followed by a delimiter.
04:24 Let us see
04:25 Type awk space minus capital F space within double quotes PIPE space within single quote front slash civil PIPE electrical front slash within curly braces print space dollar0 space awkdemo.txt
04:51 Press Enter
04:53 This print the entire line since we have used $0.
04:58 Notice that, names and stream of students are the second and third fields.
05:04 Say we only want to print two fields.
05:08 We will replace $0 with $2 and$3 in the above command.
05:15 Press Enter
05:18 Only two fields are shown.
05:21 Though it gives the right result, the display is all jagged and unformatted.
05:26 We can provide formatted output by using the C style printf statement.
05:32 We can also provide a serial number by using a builtin variable NR.
05:40 We will see more about builtin variables later.
05:44 Now Type

awk space -F”|” space '/Pass/{printf “%4d %-25s %-15s \n”, NR,$2,$3 }' space awkdemo.txt

06:33 Press Enter.
06:34 We see the difference.
06:37 Here the NR stands for number of records.
06:41 Records are integers, hence we have written %d.
06:45 Name and Stream are strings. So we have used %s.
06:50 Here 25s will reserve 25 spaces for Name field.
06:55 15s will reserve 15 spaces for Stream field.
07:01 The minus sign is used to left justify the output.
07:05 This brings us to the end of this tutorial.
07:08 Let us move back to our slides.
07:10 Let us summarize.
07:11 In this tutorial we learnt To print using awk
07:16 Regular expression in awk To list the enteries for a paritcular stream
07:21 To list only the second and the third fileds
07:24 To display a formatted output
07:28 As an assignment
07:29 Display roll no., stream and marks of Ankti Saraf
07:34 Watch the video available at the link shown below
07:37 It summarises the Spoken Tutorial project
07:40 If you do not have good bandwidth, you can download and watch it
07:45 The Spoken Tutorial Project Team Conducts workshops using spoken tutorials
07:48 Gives certificates to those who pass an online test
07:52 For more details, please write to contact@spoken-tutorial.org
07:58 Spoken Tutorial Project is a part of the Talk to a Teacher project
08:01 It is supported by the National Mission on Education through ICT,MHRD,Government of India
08:07 More information on this Mission is available at: [1]
08:12 This is Ashwini Patil from IIT Bombay signning off. Thank you for joining.

Contributors and Content Editors

PoojaMoolya, Pratik kamble, Sandhya.np14

Retrieved from "https://script.spoken-tutorial.org/index.php?title=BOSS-Linux/C3/Basics-of-awk/English-timed&oldid=20252"