Difference between revisions of "BOSS-Linux/C2/Redirection-Pipes/English"

From Script | Spoken-Tutorial
Jump to: navigation, search
(Created page with "Pending")
 
 
Line 1: Line 1:
Pending
+
'''Title of script''':  Redirection and Pipes
 +
 
 +
'''Author: Anirban Roy Choudhury'''
 +
 
 +
'''Keywords:  Standard input,standard output, standard error, redirection, pipes, file descriptor'''
 +
 
 +
 
 +
[http://spoken-tutorial.org/wiki/index.php/File:Resources_for_the_tutorial.zip Resources for the tutorial]
 +
 
 +
{| style="border-spacing:0;"
 +
| style="border-top:0.035cm solid #000000;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| <center>'''Visual Cue'''</center>
 +
| style="border:0.035cm solid #000000;padding:0.097cm;"|
 +
<center>'''Narration'''</center>
 +
 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide1
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Welcome to this spoken tutorial on Redirection and Pipes.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 2
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| I am using Linux Operating System. We assume that you already have an idea about the Linux operating system and have some basic idea about commands.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| If you are interested, it is available through another spoken tutorial, on the following website [http://spoken-tutorial.org/ http://spoken-tutorial.org].
 +
 
 +
|-
 +
| style="border:0.035cm solid #000000;padding:0.097cm;"| Display Slide 3
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Also note that linux is case sensitive.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Open a terminal window
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| All the commands used in this tutorial are in lower case unless otherwise mentioned.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Most of the work that we do in Linux is through a terminal.
 +
 
 +
When we have to execute a command, we generally type through the keyboard.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
"date"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Say we have to find the date and time. We simply type through the keyboard "date". And press enter. So we normally give input through the keyboard.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Similarly we see that the output of our cammand is also displayed at the terminal window.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
"cat aaa"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Also say some error occurs while we are executing some command. For example we type "cat space aaa". And press enter.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now no file by the name aaa exists. So there is an error displayed, saying so. Now this error also comes on the terminal window. Hence we see error reporting is also at the terminal
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 4
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now inputting, outputting and error reporting are the three special actions related to commands. Before learning about redirection we should know about two important concepts. That of stream and file descriptor.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| A Linux shell like Bash, receives input and sends output as sequences or streams of characters. Each character is independent of the one before it and the one after it.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Streams are accessed using file IO techniques. It does not matter whether or not the actual stream of characters comes from or goes to a file, a keyboard, a window etc.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| In Linux, every open file of a process is associated with an integer number. These numeric values are known as file descriptors.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 5
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Linux shells use three standard I/O streams. Each of them is associated with a well-known file descriptor.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| ''stdin is the standard input stream. It provides input to commands. It has file descriptor 0. ''
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 6
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| ''stdout'' is the ''standard output stream. It'' displays output from commands. It has file descriptor 1. 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 7
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| ''stderr is the standard error stream.It displays error output from commands. It has file descriptor 2.''
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 8
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| ''Input streams provide input to programs. By default it takes from terminal keystrokes. ''
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| ''Output streams print text characters, by default to the terminal. The terminal was originally an ASCII typewriter or display terminal. But is now more often a text window on a graphical desktop. ''
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Stay in slide 8
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We have seen that the 3 streams are connected to some files by default. But in linux, we can change this default behaviour.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We can connect these 3 streams to other files. This process is called redirection.
 +
 
 +
Now let us see how redirection is done in the 3 streams.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| First let us see how the standard input in redirected.
 +
 
 +
we redirect stdin from a file, using the < (left angled bracket) operator.
 +
 
 +
 
 +
Let us see how.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
 
 +
"wc"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We know that the use of wc command is to find the number of lines,words and characters in a file.
 +
 
 +
Type wc on the terminal window. Now press enter. What happens?? we have a blinking cursor. It means that we need to enter through the keyboard.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| press ''Enter </ ''
 +
 
 +
“This tutorial is very important"
 +
 
 +
press Enter </
 +
 
 +
Ctrl+d </
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Enter some text "This tutorial is very important".
 +
 
 +
Now press enter. Now press Ctrl and d keys together.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now the command will work on the lines that we entered. The command will give an output on the terminal.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now here no file is given after wc command . So it takes input from standard input stream. Now the standard input stream is by default connected to the keyboard. Hence wc will take input from the keyboard.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Create a file named test1.txt having some text in pwd </
 +
 
 +
 
 +
Type at the command prompt
 +
 
 +
"wc < test1.txt"</
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now if we write
 +
 
 +
"wc space 'left-angled bracket" space test1 dot txt"
 +
 
 +
what happens is that wc will tell us the number of lines,words and charecters in the file test1.txt
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
"wc test1.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now type
 +
 
 +
"wc space test1 dot txt", we see the same result. So what is the difference?
 +
 
 +
 
 +
 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| When we write "wc space test1.txt" , the command opens the file test1.txt and reads from it. But when we write "wc space 'left-angled bracket' test1.txt", wc still does not get any file to open. Instead ,it looks to pick up the input from stdin. Now we have directed the stdin to the file test1.txt. Hence the command reads from test1. But actually it is unaware as to from where the data is coming to stdin.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display slide 9
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| So, we have seen how to redirect standard input.
 +
 
 +
 
 +
Now let us see how to redirect standard output and standard error. There are two ways to redirect output or error to a file:
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Suppose, n refers to the file descriptor.
 +
 
 +
'''''n''>''' [narration- n 'single right-angled bracket']</
 +
 
 +
redirects output from file descriptor ''n'' to a file. You must have write authority to the file.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| If the file does not exist, it is created. If it does exist, the existing contents are usually lost without any warning
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display slide 10
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| '''''n''>>''' [narration- n 'double right-angled bracket']</
 +
 
 +
also redirects output from file descriptor ''n'' to a file. Again, you must have write authority to the file.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| If the file does not exist, it is created. If it does exist, the output is appended to the existing file. 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display slide 11
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| The ''n'' in n> or n>> refers to the ''file descriptor''. If it is omitted, then standard output i.e. file descriptor 1 is assumed.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| So, > is same as 1>. But, to redirect error stream, you have to use 2> or 2>>.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Switch to terminal
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Let us see this practically.
 +
 
 +
From the last example we have seen that the result of the wc command on a file or stdin is displayed at the terminal window.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| What if we do not want to display this on the terminal? We want to store it in a file , so that the information can be later used.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| By default wc writes its output to the stdout . The stdout is by default connected to the terminal window. Hence we see the output in the terminal windoow. But if we can redirect the stdout to a file, then the output from the wc command will be written to that file.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
"wc test1.txt"
 +
 
 +
wc_results.txt"
 +
 
 +
 
 +
"cat wc_results.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Say we write
 +
 
 +
"wc space test1 dot txt 'right-angled bracket' wc_results dot txt" . Press enter.
 +
 
 +
 
 +
Now to see whether this has actually happened we can display the contents of wc_results.txt by the cat command. Yes it has.
 +
 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| (also keep a file test2.txt ready)
 +
 
 +
"wc test2.txt > wc_results.txt"
 +
 
 +
"cat wc_results.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Suppose,we have another file test2 in the same directory.
 +
 
 +
Now we again execute the command with test2 file. We type
 +
 
 +
"wc space test2 dot txt 'right-angled bracket' wc_results dot txt"
 +
 
 +
So,the contents of the file wc_results would be overwritten.
 +
 
 +
Let's see this .
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "wc test1.txt >> wc_results.txt"
 +
 
 +
 
 +
"cat wc_results.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Instead if we write "wc space test1.txt 'right-angled bracket' twice wc_results.txt"
 +
 
 +
the new contents will not overwrite the already present contents of the file wc_results.txt, it would be appended.
 +
 
 +
Let's see this too.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Redirecting the standard error is done similarly. Only difference is that in this case we need to mention the file descriptor number of stderr before the > or >> sign.
 +
 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
"wc aaa"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Like we know that their exists no file by the name aaa write the following
 +
 
 +
"wc space aaa"
 +
 
 +
the shell will give the error “No such file or directory”.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now say we dont want error messages on screen. They can be redirected to some other file .
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "wc aaa 2>errorlog.txt"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| For this we may give the command
 +
 
 +
"wc space aaa space 2 'right-anged bracket' errorlog.txt"
 +
 
 +
now the error will not show on the terminal, rather it will be written in the file errorlog.txt
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cat errorlog.txt"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| we can see this by the command
 +
 
 +
"cat space errorlog dot txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cat bbb 2> errorlog.txt"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now suppose that I make some other error by running the command
 +
 
 +
"cat space bbb space 2 'right-angled bracket' errorlog.txt".
 +
 
 +
the previous error would be overwritten and the new error will show.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cat errorlog.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| See "cat space errorlog dot txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "wc aaa 2>>errorlog.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| But what if we want to list all errors??
 +
 
 +
simple we would run the command
 +
 
 +
"wc space aaa space 2 'right-angled bracket' twice errorlog.txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cat errorlog.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We check this using the cat command.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display slide 11
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We have seen how the three streams stdout,stdin,stderr are redirected and manupulated separately. But the real power of this concept can be gauged when we can manupulate the streams together, ie connect the different streams.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| This process is called pipelining. Pipes are used to create chains of command. Pipe connect the output of one command to the input of the next command in the chain.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| It looks like
 +
 
 +
command1 | command2 option1 | command3 option1 -option2 | command4
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Say we want to know the total number of files and directories present in the current directory
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Type at the command prompt
 +
 
 +
 
 +
"ls -l"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| What we can do? We know
 +
 
 +
"ls space minus l" will list all files and directories of the present directory.
 +
 
 +
 
 +
 
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "ls -l > files.txt"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We can redirect the output to a file
 +
 
 +
"ls space minus l 'right-angled bracket' files.txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cat files.txt"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Run "cat space files dot txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "wc -l files.txt"
 +
 
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Now each line is a name of a file or directory . So if we measure the total lines in this file, we can use files.txt to serve our purpose.
 +
 
 +
This we can do using the command "wc space minus l files dot txt"
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Though this serves our purpose there are a few problems.
 +
 
 +
First we need an intermediate file , here files.txt . If the first command produces a lot of data,it may unneccesarily eat away the disk memory. Also if we want to chain several commands, this method is slow.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "ls -l | wc -l"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We can do it much easily using pipes like this . We write
 +
 
 +
"ls space minus l 'vertical bar' wc space minus l"
 +
 
 +
and we can get the same result with much more ease.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| The output from the ls command goes as input for the wc command.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "cd /usr/bin"
 +
 
 +
 
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| We can add even longer chains of command using pipes.
 +
 
 +
One common use of pipes is for reading multipage displays.
 +
 
 +
Type "cd space /usr/bin". So, we are now in bin directory.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| "ls -l"
 +
 
 +
"ls -l | more"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| now run "ls -l" , we cannot see the output properly. But if we use it joined with a pipe to more, we can.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Press "q"
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| Press enter to scroll through the list.
 +
 
 +
Press "q" to come out of it.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"| Display Slide 12
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| These were some of the commands that help us to work with files. There are many more commands. Moreover each of the commands that we saw has many other options.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| I encourage you to see more about them using the 'man' command.The best way of learning commands is to use them again and again.
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| This brings me to the end of this tutorial. Spoken Tutorials are a part of the Talk to a Teacher project, supported by the National Mission on Education through ICT.
 +
 
 +
 
 +
More information on the same is available at the following link [http://spoken-tutorial.org/NMEICT-Intro http://spoken-tutorial.org/NMEICT-Intro]
 +
 
 +
|-
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:none;padding:0.097cm;"|
 +
| style="border-top:none;border-bottom:0.035cm solid #000000;border-left:0.035cm solid #000000;border-right:0.035cm solid #000000;padding:0.097cm;"| This is Abhijit Sunil signing off . Thank you for joining.
 +
 
 +
|}

Latest revision as of 16:20, 17 November 2014

Title of script: Redirection and Pipes

Author: Anirban Roy Choudhury

Keywords: Standard input,standard output, standard error, redirection, pipes, file descriptor


Resources for the tutorial

Visual Cue
Narration


Display Slide1 Welcome to this spoken tutorial on Redirection and Pipes.
Display Slide 2


I am using Linux Operating System. We assume that you already have an idea about the Linux operating system and have some basic idea about commands.
If you are interested, it is available through another spoken tutorial, on the following website http://spoken-tutorial.org.
Display Slide 3 Also note that linux is case sensitive.
Open a terminal window All the commands used in this tutorial are in lower case unless otherwise mentioned.
Most of the work that we do in Linux is through a terminal.

When we have to execute a command, we generally type through the keyboard.

Type at the command prompt

"date"

Say we have to find the date and time. We simply type through the keyboard "date". And press enter. So we normally give input through the keyboard.
Similarly we see that the output of our cammand is also displayed at the terminal window.
Type at the command prompt

"cat aaa"

Also say some error occurs while we are executing some command. For example we type "cat space aaa". And press enter.
Now no file by the name aaa exists. So there is an error displayed, saying so. Now this error also comes on the terminal window. Hence we see error reporting is also at the terminal
Display Slide 4


Now inputting, outputting and error reporting are the three special actions related to commands. Before learning about redirection we should know about two important concepts. That of stream and file descriptor.
A Linux shell like Bash, receives input and sends output as sequences or streams of characters. Each character is independent of the one before it and the one after it.
Streams are accessed using file IO techniques. It does not matter whether or not the actual stream of characters comes from or goes to a file, a keyboard, a window etc.
In Linux, every open file of a process is associated with an integer number. These numeric values are known as file descriptors.
Display Slide 5 Linux shells use three standard I/O streams. Each of them is associated with a well-known file descriptor.
stdin is the standard input stream. It provides input to commands. It has file descriptor 0.
Display Slide 6 stdout is the standard output stream. It displays output from commands. It has file descriptor 1.
Display Slide 7 stderr is the standard error stream.It displays error output from commands. It has file descriptor 2.
Display Slide 8 Input streams provide input to programs. By default it takes from terminal keystrokes.
Output streams print text characters, by default to the terminal. The terminal was originally an ASCII typewriter or display terminal. But is now more often a text window on a graphical desktop.
Stay in slide 8 We have seen that the 3 streams are connected to some files by default. But in linux, we can change this default behaviour.
We can connect these 3 streams to other files. This process is called redirection.

Now let us see how redirection is done in the 3 streams.

First let us see how the standard input in redirected.

we redirect stdin from a file, using the < (left angled bracket) operator.


Let us see how.

Type at the command prompt


"wc"

We know that the use of wc command is to find the number of lines,words and characters in a file.

Type wc on the terminal window. Now press enter. What happens?? we have a blinking cursor. It means that we need to enter through the keyboard.

press Enter </

“This tutorial is very important"

press Enter </

Ctrl+d </

Enter some text "This tutorial is very important".

Now press enter. Now press Ctrl and d keys together.

Now the command will work on the lines that we entered. The command will give an output on the terminal.
Now here no file is given after wc command . So it takes input from standard input stream. Now the standard input stream is by default connected to the keyboard. Hence wc will take input from the keyboard.
Create a file named test1.txt having some text in pwd </


Type at the command prompt

"wc < test1.txt"</


Now if we write

"wc space 'left-angled bracket" space test1 dot txt"

what happens is that wc will tell us the number of lines,words and charecters in the file test1.txt

Type at the command prompt

"wc test1.txt"

Now type

"wc space test1 dot txt", we see the same result. So what is the difference?



When we write "wc space test1.txt" , the command opens the file test1.txt and reads from it. But when we write "wc space 'left-angled bracket' test1.txt", wc still does not get any file to open. Instead ,it looks to pick up the input from stdin. Now we have directed the stdin to the file test1.txt. Hence the command reads from test1. But actually it is unaware as to from where the data is coming to stdin.
Display slide 9 So, we have seen how to redirect standard input.


Now let us see how to redirect standard output and standard error. There are two ways to redirect output or error to a file:

Suppose, n refers to the file descriptor.

n> [narration- n 'single right-angled bracket']</

redirects output from file descriptor n to a file. You must have write authority to the file.

If the file does not exist, it is created. If it does exist, the existing contents are usually lost without any warning
Display slide 10 n>> [narration- n 'double right-angled bracket']</

also redirects output from file descriptor n to a file. Again, you must have write authority to the file.

If the file does not exist, it is created. If it does exist, the output is appended to the existing file.
Display slide 11 The n in n> or n>> refers to the file descriptor. If it is omitted, then standard output i.e. file descriptor 1 is assumed.
So, > is same as 1>. But, to redirect error stream, you have to use 2> or 2>>.
Switch to terminal


Let us see this practically.

From the last example we have seen that the result of the wc command on a file or stdin is displayed at the terminal window.

What if we do not want to display this on the terminal? We want to store it in a file , so that the information can be later used.
By default wc writes its output to the stdout . The stdout is by default connected to the terminal window. Hence we see the output in the terminal windoow. But if we can redirect the stdout to a file, then the output from the wc command will be written to that file.
Type at the command prompt

"wc test1.txt"

wc_results.txt"


"cat wc_results.txt"

Say we write

"wc space test1 dot txt 'right-angled bracket' wc_results dot txt" . Press enter.


Now to see whether this has actually happened we can display the contents of wc_results.txt by the cat command. Yes it has.


(also keep a file test2.txt ready)

"wc test2.txt > wc_results.txt"

"cat wc_results.txt"

Suppose,we have another file test2 in the same directory.

Now we again execute the command with test2 file. We type

"wc space test2 dot txt 'right-angled bracket' wc_results dot txt"

So,the contents of the file wc_results would be overwritten.

Let's see this .

"wc test1.txt >> wc_results.txt"


"cat wc_results.txt"

Instead if we write "wc space test1.txt 'right-angled bracket' twice wc_results.txt"

the new contents will not overwrite the already present contents of the file wc_results.txt, it would be appended.

Let's see this too.

Redirecting the standard error is done similarly. Only difference is that in this case we need to mention the file descriptor number of stderr before the > or >> sign.


Type at the command prompt

"wc aaa"


Like we know that their exists no file by the name aaa write the following

"wc space aaa"

the shell will give the error “No such file or directory”.

Now say we dont want error messages on screen. They can be redirected to some other file .
"wc aaa 2>errorlog.txt"


For this we may give the command

"wc space aaa space 2 'right-anged bracket' errorlog.txt"

now the error will not show on the terminal, rather it will be written in the file errorlog.txt

"cat errorlog.txt"


we can see this by the command

"cat space errorlog dot txt"

"cat bbb 2> errorlog.txt"


Now suppose that I make some other error by running the command

"cat space bbb space 2 'right-angled bracket' errorlog.txt".

the previous error would be overwritten and the new error will show.

"cat errorlog.txt" See "cat space errorlog dot txt"
"wc aaa 2>>errorlog.txt" But what if we want to list all errors??

simple we would run the command

"wc space aaa space 2 'right-angled bracket' twice errorlog.txt"

"cat errorlog.txt" We check this using the cat command.
Display slide 11 We have seen how the three streams stdout,stdin,stderr are redirected and manupulated separately. But the real power of this concept can be gauged when we can manupulate the streams together, ie connect the different streams.
This process is called pipelining. Pipes are used to create chains of command. Pipe connect the output of one command to the input of the next command in the chain.
It looks like
command1 | command2 option1 | command3 option1 -option2 | command4
Say we want to know the total number of files and directories present in the current directory
Type at the command prompt


"ls -l"

What we can do? We know

"ls space minus l" will list all files and directories of the present directory.



"ls -l > files.txt"


We can redirect the output to a file

"ls space minus l 'right-angled bracket' files.txt"

"cat files.txt" Run "cat space files dot txt"
"wc -l files.txt"


Now each line is a name of a file or directory . So if we measure the total lines in this file, we can use files.txt to serve our purpose.

This we can do using the command "wc space minus l files dot txt"

Though this serves our purpose there are a few problems.

First we need an intermediate file , here files.txt . If the first command produces a lot of data,it may unneccesarily eat away the disk memory. Also if we want to chain several commands, this method is slow.

"ls -l | wc -l" We can do it much easily using pipes like this . We write

"ls space minus l 'vertical bar' wc space minus l"

and we can get the same result with much more ease.

The output from the ls command goes as input for the wc command.
"cd /usr/bin"


We can add even longer chains of command using pipes.

One common use of pipes is for reading multipage displays.

Type "cd space /usr/bin". So, we are now in bin directory.

"ls -l"

"ls -l | more"

now run "ls -l" , we cannot see the output properly. But if we use it joined with a pipe to more, we can.
Press "q" Press enter to scroll through the list.

Press "q" to come out of it.

Display Slide 12 These were some of the commands that help us to work with files. There are many more commands. Moreover each of the commands that we saw has many other options.
I encourage you to see more about them using the 'man' command.The best way of learning commands is to use them again and again.
This brings me to the end of this tutorial. Spoken Tutorials are a part of the Talk to a Teacher project, supported by the National Mission on Education through ICT.


More information on the same is available at the following link http://spoken-tutorial.org/NMEICT-Intro

This is Abhijit Sunil signing off . Thank you for joining.

Contributors and Content Editors

Nancyvarkey, PoojaMoolya