Linux-Ubuntu/C2/Redirecting-Streams-and-Pipes/English

From Script | Spoken-Tutorial
Jump to: navigation, search

TITLE: Redirecting Streams and Pipes.

Author: EduPyramids

Keywords: Standard input,standard output, standard error, redirection, pipes, file descriptors, input redirection, output redirection, error redirection, pipes, pipelining, wc, ls, more, Linux terminal, command chaining, EduPyramids, video tutorial.


Visual Cue Narration
Slide 1

Title Slide

Welcome to this spoken tutorial on Redirecting Streams and Pipes.
Slide 2

Learning Objectives

In this tutorial, we will learn to:

  • Identify stdin, stdout and stderr streams in Linux.
  • Redirect standard output and error to files using redirection operators (>, >>, 2>, 2>>).
  • Explain file descriptors and their role in redirection.
  • Connect multiple Linux commands using pipes (|) to manipulate data efficiently.
In this tutorial, we will learn to:
  • Identify standard input, standard output and standard error streams in Linux.
  • Redirect standard output and error to files using redirection operators
  • Explain file descriptors and their role in redirection.
  • Connect multiple Linux commands using pipes (|) to manipulate data efficiently.
Slide 3

System Requirements

To record this tutorial, I am using

Ubuntu OS version 24 point zero 4

Slide 4

Pre-requisites

https://EduPyramids.org

To follow this tutorial,
  • Learners should have Ubuntu version 24 point zero 4.
  • For the prerequisite Linux tutorials please visit this website.
Slide 5

Code files

The following code file is required to practice this tutorial

  1. rsp-commands.txt

This file is provided in the Code Files link of this tutorial page.

The following code file is required to practice this tutorial.

This file is provided in the Code Files link of this tutorial page.

Press Ctrl + Alt + T keys Let us get started.

Open the terminal.

Open Terminal

Type cat aaa press Enter

Highlight Errored output.

cat: aaa: No such file or directory

Let’s type cat a a a and press Enter.

Since this file does not exist, terminal shows an error message.

Notice that errors are reported on the terminal.

Slide 6

streams.png

Streams: are continuous flow of data used for input, output, and errors.

File Descriptors: are numbers used by Linux to identify open files and streams.

Bash: is a Linux shell that runs commands and manages input and output.

Slide 7

Standard Streams

std-streams.png

Linux commands handle input, output, and errors using streams and file descriptors.

Linux uses three standard streams:

  • s t d in (0) for input
  • s t d out (1) for output
  • s t d e r r (2) for errors

By default, the standard streams are connected to the terminal.

Using redirection, we can change this behavior.

Standard input can be redirected using the less than(<) operator.

In a terminal type cat >test1.txt press Enter

Type: Sample to learn Linux.

Press Enter.Press Ctrl and D keys together.

Now, let us learn to redirect standard input.

Let's create a file named test1 dot t x t in the current directory and add some text to it.

Users can pause the video and create the file test1 dot t x t.

Type wc < test1.txtPress Enter.

Point to the output on the terminal.

Now, type w c space less than space test1 dot t x t and press Enter.

w c displays the number of lines, words, and characters in the test1 dot t x t file.

This command reads the file using s t d in, showing input redirection.

Now, let us learn to redirect standard output and standard error.
Slide 8

Channel Number

Think of n as a channel number.

Linux uses numbers to decide where output goes.

  • n identifies the output channel.
  • > means send output
  • filename is where the output is saved
There are two ways to redirect them to a file:

Think of n as a channel number.

Linux uses numbers to decide where output goes.

  • n identifies the output channel.
  • Greater than(>) means send output
  • filename is where the output is saved
Slide 9

Redirection Operations

n >filename means send output from channel n to a file.

You need write permission for that file.n >> redirects output from file descriptor n to a file.

If the file exists, output is appended, not overwritten.If n is not specified, stdout (1) is used.

> is the same as 1>.

To redirect the error stream, use 2> or 2>>.

So, n greater than(>) filename means send output from channel n to a file.

You need write permission for that file.

n greater than greater than(>>) redirects output from file descriptor n to a file.

If the file exists, output is appended, not overwritten.

If n is not specified, stdout (1) is used.

So, greater than( >) is the same as 1 greater than(>).

To redirect the error stream, use 2 greater than(>) or 2 greater than greater than (>>).

Switch to TerminalShow the terminal output of wc command Let us see this practically.

Suppose we want to save the output to a file for later use. By default, w c sends output to s t d out

Redirecting s t d out saves the output to a file instead.

Type wc test1.txt > wc-results.txt

Type cat wc-results.txt

Point to the output

Type: w c space test1 dot t x t space greater than space w c hyphen results dot t x t

Press Enter.

Now, to verify this, display the contents of w c hyphen results dot t x t using the cat command.

Type

cat space w c hyphen results dot t x t

And press Enter

The output from w c has been written to the file.

Type: echo "hello" > test2.txt press Enter

Show the output line with test2.txt

Type wc test2.txt > wc-results.txt

Type cat wc-results.txt press Enter

Let me create a new file by typing echo space “hello” space greater than space test2 dot t x t and press Enter.

We see the file test2 dot t x t is created.

Type w c space test2 dot t x t space greater than space w c hyphen results dot t x t.

The contents of w c hyphen results dot t x t will be overwritten.

Let us verify using the cat command.

Type

wc test1.txt >> wc-results.txt and Press Enter

Type cat wc-results.txt

Instead, if we run:w c space test1 dot t x t greater than greater than (>>) w c hyphen results dot t x t and press Enter.

The new contents will not overwrite the existing contents of w c hyphen results dot t x t.

Instead, the output will be appended to the file.

Let us verify using the cat command

Let me clear the terminal.

Redirecting standard error is done in a similar way.

The only difference is that we need to specify the file descriptor number of standard error.

This number comes before the greater than (>) or greater than greater than(>>) sign.

Type wc aaa press Enter


Highlight error. There is no file named a a a.

Type: w c space a a a and press Enter.

The shell will display an error as No such file or directory.

If we don’t want error messages to show on the screen, we can redirect them to a file.
Type: wc aaa 2>errorlog.txt and press Enter To redirect the error, type the following command:w c space a a a space 2 greater than(>) errorlog dot t x t and press Enter.

Now, the error will not appear on the terminal.

Instead, it will be written to the file errorlog dot t x t

Type: cat errorlog.txt press Enter We can check the contents of errorlog dot t x t by typing, cat space errorlog dot t x t press Enter.

This will display the error message that was redirected from the terminal.

Type cat bbb 2> errorlog.txt

Press Enter.

Type: cat errorlog.txt press Enter

Now, suppose we make another error by typing:cat space b b b space 2 greater than(>)space errorlog dot t x t and press Enter.

The previous contents of errorlog dot t x t will be overwritten.

The new error message will replace the old one.

Type cat space errorlog dot t x t and press Enter to see the changed error.

Type wc aaa 2>>errorlog.txt

Press Enter.

But what if we want to keep a record of all errors instead of overwriting them?

We can append errors to a file using:w c space a a a space 2 greater than greater than(>>) errorlog dot t x t press Enter.

Now, all error messages will be added to errorlog dot t x t instead of replacing the previous ones.

To keep all errors instead of overwriting them, we can append them to a file.

cat errorlog.txt press Enter We can check all the accumulated errors by typing:cat space errorlog dot t x t and press Enter.

This will display all the error messages that were appended to the file.

Type

wc aaaa.txt

Press Enter

Show the outputstdin , stdoutput, std error.

Now type:w c space a a a a dot t x t and press Enter.

The error caused by this command is displayed on the terminal.

If no redirection is used, future errors will also be shown on the terminal.

We saw how the three streams, standard output, standard input, and standard error can be redirected separately.

Slide 10

Pipelining

pipelining.png

We can gain more control if we can manipulate multiple streams together.

This process is called pipelining.

Pipes link commands into a chain.The output of one command becomes the input of the next and so on.

A pipeline connects multiple commands using the pipe, that is, vertical bar symbol.

Image: pipe1.png

command1 | command2 | command3 | command4

A pipeline looks like this:

Here, the output of command1 is sent as input to command2, then its output is sent to command3, and so on, forming a chain of commands.

Back to terminal Suppose we want to know the total number of files and directories in the current directory.

We can achieve this using a pipeline of commands.

Let me clear the terminal.
Type cat test1.txt press Enter To see the contents of the file type:cat space test1 dot t x t and press Enter.

Type l s and press Enter.This will display the list of files and directories.

Type: ls -l > test1.txt press Enter To redirect type l s space hyphen l greater than (>) space test1 dot t x t and press Enter.

The file list will be saved in test1 dot t x t instead of the terminal.

Type wc -l test1.txt press Enter Now, to count the total lines, we can use:w c space hyphen l space test1 dot t x t and press Enter.

This gives us the total number of files and directories.

Type ls -l | wc -l press Enter We can do this more efficiently using a pipe:

l s space hyphen l space vertical bar space w c space hyphen l and press Enter.

Here, the output of the l s hyphen l command is sent directly as input to the w c hyphen l command.

The output appears on the terminal, giving the total files and directories without an extra file.

Type cd /usr/bin and press Enter.

Highlight changed directory.

We can create longer chains of commands using pipes.

One common use of pipes is for reading multipage displays.

First, navigate to the slash u s r slash bin directory.

Type c d space slash u s r slash bin and press Enter.

Now, we are in the slash u s r slash bin directory.

Type: ls -l press Enter

Type: ls -l | more press Enter

Now, run:l s space hyphen l and press Enter .

The output is too long to view on the terminal.

Using a pipe with more lets us view it page by page:

Type l s space hyphen l space vertical bar space more and press Enter.

Here, the output of l s hyphen l is passed to more, letting us view one screen at a time.

Press Enter

Press q

Press Enter to scroll through the list one line at a time.

Press q to exit the more command and return to the terminal.

Slide 11

Summary

In this tutorial, you will learn to:

  • Identify stdin, stdout and stderr streams in Linux.
  • Redirect standard output and error to files using redirection operators (>, >>, 2>, and 2>>).
  • Explain file descriptors and their role in redirection.
  • Connect multiple Linux commands using pipes (|) to manipulate data efficiently.
With this we come to the end of this tutorial.

Let us summarise.

Slide 12

Assignment

  1. Create a file named mydata.txt and add 5 lines of text in it.
  2. Count the number of lines in mydata.txt and store the result in lines.txt.
  3. Display mydata.txt page by page in the terminal using more with a pipe.
  4. Display a file nofile.txt and redirect the error message to errorlog.txt.
  5. Count words in mydata.txt and show the result using a pipe.
As an assignment, please do the following.
Slide 13

Thank you

This Spoken Tutorial is brought to you by EduPyramids Educational Services Private Limited SINE IIT Bombay.

Thank you.

Contributors and Content Editors

Ketkinaina, Madhurig