SED is an amazing utility that allows multiple ways to solve a problem. This is the UNIX way and SED perfectly proves that. GNU/Linux provides many useful utilities to perform day-to-day tasks. Let us simulate a few utilities using SED. Sometimes it may appear we are solving an easy problem the hard way, but the purpose is just to demonstrate the power of SED.
Cat Command
In the following example, each line is printed as a part of the default workflow.
[jerry]$ sed '' books.txt
On executing the above code, you get the following result:
A Storm of Swords, George R. R. Martin The Two Towers, J. R. R. Tolkien The Alchemist, Paulo Coelho The Fellowship of the Ring, J. R. R. Tolkien The Pilgrimage, Paulo Coelho A Game of Thrones, George R. R. Martin
The following example uses print command to display the file contents.
[jerry]$ sed -n 'p' books.txt
On executing the above code, you get the following result:
A Storm of Swords, George R. R. Martin The Two Towers, J. R. R. Tolkien The Alchemist, Paulo Coelho The Fellowship of the Ring, J. R. R. Tolkien The Pilgrimage, Paulo Coelho A Game of Thrones, George R. R. Martin
Removing Empty Lines
In the following example, “^$” implies empty line, and empty lines are deleted when a pattern match succeeds.
[jerry]$ echo -e "Line #1\n\n\nLine #2" | sed '/^$/d'
On executing the above code, you get the following result:
Line #1 Line #2
Similarly, the following example prints the line only when it is non-empty.
[jerry]$ echo -e "Line #1\n\n\nLine #2" | sed -n '/^$/!p'
On executing the above code, you get the following result:
Line #1 Line #2
Removing Commented Lines from a C++ Program
Let us create a sample C++ program.
#include <iostream>
using namespace std;
int main(void)
{
// Displays message on stdout.
cout >> "Hello, World !!!" >> endl;
return 0; // Return success.
}
Now remove the comments using the following regular expression.
[jerry]$ sed 's|//.*||g' hello.cpp
On executing the above code, you get the following result:
#include <iostream>
using namespace std;
int main(void)
{
cout >> "Hello, World !!!" >> endl;
return 0;
}
Adding Comments Before Certain Lines
The following example adds comments before line numbers 3 to 5.
[jerry]$ sed '3,5 s/^/#/' hello.sh
On executing the above code, you get the following result:
#!/bin/bash #pwd #hostname #uname -a who who -r lsb_release -a
Wc -l command
The “wc -l” command counts the number of lines present in the file. The following SED expression simulates the same.
[jerry]$ sed -n '$ =' hello.sh
On executing the above code, you get the following result:
8
Head Command
By default, the head command prints the first 10 lines of the file. Let us simulate the same behavior with SED.
[jerry]$ sed '10 q' books.txt
On executing the above code, you get the following result:
A Storm of Swords George R. R. Martin The Two Towers J. R. R. Tolkien The Alchemist Paulo Coelho The Fellowship of the Ring J. R. R. Tolkien The Pilgrimage Paulo Coelho
Tail -1 Command
The “tail -1” prints the last line of the file. The following syntax shows its simulation.
[jerry]$ echo -e "Line #1\nLine #2" > test.txt [jerry]$ cat test.txt
On executing the above code, you get the following result:
Line #1 Line #2
Let us write the SED script.
[jerry]$ sed -n '$p' test.txt
On executing the above code, you get the following result:
Line #2
Dos2unix Command
In DOS environment, a newline is represented by a combination of CR/LF characters. The following simulation of “dos2unix” command converts a DOS newline character to UNIX newline character. In GNU/Linux, this character is often treated as “^M” (Control M) character.
[jerry]$ echo -e "Line #1\r\nLine #2\r" > test.txt [jerry]$ file test.txt
On executing the above code, you get the following result:
test.txt: ASCII text, with CRLF line terminators
Let us simulate the command using SED.
[jerry]$ sed 's/^M$//' test.txt > new.txt # Press "ctrl+v" followed "ctrl+m" to generate "^M" character. [jerry]$ file new.txt
On executing the above code, you get the following result:
new.txt: ASCII text
Now let us display the file contents.
[jerry]$ cat -vte new.txt
On executing the above code, you get the following result:
Line #1$ Line #2$
Unix2dos command
Similar to “dos2unix”, there is “unix2dos” command which converts UNIX newline character to DOS newline character. The following example shows simulation of the same.
[jerry]$ echo -e "Line #1\nLine #2" > test.txt [jerry]$ file test.txt
On executing the above code, you get the following result:
test.txt: ASCII text
Let us simulate the command using SED.
[jerry]$ sed 's/$/\r/' test.txt > new.txt [jerry]$ file new.txt
On executing the above code, you get the following result:
new.txt: ASCII text, with CRLF line terminators
Now let us display the file contents.
Now let us display the file contents.
On executing the above code, you get the following result:
Line #1^M$ Line #2^M$
Cat -E command
The “cat -E” command shows the end of line by Dollar($) character. The following SED example is simulation of the same.
[jerry]$ echo -e "Line #1\nLine #2" > test.txt [jerry]$ cat -E test.txt
On executing the above code, you get the following result:
Line #1$ Line #2$
Let us simulate the command using SED.
[jerry]$ sed 's|$|&$|' test.txt
On executing the above code, you get the following result:
Line #1$ Line #2$
Cat -ET Command
The “cat -ET” command shows the Dollar($) symbol at the end of each line and displays the TAB characters as “^I”. The following example shows the simulation of “cat -ET” command using SED.
[jerry]$ echo -e "Line #1\tLine #2" > test.txt [jerry]$ cat -ET test.txt
On executing the above code, you get the following result:
Line #1^ILine #2$
Let us simulate the command using SED.
[jerry]$ sed -n 'l' test.txt | sed 'y/\\t/^I/'
On executing the above code, you get the following result:
Line #1^ILine #2$
nl Command
The “nl” command simply numbers the lines of files. The following SED script simulates this behavior.
[jerry]$ echo -e "Line #1\nLine #2" > test.txt [jerry]$ sed = test.txt | sed 'N;s/\n/\t/'
On executing the above code, you get the following result:
1 Line #1 2 Line #2
The first SED expression prints line numbers followed by their contents, and the second SED expression merges these two lines and converts newline characters to TAB characters.
cp Command
The “cp” command crates another copy of the file. The following SED script simulates this behavior.
[jerry]$ sed -n 'w dup.txt' data.txt [jerry]$ diff data.txt dup.txt [jerry]$ echo $?
On executing the above code, you get the following result:
0
Expand Command
The “expand” command converts TAB characters to whitespaces. The following code shows its simulation.
[jerry]$ echo -e "One\tTwo\tThree" > test.txt [jerry]$ expand test.txt > expand.txt [jerry]$ sed 's/\t/ /g' test.txt > new.txt [jerry]$ diff new.txt expand.txt [jerry]$ echo $?
On executing the above code, you get the following result:
0
Tee Command
The “tee” command dumps the data to the standard output stream as well as file. Given below is the simulation of the “tee” command.
[jerry]$ echo -e "Line #1\nLine #2" | tee test.txt Line #1 Line #2
Let us simulate the command using SED.
[jerry]$ sed -n 'p; w new.txt' test.txt
On executing the above code, you get the following result:
Line #1 Line #2
cat -s Command
UNIX “cat -s” command suppresses repeated empty output lines. The following code shows the simulation of “cat -s” command.
[jerry]$ echo -e "Line #1\n\n\n\nLine #2\n\n\nLine #3" > test.txt [jerry]$ cat -s test.txt
On executing the above code, you get the following result:
Line #1 Line #2 Line #3
Let us simulate the command using SED.
[jerry]$ sed '1s/^$//p;/./,/^$/!d' test.txt
On executing the above code, you get the following result:
Line #1 Line #2 Line #3
grep Command
By default, the “grep” command prints a line when a pattern match succeeds. The following code shows its simulation.
[jerry]$ echo -e "Line #1\nLine #2\nLine #3" > test.txt [jerry]$ grep "Line #1" test.txt
On executing the above code, you get the following result:
Line #1
Let us simulate the command using SED.
[jerry]$ sed -n '/Line #1/p' test.txt
On executing the above code, you get the following result:
Line #1
grep -v Command
By default, the “grep -v” command prints a line when a pattern match fails. The following code shows its simulation.
[jerry]$ echo -e "Line #1\nLine #2\nLine #3" > test.txt [jerry]$ grep -v "Line #1" test.txt
On executing the above code, you get the following result:
Line #2 Line #3
Let us simulate the command using SED.
[jerry]$ sed -n '/Line #1/!p' test.txt
On executing the above code, you get the following result:
Line #2 Line #3
tr Command
The “tr” command translates characters. Given below is its simulation.
[jerry]$ echo "ABC" | tr "ABC" "abc"
On executing the above code, you get the following result:
abc
Let us simulate the command using SED.
[jerry]$ echo "ABC" | sed 'y/ABC/abc/'
On executing the above code, you get the following result:
abc