Advanced tricks to use sed Command for File Manipulation

How to install and configure Wine                                          Netstat Command in Linux

In last article, we saw basic use of sed command but we can use sed command as like awk as well. In this article, we will overlook the advanced use of sed command in file manipulation operation as well as grouping functionality. Before going to the example if you are new to this post then please go through this link first:

Basic use of Sed command in file manipulation.

Don’t confuse with flags and options, options which we used with hyphen and flags which we use in single quotes. Sed has many flags. Below are the flags which we need in next couple of examples:

Flag	Meaning	Description
g	global	This flag will help to replace pattern in whole file at multiple occurrences
i	Ignore case(present in GNU Sed)	This flag will help to match the pattern ignoring case of pattern.
w filename	Write the output	This flag will help to write the desired output in filename which you have specified.
P	Print the lines	It will help to print the desired output at your screen

Some of the examples for these flags as below:

Flag i:

root@rhel:~/test# cat salary.txt 1001,Ajay,Manager,25000 101,Satish,Founder,30000 302,Atul,CEO,26000 434,Raj,Senior Manager, 26000 1231,Kalyan,Human Resource,20000 root@rhel:~/test# root@rhel:~/test# sed 's/atul/Raman/i' salary.txt 1001,Ajay,Manager,25000 101,Satish,Founder,30000 302,Raman,CEO,26000 434,Raj,Senior Manager, 26000 1231,Kalyan,Human Resource,20000 root@rhel:~/test#

Flag g: (Replacing “a” by “R” everywhere and with ignoring case of “a”.

root@rhel:~/test# sed 's/a/R/ig' salary.txt 1001,RjRy,MRnRger,25000 101,SRtish,Founder,30000 302,Rtul,CEO,26000 434,RRj,Senior MRnRger, 26000 1231,KRlyRn,HumRn Resource,20000 root@rhel:~/test#

For flag “w” and flag “p”, I have provided examples in this article: Basic use of Sed command infile manipulation.

Let’s go for some advanced examples of sed. 

1.       How to change the substitution delimiter of sed ?

Suppose we want to replace the path (/usr/local/bin/) by some different path (/opt/bin/mysql/) then it is very difficult to suppress the meaning of each “/” in sed. To avoid complexity, we will use different delimiter instead of “/”. So, it is very easy to use any delimiter among these:  “|” or “^” or “@” or “!”.

sed 's|/usr/local/bin|/opt/bin/mysql/|' path.txt sed 's^/usr/local/bin^/opt/bin/mysql/^' path.txt sed 's@/usr/local/bin@/opt/bin/mysql/@' path.txt sed 's!/usr/local/bin!/opt/bin/mysql/!' path.txt

I haven’t provided output there, because for all command’s output will be same.

Recommended Article: Basic Use of Sed command in Linux

2             Use sed command to fetch 1^st and 3^rd column of data with separator “,”.

root@rhel:~/test# sed 's/\([^,]*\),\([^,]*\),\([^,]*\),\([^,]*\).*/\1, \3/g' salary.txt 1001, Manager 101, Founder 302, CEO 434, Senior Manager 1231, Human Resource root@rhel:~/test#

sed 's/\([^,]*\),\([^,]*\),\([^,]*\),\([^,]*\).*/\1, \3/g' salary.txt

\([^,]*\) – This is the first Column. First, we removed special meaning of “(“ and then [^,]  “,” separator provided in square brackets. We needed first and third column that’s why we defined 4 column definitions. We can extend it as per our requirement.

Like above example, you can use sed command as like awk or cut command.

3.       GNU sed having some replacement flags as below:

\l	Considers one character in lower case	Sed ‘s/Atul/RA\lMAN/g’	Atul will replace by “RAmAN”
\L	Considers all characters after this flag in lower case	Sed ‘s/Atul/RA\LMAN/g’	“Atul’ will replace by “Raman”
\u	Same as \l i.e. one character consider in upper case	Sed‘s/Atul/ra\uman/g’	“Atul” will replace by “raMan”
\U	Same as \L but it will convert char into upper case	Sed‘s/Atul/ra\Uman/g’	“Atul” will replace by “raMAN”

If we need whole string in upper case then “\U”at beginning of pattern. Ex. /\U ramanDev/ then output pattern will be: “RAMAN DEV” but suppose we want to “RAMAN” only in capital and remained pattern as it is then use “\E” flag after “raman” as like : /\Uraman\E Dev/ its output will be “RAMAN Dev”.

Regular Expression in Sed command

Beginning of Line (^)

The carrot symbol matches at the start of line. Ex. Display lines which starts with 1001

root@ctier:~/test# sed -n '/^1001/p' salary.txt 1001,Ajay,Manager,25000 root@ctier:~/test#

In above example, used –n option to display only commands output otherwise sed displays files content as well as command executed output as well.

End of line ( $)

The dollar symbol $ matches the end of a line. Ex. Display lines which end with the number 20000:

root@ctier:~/test# sed -n '/20000$/p' salary.txt 1231,Kalyan,Human Resource,20000 root@ctier:~/test#

Single Character (.)

The special meta-character dot matches any character except end of line character. “.” Single dot matches single character, “..” double dot character matches two character and so on ..

root@ctier:~/test# sed -n '/1..1/p' salary.txt 1001,Ajay,Manager,25000 1231,Kalyan,Human Resource,20000 root@ctier:~/test#

Zero or more occurrences (*)

This special character “*” matches the zero or more occurrences previous characters.  Ex. ‘/Q*/p’ means – It will matches zero or more “Q”.

This feature useful while finding the error in log file. You can find “Error:  *.” If there are spaces before and after the “Error” keyword then It will find out lines by ignoring the spaces.

How to Run Unix Command in Windows                  Get Java Process Statistics information