Friday, March 29, 2013


Commands and Tools to get network statistics                                                  Netstat Command

If you are working or using unix operating system, then you must be proficient in sed language. Sed command in unix is very helpful for manipulation of file. While writing this article, I am considering this you are aware about the sed’s little basics. Sed is a great utility which solves complex tasks by few lines of code.

Sed stands for Stream Editor. It is very powerful command to file manipulatation, filter lines while reading file and transform text. Sed works both the way i.e. Sed can read file and do operations on it or Sed can take inputs from pipe.  If you are new learner then you will face difficulty to understand the sed command but once you understood then It will be easier to use as you need.

We are going to use below file in all 10 examples.
root@hello:~/test# cat salary.txt
1001,Ajay,Manager,25000
101,Satish,Founder,30000
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000

The above salary database contains the columns as
  •           Employee ID
  •        Name
  •          Designation
  •          Salary

Basic Syntax of sed command:

sed [options] {sed-commands} {input-file}
sed [options] -e {sed-command-1} -e {sed-command-2}{input-file}

Sed reads one line at a time from input file and executes sed - command on that particular line. If we want to execute multiple commands in a single line using several –e arguments, you can split them into multiple lines using a backslash as below:

sed -n \
-e '/^mysql/ p' \
-e '/^other/ p' \
/etc/groups

10 special tricks of Sed as follows :

1.       Print the lines of input file after specific interval i.e. print 1st, 3rd, 5th line of file. Operator   ~ used in an address range. Its special meaning is to skip lines between commands. Below command you can use to print the odd numbered lines or to print the even numbered lines.


root@hello:~/test# sed -n '1~2 p' salary.txt
1001,Ajay,Manager,25000
302,Atul,CEO,26000
1231,Kalyan,Human Resource,20000

2.       Print the lines matching with keyword.


root@hello:~/test# sed -n '/Atul/p' salary.txt
302,Atul,CEO,26000

3.       Print the lines matching with keyword and 2 lines immediately after that. i.e. If matched keyword found then print 3 lines including matched keyword line and 2 lines immediately after that line.

root@hello:~/test# sed -n '/Atul/, +2p' salary.txt
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000

You can use above command for deleting lines in output by using “d” instead of “p” option. i.e. if you want to delete lines starting from 1st match of “Atul” keyword till 4th line of file then you can use it as

sed '/Atul/,4 d' salary.txt

4.       Write the content of salary.txt file to new_salary.txt file. i.e. simple copy the salary.txt file into new_salary.txt file.

root@hello:~/test# sed 'w new_salary.txt' salary.txt
1001,Ajay,Manager,25000
101,Satish,Founder,30000
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000
root@hello:~/test#
root@hello:~/test# cat new_salary.txt
1001,Ajay,Manager,25000
101,Satish,Founder,30000
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000
root@hello:~/test#

5.       Write the lines matching pattern “Atul” to pattern_found.txt file. Below command can be used for write only odd numbered line (1~2) or write only even numbered lines (2~2).

root@ctier:~/test# sed -n '/Atul/ w pattern_found.txt' salary.txt
root@ctier:~/test# cat pattern_found.txt
302,Atul,CEO,26000
root@ctier:~/test#
 You can all above commands by replacing “p” or “d” by “w” option.

Want to know more about : How to run Windows softwares on Linux

6.       Mostly, sed used to substitute pattern by another pattern.

sed '[address-range|pattern-range] s/originalstring/replacement-string/[substitute-flags]' inputfile

Replace all occurrences of Atul with Mohan:

root@ctier:~/test# sed 's/Atul/Mohan/' salary.txt
1001,Ajay,Manager,25000
101,Satish,Founder,30000
302,Mohan,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000
root@ctier:~/test#

Note: We are not changing the file. Sed only changes the display output not a file. If you want to ignore the case the use “I” option at end pattern. i.e. ‘s/Atul/Mohan/i’

Replace Satish with Pravin only on lines that contain thekeyword 'Founder':

root@ctier:~/test# sed '/Founder/s/Satish/Pravin/' salary.txt
1001,Ajay,Manager,25000
101,Pravin,Founder,30000
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
1231,Kalyan,Human Resource,20000
root@ctier:~/test#

7.       Replace the 1st occurrence of lower case “a” with upper case “A” in a line

root@ctier:~/test# sed 's/a/A/' salary.txt
1001,AjAy,Manager,25000
101,SAtish,Founder,30000
302,Atul,CEO,26000
434,RAj,Senior Manager, 26000
1231,KAlyan,Human Resource,20000
root@ctier:~/test#
                Same command we can use for replace all occurrences of “a” in a line by upper case “A” with below command:
               
root@ctier:~/test# sed 's/a/A/g' salary.txt
1001,AjAy,MAnAger,25000
101,SAtish,Founder,30000
302,Atul,CEO,26000
434,RAj,Senior MAnAger, 26000
1231,KAlyAn,HumAn Resource,20000
root@ctier:~/test#

If you want to replace the second occurrence of “a” with “A” in each line then use following syntax: sed 's/a/A/2' salary.txt

Sed flags in short:
g-            global
i-                    Ignore case
p-            Print
w-           Write

You can use this flags as per your need in sed command.

8.       Suppress the meaning of special characters in sed while mataching pattern. Use backslash to remove special meaning.


[root@rhel Documents]# cat test.text
Hello How are you ?
Hello /usr/local/bin directory used to store the commands script
[root@rhel Documents]#
[root@rhel Documents]# sed 's/\/usr\/local\/bin/\/usr\/bin/g' test.text
Hello How are you ?
Hello /usr/bin directory used to store the commands script
[root@rhel Documents]#


9.       Get matched pattern in “&”. Lets see the use of “&” operator.

Enclose the employee id in [EMP ID] and show output.

[root@rhel Documents]# sed 's/^[0-9][0-9][0-9]/[&]/' salary.txt
[100]1,Ajay,Manager,25000
[101],Satish,Founder,30000
[302],Atul,CEO,26000
[434],Raj,Senior Manager, 26000
[123]1,Kalyan,Human Resource,20000
[root@rhel Documents]#
[root@rhel Documents]# sed 's/^[0-9][0-9][0-9][0-9]/[&]/' salary.txt
[1001],Ajay,Manager,25000
101,Satish,Founder,30000
302,Atul,CEO,26000
434,Raj,Senior Manager, 26000
[1231],Kalyan,Human Resource,20000
[root@rhel Documents]# sed -e  's/^[0-9][0-9][0-9]/[&]/' -e 's/^[0-9][0-9][0-9][0-9]/[&]/' salary.txt
[100]1,Ajay,Manager,25000
[101],Satish,Founder,30000
[302],Atul,CEO,26000
[434],Raj,Senior Manager, 26000
[123]1,Kalyan,Human Resource,20000
[root@rhel Documents]#


10.       Use execute flag (e). Run commands inside the files by sed command.

Suppose, we add the ls –l command in file.txt file at starting of each line. Then we want to run these lines then how to run :
[root@rhel Documents]# cat > file.txt
/etc/passwd
/etc/group
[root@rhel Documents]#
root@rhel Documents]# sed -i 's/^/ls -l /' file.txt
[root@rhel Documents]#
[root@rhel Documents]# cat file.txt
ls -l /etc/passwd
ls -l /etc/group
[root@rhel Documents]# sed 's/^/ /e' file.txt
-rw-r--r--. 1 root root 1527 Mar 20 18:18 /etc/passwd
-rw-r--r--. 1 root root 786 Mar 20 18:18 /etc/group
[root@rhel Documents]#


Sed has many more uses. This is the basic sed commands use. I have given here most of the sed Command in Unix with examples. If you need more details you can post on my facebook page.

Jinfo Command                                         Jstat Command           File System in Linux

Posted by Machindra Dharmadhikari On 3/29/2013 01:07:00 AM 4 comments

4 comments:

  1. Very useful for quick revision of SED command.And SED is among the most important and useful command to go through...

    ReplyDelete
  2. Using sed, write a script that takes a filename and a pattern to do the following.

    If the given pattern exists in the file with the very next line starting and ending with the same pattern, delete the line that starts and ends with the given pattern.

    So after running on this file

    =====================
    hai people we had
    a lot of fun

    writing scripts
    We will
    have more
    tomorrow
    Hurray!
    ======================

    should change ro

    ======================
    hai people we had

    writing scripts
    We will
    have more
    tomorrow
    Hurray!
    ======================


    ReplyDelete
  3. This is an informative blog post, are you looking for EAS Anti Theft Solutions find these at affordable price for your any purpose.

    ReplyDelete

  • RSS
  • Delicious
  • Digg
  • Facebook
  • Twitter
  • Linkedin
  • Youtube

    Chitika Ads 2

    Histat

    About

    Enter your email address:

    Delivered by FeedBurner