Basic tutorial of SED (Stream Editior) for beginners

What is SED?

Sed is stream editor and ultimate editor (non-interactive text editor)for modifying files automatically. Commonly used in the Linux/Unix based system. Sed inputs in the form of a stream and update the stream or input depends on the instructions.

Many System developers or admins use this commands on daily basis to update or replace text or filter from the strings or files.

How to use?

I will use the given file reference to explain the commands:

for seq in `seq 1 5`; do echo “CAT_$seq” >> exp.txt; done

the above command will create file “exp.txt” which has content CAT_1 . to CAT_5 separated by lines.

Delimiter IN SED:

Most of the people know that only ‘/’ slash is a delimiter this is a myth you can use like “|”, “,”, “_”, “:” etc.

Example:

echo "CAT"| sed 's:CAT:DOG:'
echo "CAT"| sed 's|CAT|DOG|'
echo "CAT"| sed 's_CAT_DOG_'
echo "CAT"| sed 's;CAT;DOG;'
echo "CAT"| sed 's,CAT,DOG,'

so all above command yields the same result.

How to print line no using sed:

Using “=” we can print line and line no:

Example:

sed ‘=’ exp.txt

OUTPUT:

1
CAT_1
2
CAT_2
3
CAT_3
4
CAT_4
5
CAT_5

Print file using SED:

Example: Print from line no1 to 5.

sed '1,3p' exp.txt

OUTPUT:

CAT_1
CAT_1
CAT_2
CAT_2
CAT_3
CAT_3
CAT_4
CAT_5

By default, each line of input is printed to the standard output, after all of the commands have been applied to it to suppress this behavior we have -n

sed -n '1,5p' exp.txt

OUTPUT:

CAT_1
CAT_2
CAT_3

Print Non-consecutive lines:

How to print non-consecutive lines like print from line 1to3 and 5.

Example:

sed -n -e '1,3p' -e '5p' exp.txt

Output:

CAT_1
CAT_2
CAT_3
CAT_5

Here we have used -e flag basically means append the editing commands specified by the command argument to the list of commands.

it’s similar to execute multiple sed commands same as below.

sed -n '1,3p'  exp.txt ; sed -n 5p exp.txt

Delete Lines and Print:

How to delete or remove some of the lines and print rest all the lines.

Example:

sed '3d'  exp.txt

so above command delete 3rd line and print all the lines.

Output:

CAT_1
CAT_2
CAT_4
CAT_5

Inserting spaces in files:

Using “G” we can insert an empty line with every non-empty line present in the file.

Example:

sed ‘G’ exp.txt

Output:

CAT_1

CAT_2

CAT_3

CAT_4

CAT_5

you can also do sed ‘G; G’ exp.txt to insert 2 blank lines, similarly, no of G’s separated by semicolon insert blank line same as no of “G”

In Place Editing in Sed:

Using the “-i” flag we can edit the file in place and changes are updated in the same file without printing output of file in the console.

Example:

sed -in 's/CAT/DOG/' exp.txt

Output:

DOG_1
DOG_2
DOG_3
DOG_4
DOG_5

Occurrences of pattern in SED:

Without giving any occurrence first matched character is replaced on giving “g” flag all occurrences are replaced. In case if you want to modify a particular pattern in sed then you can do like below.

Example:

echo "CAT CAT CAT CAT CAT"| sed 's/CAT/DOG/2'

OUTPUT: CAT DOG CAT CAT CAT

as you can see in the above example the second occurrence is replaced.

if you want to replace from second onwards you can do like this

echo "CAT CAT CAT CAT CAT"| sed 's/CAT/DOG/2g'

OUTPUT: CAT DOG DOG DOG DOG

Command S for substitution:

it will replace the occurrence of pattern to a newly given pattern

Replace String using String:

Example: Let's replace CAT_2 to DOG_2

sed ‘s/CAT_2/DOG_2/’ exp.txt

          or

cat exp.txt | sed 's/CAT_2/DOG_2/'

OUTPUT:

CAT_1
DOG_2
CAT_3
CAT_4
CAT_5

It will replace CAT_2to DOG_2

Note: Most of the Linux utilities works on reading the file line by line similarily sed works, in the same way, it will replace the first occurrence of pattern and go to next line if you want to replace all the occurrences then use “g” means global.

Example:

sed ‘s/cat_2/DOG_2/g’ exp.txt

or

cat exp.txt | sed 's/CAT_2/DOG_2/g'

We can also uses a number instead of “g” which will tell every number th position character is replaced

Example:

echo “my name is name and name” | sed ‘s/name/saurav/2’

Output: my name is saurav and name

second position name is replaced with saurav

Replace String using REGEX:

Example Replace CAT from DOG

sed ‘s/^CAT*/DOG/’ exp.txt

OUTPUT:

DOG_1
DOG_2
DOG_3
DOG_4
DOG_5

Sometimes we used -E flag while regex matching in sed for example

sed -E ‘s/^CAT*/DOG/’ exp.txt

This is an extended regular expression flag, this means the behavior of a few characters: ‘?’, ‘+’, ‘()’,’{}’ etc does not require to escape while in regular (or not using -E flag) we need to escape. Extended regular expressions have more power than normal

Example:

, but sed scripts that treated “+”

echo “123 abc” | sed ‘s/[0–9]+//’

Output: 123 abc

echo “123 abc” | sed -E‘s/[0–9]+//’

Output: abc

so in above example as you can “+ ” is special character when use “-E” sed take as regular expression where as without “-E” sed take as normal string.

That's it after going through this article you can get an idea of how sed works and different flags present in flags. Some of the flags are not covered like”r” (for reading from the file), “w” for writing in file etc. these are basic and easy sed flags.

In case of any doubts or concerns please comment below.

Happy Coding . :)