I have a file abc.txt with following data.
This is is how it works buddy
What else else you want
Remove repeated words with SED as given below.
sed -ri ‘s/(.* )1/1/g’ abc.txt
This is how it works buddy
What else you want
Let me explain sed command which we used.
-r option is for enabling Extended Regular Expression which have grouping option with () braces.
-i option for inserting the changes to original file, Be careful with this option as you can not get your original file once modified.
(.* ) for mentioning any group of characters and which is followed by same set of characters which is represented by 1. This concept is called back reference, where 1 can store first set of characters enclosed in first (). And these two things (.* )1 is replaced by same word with 1 which is actual back reference to first (.* ).