It came up at work that I had to find ~700 numerical IDs in a file and delete
every line in which these IDs occur. Doing this by hand would have taken me forever and
probably also wouldn’t be the best use of my time. I had a hunch that tools like grep and
sed might be useful for this kind of task so I decided to give it a try and automate it.
This post is just focusing on the xargs part of the script I eventually used, so here is
the (slightly simplified) command in it’s entirety first:
$ cat list_of_ids.txt | xargs -I '{}' sed -i '' 's/{}: true,//g' target_file.txt
Let’s split this up:
cat is a unix command which prints the entire content of a given file. We then use the pipe (|)
command to use the stdout of the cat command as the stdin of the next command.
Given that we want to invoke the sed command for every line of our input file instead of once for
the entire content, we need a command in between cat and sed.
That’s where xargs helps us out.
According to its man page it does the following:
The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments.
utility in our case being the sed command.
We have to be able to use the strings passed from the xargs command within the sed command. That’s why we pass
the -I option: It tells the utility command (in this case sed) to replace occurrences of {}
within the arguments of the utility command with each input line of xargs.
Given a list_of_ids.txt with:
123
444
789
Behind the scenes this will result in the following sed invocations:
$ sed -i '' 's/123: true,//g' target_file.txt
$ sed -i '' 's/444: true,//g' target_file.txt
$ sed -i '' 's/789: true,//g' target_file.txt
Which is exactly what we wanted! This script only used a few minutes of my time, it’s reusable and let me get back to my other tasks much more quickly than doing all this by hand.
N.B. The
sedcommand replaced the matched lines with an empty string as it seemed like quite a struggle to remove the entire line withsed. So I was pragmatic and removed all the leftover empty lines with just one quick command in my editor afterwards.