Coloring output of a script that overwrites lines?

Question

I'm using this to color the output of a script/command:

commandWithOutput | sed -r 's/(pattern)/'"${COLOR_RED}"'\1'"${COLOR_DEFAULT}"'/g'

(This will color all occurences of string "pattern" in the command's output.) And it works fine with traditional commands. However, if the script/command overwrites lines in its output (maybe this has more to do with a terminal/console than just standard output?), e.g.:

Building project X:
CXX Building file XYZ.cpp... [123/1034]

the behavior isn't as expected. My sed will still color the output but the overwriting doesn't work anymore, i.e.:

Building project X:
CXX Building file ABC.cpp... [1/1034]
CXX Building file DEF.cpp... [2/1034]
CXX Building file GHI.cpp... [3/1034]
CXX Building file JKL.cpp... [4/1034]
CXX Building file MNO.cpp... [5/1034]
// and so on...
CXX Building file XYZ.cpp... [123/1034]

Is there a way to color the output of a script/command that overwrites lines?

Answer 1

I've tried several different ideas ... IFS=$'\r' + OP's sed command ... trying to use an intermediate pipe (mkfifo) for processing the output from commandWithOutput ... a few attempts at trying to unbuffer stdout and/or stdin ... but (so far) could only get a awk solution to work, so fwiw ...

NOTE: I'm assuming OP's command is generating a \r when overwriting a line; if this is not the case the OP can try piping their command's output to | od -c to see what character is at the 'end of the line', with the idea being to use said character in place of my \r references (below).

---

First we'll write a small script to generate some data, (re)printing over the first few lines, and then printing some 'standalone' lines:

$ cat overwrite
#!/usr/bin/bash

for (( i=1 ; i<="${1}" ; i++ ))
do
    printf "this is a test ... ${i}\r"
    sleep 1
done

printf "\nanother test output \t and a tab\n"

echo "X."

Running the above generates the following output:

$ overwrite 3
this is a test ... 3                      << this line is actually printed 3x times with suffixes of '1', '2' and '3'
another test output      and a tab
X.

Running this through od shows the \r at the end of the first 3 lines:

$ overwrite 3 | od -c
0000000   t   h   i   s       i   s       a       t   e   s   t       .
0000020   .   .       1  \r   t   h   i   s       i   s       a       t
0000040   e   s   t       .   .   .       2  \r   t   h   i   s       i
0000060   s       a       t   e   s   t       .   .   .       3  \r  \n
0000100   a   n   o   t   h   e   r       t   e   s   t       o   u   t
0000120   p   u   t      \t       a   n   d       a       t   a   b  \n
0000140   X   .  \n
0000143

---

We'll now look at one awk solution for recoloring a specific pattern in the output from our overwrite script ...

First we'll define the start and clear/reset variables for our desired color; for this exercise I'm going to use 'red':

$ myred=$(tput setaf 1)           # set our highlight color to red
$ myreset=$(tput sgr0)            # disable coloring

NOTE: There are a few ways to define these colors (and the disable/reset); I'll leave that up to the reader to pick what works best in their environment.

Here's one awk solution I found that works:

$ overwrite 3 | awk -v ptn="test" -v cstart="${myred}" -v creset="${myreset}" -v RS="[\n\r]" '{ sub(ptn,cstart ptn creset) ; printf $0 RT }'

Where:

• -v ptn="test" - we want to recolor all instances of the string test; we'll pass this in as awk variable ptn
• -v cstart="${myred}" - assign our highlight color code (red) to our awk variable cstart
• -v creset="${myreset}" - assign our color clear/reset code to the awk variable creset
• -v RS="[\n\r]" - redefine our input record separator as either \r or \n
• sub(ptn,cstart ptn creset) - replace all instances of test with <red> + test + <reset>
• printf $0 RT - print our new line; RT allows us to make use of the same RS that was used to parse out this record

Running the above generates:

this is a test ... 3                      << this line is actually printed 3x times with suffixes of '1', '2' and '3', and the 'test' string printed in red
another test output      and a tab        << the 'test' string is printed in red
X.