Hidden characters added while converting from dos2unix

Question

Input file in comma separated .csv format:

25 Mbps^M
25 Mbps^M
25 Mbps^M
0 Mbps ^M
0 Mbps ^M
0 Mbps ^M
0 Mbps ^M
ash Back  ^M
ash Back  ^M
ash Back  ^M

while converting from dos2unix, the spaces before ^M is getting converted to hidden characters.

Need to get rid of it the spaces so that on converting from dos2unix hidden characters are not present.

Tried using sed, tr but nothing worked

for filename in *.csv
    do
        echo "Converting $filename to UNIX format!!!!!\n \n"

        tr -d '\r' $filename > test_2
        #Converting file into Unix format and moving to input dir
        cp test_2 $filename
        dos2unix $filename > /input/$filename
    done

After convertion from dos2unix, file should not contain any hidden characters.

Answer 1

As @Oguz Ismail said, dos2unix removes CR from the data; what you call "hidden characters" are what are commonly called "spaces" (usually made by pressing on the large bar at the bottom of your keyboard).

You can achieve what you want with GNU sed:

sed 's/[[:space:]]*$//g' -i *.csv

• [[:space:]]: Space characters: in the ‘C’ locale, this is tab, newline, vertical tab, form feed, carriage return, and space. ^[reference]
• -i: specifies that files are to be edited in-place. ^[reference]

You do not have GNU sed:

for filename in *.csv
do
  sed 's/[[:space:]]*$//g' "$filename" > "${filename}.out"
  mv -f "${filename}.out" "${filename}"
done