How to insert newline in the argument text of "sed" utility under both single and double quotes?

Question

I'd like to insert multiple lines above a target line in a file by using sed.

The file.txt below contains one line "target line". My initial version is using the single quote:

sed '/target line/ i\
     inserted line1;\
     inserted line2;\
     inserted line3;' file.txt

The result is:

    inserted line1;
    inserted line2;
    inserted line3;
target line

This version works as expected that the newline at the end of each line is escaped by \ to a literal newline instead of a command terminator. Refer to here.

Then I'd like to use shell variable in the replacement string, so I tried to use double quotes to enable the variable expansion:

sed "/target line/ i\
     inserted line1;\
     inserted line2;\
     inserted line3;" file.txt

But this time the newline and the first four spaces disappeared:

inserted line1;    inserted line2;    inserted line3;
target line

How do I correctly insert a newline in double quotes here?

Answer 1

With single-quotes:

The backslash followed by the newline are transmitted as-is to sed. Then sed actually uses the backslash to escape the raw newline character into the string rather than terminating the command. See:

$ printf %s 'hello\
world' | hexdump -C

Which clearly shows the backslash 5c followed by the newline 0a contained in the string.

00000000  68 65 6c 6c 6f 5c 0a 77  6f 72 6c 64              |hello\.world|
0000000c

With double-quotes:

The backslash has special meaning in double-quotes. It causes the following newline character to be interpreted as a string continuation character. The consequence is that neither the backslash or the newline are contained in the string and so not seen by sed.

$ printf %s "hello\
world" | hexdump -C

The string is continued without backslash and without newline:

00000000  68 65 6c 6c 6f 77 6f 72  6c 64                    |helloworld|
0000000a

---

EDIT:

• Precised that sed actually uses the backslash to escape the following newline character as @dan pointed out.

Answer 2

In double quotes, backslash escapes these characters (only):

$`"\

and newline character. So eg echo "\$" prints $.

To preserve the backslash in double quotes, escape it with another backslash:

 sed "/target line/ i\\
     inserted line1;\\
     inserted line2;\\
     inserted line3;" file.txt

Answer 3

If it is not imperative to use sed, there are much easier ways of doing this.

#!/bin/sh

var1=foo
var2=bar
var3=baz

cat > stack <<EOF
${var1}
${var2}
${var3}
EOF

You can likewise create a here document for use with ed, to insert lines before existing ones.

cat > ed1 <<EOF
1i
line1
line2
line3
.
wq
EOF

ed -s existing_file < ed1

That will insert those three lines before line 1 in your existing file, so that what was the first line will become line 4.