How to use a string with arguments to call a script as bash would do it when interprets the command line?

Question

The desired outcome

Is there a way to use a string that contains the arguments to call a script?

str_params="this/one 'that one' and \"yet another\""

The function below prints feedback on the stdout on how the arguments were received:

display_args () {
  all_args=("${0}" "${@}")
  for i in "${!all_args[@]}"; do 
    printf "  $%d is '%q'\n" "${i}" "${!i}"
  done
}

The desired result would be the below, where array expansion is use instead (see ary_params is used, in split of str_params):

ary_params=(this/one 'that one' and "yet another")
display_args "${ary_params[@]}"
 $0 is 'bash'
 $1 is 'this/one'
 $2 is 'that\ one'
 $3 is 'and'
 $4 is 'yet\ another'

The Problem

The first attempt using str_params shows that the string is split by blank space when transformed into arguments, regardless of the single and double quotes:

$ display_args ${str_params}
 $0 is 'bash'
 $1 is 'this/one'
 $2 is '\'that'
 $3 is 'one\''
 $4 is 'and'
 $5 is '\"yet'
 $6 is 'another\"'

The second attempt throws an error that am still trying to understand. Does bash really try to find a file called all that string?

$ cmd="display_args ${str_params}"
$ echo "${cmd}"
display_args this/one 'that one' and "yet another"
$ "$cmd"
bash: display_args this/one 'that one' and "yet another": No such file or directory

• And calling with just $cmd (with no double quotes) brings us back to the result of the first attempt (further up).

Research

There is somehow good documentation and posts around this particular problem. Just to mention some:

1. This FAQ suggests building an array with the arguments (as mentioned in this question) and expanding its elements, some caveats around parameter expansion, the IFS= read -r ramblings and some other unrelated example. Digging in a bit on passing array arguments to bash scripts, this answer clearly reflects that, but for some work around that may work in some circumstances, this is not a well supported approach (it also refers to the bash author's quote: there isn't really a good way to encode an array variable into the environment).

2. This answer (bash splitting line with quotes into parameters), clearly reflects the difference on how the shell treats commands vs strings. The shell performs quote removal as the last step, before executing the command. In contrast, when interpreting the str_params variable, the shell treats the quotes as just another character and, as a consequence, they are not subject of quote removal. Most surprisingly, in another answer of the same post, there seems to be some example to re-write the arguments list (whether or not that can be a universal approach).

3. This other answer (how to iterate over arguments in bash script) explains that the shell processes quotes before it expands variables, and to make the shell paying attention to the quotes in the variable (str_params and cmd in the examples above) it is required to use eval. However the use of eval seems rather a risky approach and I am still doubtful there aren't other solutions for something that, in principle, seems easier than this.

Using eval

By following the examples above, I gave a try to eval and seems to work:

$ echo "$cmd"
display_args this/one 'that one' and "yet another"
$ eval $cmd
 $0 is 'bash'
 $1 is 'this/one'
 $2 is 'that\ one'
 $3 is 'and'
 $4 is 'yet\ another'
$ eval display_args $str_params
 $0 is 'bash'
 $1 is 'this/one'
 $2 is 'that\ one'
 $3 is 'and'
 $4 is 'yet\ another'

The Questions

• Is there a way to use a string containing the arguments we will use to pass them to a script without the use of eval?
• Alternatively, is there a straight forward way to transform that string into an Array of parameters seemingly to what bash would do? (this would allow to just call by extending the elements of the array, as mentioned above this lines).

Answer 1

Typically, xargs can do that. Xargs can't handle newlines inside quotes, a newline ends arguments and starts another line of arguments. When, for example, you want to run a command multiple times and have arguments on separate lines in a file, xargs is just the perfect tool for that.

$ echo "this/one 'that one' and \"yet another\"" | xargs -t printf "%q\n"
printf '%q\n' this/one 'that one' and 'yet another'
this/one
'that one'
and
'yet another'

The good solution is not to store arguments in a string in the first place. Serialize arguments in something you can read easily in Bash. For example, each argument on a separate line or zero separated. Or output from declare -p.

Another solution, is not to use Bash. Use python with shlex.split().

Is there a way to use a string containing the arguments we will use to pass them to a script without the use of eval?

Yes, write a parser that parses the input into arguments. For every character, tokenize, detect if it's quotes or not, handle escape sequences, and build the array of arguments yourself. Like you would in any other programming language. Or use an external existing program that exactly does that work. Just like xargs or shlex.split() in python.

Looks easy enough to use xargs to load it to an array:

$ readarray -t -d '' args < <(echo "this/one 'that one' and \"yet another\"" | xargs -t printf "%s\0"); declare -p args
printf '%s\0' this/one 'that one' and 'yet another'
declare -a args=([0]="this/one" [1]="that one" [2]="and" [3]="yet another")

With Python, you can properly handle newlines, but if you have to use Python for your Bash script, you might as well rewrite your whole Bash to Python:

$ readarray -t -d '' args < <(python -c 'import sys, shlex; print("\0".join(shlex.split(sys.argv[1])), end="")' "this/one 'that "$'\n'"one' and \"yet another\""); declare -p args
declare -a args=([0]="this/one" [1]=$'that \none' [2]="and" [3]="yet another")

Does bash really try to find a file called all that string?

Yes, you quoted "$cmd", so it's one argument and all the content of cmd is one argument. And because it's the first on the line, it's the name of the command. Note also, that unquoted expansion undergo word splitting and also filename expansion! Lucky you didn't test with * in the string to notice it. Quoting is really important in shell. Remember to check your scripts with shellcheck.