Preserving Spaces in Array Elements with Mixed Strings and Command-Line Flags

47 Views Asked by At

I dislike bash, I dislike it a lot,

because I'm fighting with array expansion, believe it or not.

I'm using an array to pass arguments to hyperfine, which expects commands as strings,

commands+=("$top_dir/bin/whisper --model medium.en --language en --input {input_file}")
hyperfine --warmup 3 --runs 10 --show-output \
        --parameter-list input_file $ihm_files \
        --export-json $results_path/bench/json/$device/group.json --export-markdown $results_path/bench/md/$device/group.md \
        "${commands[@]}"

but no matter how I unpack it, spaces separate the things:

`error: unexpected argument --model...

the array contains both flags for the bench and the commands being run:

echo_python() {
...
    command="$command {input_file} --device $device --model $model_name --model_dir $model_directory --output_dir $results_path/output/$device/$contender --output_format json"
    echo -e "${command}"
}
...
commands=()
commands+=(--prepare "use_venv ${top_dir}/original")
commands+=("'$(echo_python ${top_dir}/original whisper ${model_path} medium.en $device)'")

but anyway I expand it, the commands come undone!

Let me list what I've tried and failed, although it does sting:

  1. I've obviously tried to pass it as a string
commands+=("$top_dir/bin/whisper --model medium.en --language en --input {input_file}")
  1. I've tried unpacking with a wildcarded array, but then its contents became one string as the error will display:
hyperfine --warmup 3 --runs 10 --show-output \
        --parameter-list input_file $ihm_files \
        --export-json $results_path/bench/json/$device/group.json --export-markdown $results_path/bench/md/$device/group.md \
        "${commands[*]}"
`error: unexpected argument '--prepare use_venv ... --cpu'' found`
  1. I've even tried escaping quotes in this stupid way, but it was same expansion as before, to my growing dismay.
commands+=(--prepare "\'use_venv ${top_dir}/original\'")
  1. I tried escaping the spaces but nothing changed. I thought that would solve it? expansion is strange

I want to use an array, so I can arbitrarily add more commands,

but no matter how it's unpacked, I cannot meet this demand.

I got a little closer once I unquoted the substitution:

$results_path/bench/md/$device/group.md \
        ${commands[@]}

now the --prepare flag is recognized, though that's not quite the solution...

so is there anyway to preserve spaces in this array of strings and params,

so that I might put this to bed and finish this program?

EDIT

  1. I tried changing IFS and testing the entries with this:
OLD_IFS="$IFS"
    IFS=$""
    commands+=(--prepare "use_venv $top_dir/original")
    commands+=("$(echo_python ${top_dir}/original whisper ${model_path} medium.en $device)")

    ...
    IFS="$OLD_IFS"
    for command in "${commands[@]}"; do
        echo $command '|'
    done

the spacing seems correct but hyperfine complains that the number of --prepares mismatches commands what gives?

--prepare |
use_venv /home/Daedalus/Workspace/whisper-bench/original |
whisper {input_file} --device cpu --model medium.en --model_dir /home/Daedalus/Workspace/whisper-bench/models --output_dir /home/Daedalus/Workspace/whisper-bench/results/output/cpu/original --output_format json |
--prepare |
use_venv /home/Daedalus/Workspace/whisper-bench/ctranslate2 |
whisper-ctranslate2 {input_file} --device cpu --model medium.en --model_dir /home/Daedalus/Workspace/whisper-bench/models --output_dir /home/Daedalus/Workspace/whisper-bench/results/output/cpu/ctranslate2 --output_format json |
--prepare |
true |
/home/Daedalus/Workspace/whisper-bench/bin/whisper --model medium.en --language en --input {input_file} --cpu |
Error: The '--prepare' option has to be provided just once or N times, where N is the number of benchmark commands.
0

There are 0 best solutions below