Ansible Roles and vars/vault.yml file

Question

I usually use encrypt_string for encrypting secrets in my ansible roles but I have a new boss and he insists on encrypting the whole file but I cannot not seem to get that to work with an Ansible role.

Here is a simple example that illustrates what my problem is.

bash-3.2$ tree
.
├── playbooks
│   └── run_foo.yml
└── roles
    └── foo
        ├── tasks
        │   └── main.yml
        └── vars
            ├── main.yml
            └── vault.yml

5 directories, 4 files

Here is a playbook that uses the role.

bash-3.2$ cat playbooks/run_foo.yml
---
- name: Run Foo
  hosts: all
  roles:
    - role: ../roles/foo

Here's the role:

bash-3.2$ cat roles/foo/tasks/main.yml
---
# tasks file for foo
- name: Show us debug clear_text
  debug:
    var: clear_text
- name: Show us debug password
  debug:
    var: password

Here is my vars/main.yml file:

bash-3.2$ cat roles/foo/vars/main.yml
---
# vars file for foo
include_vars: vault.yml
clear_text: 12345

Here is my encrypted vault.yml file:

bash-3.2$ cat roles/foo/vars/vault.yml
$ANSIBLE_VAULT;1.1;AES256
64653334323939303461653839353634666162383130326533306234636232656162306661383761
6339376233366638643331373831316638373263663830350a333733653039353939656633376333
31323232666633373633393032396232613830393735396139376333353035633566376465636536
3930633964633861620a633735343066313938633733303538333864353665393062626338356665
63363666636638623964336461346366323565323563316434323439626239633734

This is what it looks like decrypted:

bash-3.2$ ansible-vault view roles/foo/vars/vault.yml
Vault password:
password: hovercraft

Here's what happens when I run the playbook:

bash-3.2$ ansible-playbook -i,localhost playbooks/run_foo.yml --connection=local --ask-vault-pass
Vault password:

PLAY [Run Foo] ******************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************
[WARNING]: error loading fact - please check content
[WARNING]: Platform darwin on host localhost is using the discovered Python interpreter at /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6, but future installation of
another Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information.
ok: [localhost]

TASK [../roles/foo : Show us debug clear_text] **********************************************************************************************************************************************
ok: [localhost] => {
    "clear_text": 12345
}

TASK [../roles/foo : Show us debug password] ************************************************************************************************************************************************
ok: [localhost] => {
    "password": "VARIABLE IS NOT DEFINED!"
}

PLAY RECAP **********************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Is it possible to use a encrypted vault.yml file like my new manager wants?

Answer 1

Put both files into the directory vars/main. All files in vars/main are included automatically. For example,

shell> tree .
.
├── ansible.cfg
├── hosts
├── roles
│   └── foo
│       ├── tasks
│       │   └── main.yml
│       └── vars
│           └── main
│               ├── main.yml
│               └── vault.yml
└── run_foo.yml

shell> cat roles/foo/tasks/main.yml 
- debug:
    var: clear_text
- debug:
    var: password

shell> cat roles/foo/vars/main/main.yml 
clear_text: 12345

shell> cat roles/foo/vars/main/vault.yml 
password: hovercraft

shell> cat run_foo.yml 
- hosts: all
  roles:
    - foo

gives

shell> cat hosts
localhost

shell> ansible-playbook run_foo.yml 

PLAY [all] ***********************************************************************************

TASK [foo : debug] ***************************************************************************
ok: [localhost] => 
  clear_text: 12345

TASK [foo : debug] ***************************************************************************
ok: [localhost] => 
  password: hovercraft

PLAY RECAP ***********************************************************************************
localhost: ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

---

You'll get the same result with encrypted vault.yml. Encrypting the whole file is simpler and safer. Instead of the code, you update and encrypt this single file only. In addition to this, a use case is possible where each user maps its own vault.yml overlay.

Notes

1. Use the role's directory defaults

Don't distribute roles with the directory vars/main or file vars/main.yml. The purpose of the role's vars is to override the role's defaults. See Understanding variable precedence. An update of a role will overwrite potential user's customization in vars if you distribute it. In development, ignore vars/main in your version control.

Put vault.yml into the directory defaults

shell> cat roles/foo/defaults/main/vault.yml 
password: hovercraft

and test the password

shell> cat roles/foo/tasks/main.yml
- assert:
    that: password != 'hovercraft'
    fail_msg: Change and encrypt password!
  when: assert_password|d(true)|bool

- debug:
    var: clear_text
- debug:
    var: password

By default, the user'll see

shell> ansible-playbook run_foo.yml 

PLAY [all] ***********************************************************************************

TASK [foo : assert] **************************************************************************
fatal: [localhost]: FAILED! => changed=false 
  assertion: password != 'hovercraft'
  evaluated_to: false
  msg: Change and encrypt password!

Turn the assert off for testing

shell> ansible-playbook run_foo.yml -e assert_password=false

2. Limit the scope of secrets

You can limit the scope of secrets to the tasks where you need them. For example, put the password to a file

shell> cat vars/password.yml 
my secret password

and read it in the tasks

shell> cat roles/foo/tasks/main.yml 
- assert:
    that: password != 'hovercraft'
    fail_msg: Change and encrypt password!
  when: assert_password|d(true)|bool
  vars:
    password: "{{ lookup('file', password_path) }}"

- debug:
    var: clear_text
- debug:
    var: password
  vars:
    password: "{{ lookup('file', password_path) }}"

Provide the path to the file, for example, in the playbook

shell> cat run_foo.yml 
- hosts: all
  vars:
    password_path: "{{ playbook_dir }}/vars/password.yml"
  roles:
    - foo

gives

shell> ansible-playbook run_foo.yml

PLAY [all] ***********************************************************************************

TASK [foo : assert] **************************************************************************
ok: [localhost] => changed=false 
  msg: All assertions passed

TASK [foo : debug] ***************************************************************************
ok: [localhost] => 
  clear_text: 12345

TASK [foo : debug] ***************************************************************************
ok: [localhost] => 
  password: my secret password

PLAY RECAP ***********************************************************************************
localhost: ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The encryption of the files is transparent to the file lookup. This also solves your other problem to "use grep to find things in the code"

shell> grep -r password: roles/
roles/foo/defaults/main/vault.yml:password: hovercraft
roles/foo/tasks/main.yml:    password: "{{ lookup('file', password_path) }}"
roles/foo/tasks/main.yml:    password: "{{ lookup('file', password_path) }}"

Answer 2

It looks like you're trying to use include_vars in a vars file. That doesn't work; include_vars is an Ansible module and will only work a task list (i.e., a playbook, role, or similar). So you can write roles/foo/tasks/main.yaml like this:

- name: load encrypted vars
  include_vars:
    file: vault.yaml

- name: show us debug clear_text
  debug:
    var: clear_text

- name: show us debug password
  debug:
    var: password

Assuming that you have roles/foo/vars/main.yaml that looks like this:

clear_text: this is clear text

And roles/foo/vars/vault.yaml that you created like this:

echo 'password: hovercraft' |
  ansible-vault encrypt > roles/foo/vars/vault.yaml

We can write a playbook like this:

- hosts: localhost
  gather_facts: false
  roles:
    - foo

And run it like this:

$ ansible-playbook playbook.yaml --ask-vault-password

And get the following output:

PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [foo : load encrypted vars] **********************************************************************************************************************************************************************************
ok: [localhost]

TASK [foo : show us debug clear_text] *****************************************************************************************************************************************************************************
ok: [localhost] => {
    "clear_text": "this is clear text"
}

TASK [foo : show us debug password] *******************************************************************************************************************************************************************************
ok: [localhost] => {
    "password": "hovercraft"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

---

You can find a complete example in this repository.

---

If you want the encrypted file to load automatically without requiring an explicit include_vars, then move it out of the role and into group_vars or host_vars.