Ansible - parse a list of nested dictionaries to create a new list

Question

I have the following list of dictionaries

"tor_vlan_list": [
    {
        "switch_01": {
            "Vlan1": {
                "id": "1",
                "ip": "unassigned",
                "ok": "NO",
                "protocol": "down",
                "status": "up"
            },
            "Vlan10": {
                "id": "10",
                "ip": "10.10.10.2/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            },
            "Vlan20": {
                "id": "20",
                "ip": "10.10.20.2/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            },
            "Vlan30": {
                "id": "30",
                "ip": "10.10.30.2/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            }
        }
    },
    {
        "switch_02": {
            "Vlan1": {
                "id": "1",
                "ip": "unassigned",
                "ok": "NO",
                "protocol": "down",
                "status": "up"
            },
            "Vlan10": {
                "id": "10",
                "ip": "10.10.10.3/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            },
            "Vlan20": {
                "id": "20",
                "ip": "10.10.20.3/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            },
            "Vlan30": {
                "id": "30",
                "ip": "10.10.30.3/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            }
        }
    }
]

from this, i need to extract the list of vlan IDs for each switch and store these as follows

"new_list": [
    { 
     "switch_01": [1,10,20,30]
    },
    {
     "switch_02": [1,10,20,30]
    }
]

I would like to pass the list values as a var in the command *show vlan {{item}}"

I've tried getting the needed detail using combinations of selectattr, map, combine, json_query, etc to set a new ansible fact but, all have eluded me.

Answer 1

The below expressions

  new_list_keys: "{{ tor_vlan_list|json_query('[].keys(@)') }}"
  new_list_vals: "{{ tor_vlan_list|json_query('[].*.*.id') }}"
  new_list: |
    [{% for i in new_list_keys|zip(new_list_vals) %}
    {{ dict(i.0|zip(i.1|map('map', 'int'))) }},
    {% endfor %}]

give what you want

  new_list:
    - switch_01: [1, 10, 20, 30]
    - switch_02: [1, 10, 20, 30]

  new_list_keys:
    - [switch_01]
    - [switch_02]

  new_list_vals:
    - - ['1', '10', '20', '30']
    - - ['1', '10', '20', '30']

---

Example of a complete playbook for testing

- hosts: localhost

  vars:

    tor_vlan_list:
    - switch_01:
        Vlan1: {id: '1', ip: unassigned, ok: 'NO', protocol: down, status: up}
        Vlan10: {id: '10', ip: 10.10.10.2/24, ok: 'YES', protocol: up, status: up}
        Vlan20: {id: '20', ip: 10.10.20.2/24, ok: 'YES', protocol: up, status: up}
        Vlan30: {id: '30', ip: 10.10.30.2/24, ok: 'YES', protocol: up, status: up}
    - switch_02:
        Vlan1: {id: '1', ip: unassigned, ok: 'NO', protocol: down, status: up}
        Vlan10: {id: '10', ip: 10.10.10.3/24, ok: 'YES', protocol: up, status: up}
        Vlan20: {id: '20', ip: 10.10.20.3/24, ok: 'YES', protocol: up, status: up}
        Vlan30: {id: '30', ip: 10.10.30.3/24, ok: 'YES', protocol: up, status: up}

    new_list_keys: "{{ tor_vlan_list|json_query('[].keys(@)') }}"
    new_list_vals: "{{ tor_vlan_list|json_query('[].*.*.id') }}"
    new_list: |
      [{% for i in new_list_keys|zip(new_list_vals) %}
      {{ dict(i.0|zip(i.1|map('map', 'int'))) }},
      {% endfor %}]

  tasks:

    - debug:
        var: new_list_keys|to_yaml
    - debug:
        var: new_list_vals|to_yaml
    - debug:
        var: new_list|type_debug
    - debug:
        var: new_list|to_yaml

Answer 2

Ansible isn't great at manipulating data structures, but you could do something like this:

- set_fact:
    new_list: >
      {{
        new_list + [
          {
            switch[0]: switch[1].values()|list|map(attribute='id')|map('int')
          }
        ]
      }}
  vars:
    new_list: []
    switch: "{{ item.items()|first|list }}"
  loop: "{{ tor_vlan_list }}"
  loop_control:
    label: "{{ switch[0] }}"

- debug:
    var: new_list

This makes somewhat complicated use of set_fact in a loop. Given your example input, the debug task prints your desired data structure:

ok: [localhost] => {
    "new_list": [
        {
            "switch_01": [
                1,
                10,
                20,
                30
            ]
        },
        {
            "switch_02": [
                1,
                10,
                20,
                30
            ]
        }
    ]
}

Alternately, you could place your logic into a custom filter in filter_plugins/switches.py, like this:

def transform_vlan_list(vlan_list):
    return [
        {switch[0]: [int(vlan["id"]) for vlan in switch[1].values()]}
        for x in vlan_list
        for switch in x.items()
    ]


class FilterModule:
    def filters(self):
        return {"transform_vlan_list": transform_vlan_list}

The advantage here is that you've moved your logic into Python, which is much better at manipulating data.

With this custom filter plugin, you can produce the same output as the first example like this:

- set_fact:
    new_list: "{{ tor_vlan_list | transform_vlan_list }}"

- debug:
    var: new_list