Ansible role to filter host based on host variables

Question

I am trying to create a ansible role which filter out host based on dictionary(passed to role) in comparison with host variables. If host variables contains key:value from dictionary it will added to filtered_host list. This is what I found till point but not getting desired output:

Ansible role:

---

- name: Filtering host
  debug:
    msg: "Here is the response {{ filters }} {{groups['all']}}"

- name: Setting filter
  set_fact:
    input_filter: "{{ filters }}"
  
- name: Setting filtered hosts
  set_fact:
    filtered_hosts: "{{ groups['all'] | map('extract', hostvars) | select('@ in filters') | map(attribute='inventory_hostname') | list }}"
  vars:
    input_filter: "{{ filters | to_json }}"

- name: Print filtered hosts
  debug:
    var: filtered_hosts

Here filters is a dictionary(filter) passed from playbook to test but actually i want to pass a dictionary, and this role should return list of hosts which contains key:value from this dictionary.

Sample dictionary:

{
   "os_type":"linux",
   "datacenter" : "REM",
   "location" : "IND"
}

Sample Host variables:

HOST A variables:

{
   "os_type":"linux",
   "datacenter" : "REM",
   "location" : "IND"
}

HOST B variables:

{
   "os_type":"linux",
   "datacenter" : "REM",
   "location" : "USA",
   "status" : "success"
}

HOST C variables:

{
   "os_type":"linux",
   "datacenter" : "REM",
   "location" : "IND",
   "status" : "success"
}

OUTPUT:

[Host A, Host C]

because both host contains key:value passed through dictionary. Also number of keys differ from different hosts.

Actual Scenario: I created a dynamic inventory which will fetch all host and host vars from DB based on some filters and assign it to inventory. Groups are created using some general info based on host vars. Now templates are created for different organizations/product who require to filter host based on some conditions(dynamically) out of the host vars. Since we cant create groups for everything, we decided to have a common ansible role which will act as a filter.

Within playbook there are multiple task will be performed on different host and to get those specific host some conditions are generated in the form of dictionary. These dictionary will be passed to ansible role to filter further based on the dynamic condition. The filter in given scenario is comparing 2 dictionaries to find out proper host for that task. A simplified example is what I have shown in the question.

Answer 1

Given the dictionary

  filters:
    os_type: linux
    datacenter: REM
    location: IND

In the vars, select the keys

  fkeys: "{{ filters.keys() }}"

Select and join the values

  fvals: "{{ filters.values()|join(',') }}"

gives

  fvals: linux,REM,IND

In the tasks, create the variable my_fvals

    - set_fact:
        my_fvals: "{{ lookup('vars', *fkeys, default='UNDEF') }}"
    - debug:
        var: my_fvals

gives (abridged)

ok: [Host_A] => 
  my_fvals: linux,REM,IND
ok: [Host_B] => 
  my_fvals: linux,REM,USA
ok: [Host_C] => 
  my_fvals: linux,REM,IND

Select hosts that match the criteria

  filtered_hosts: "{{ hostvars|dict2items|
                      selectattr('value.my_fvals', '==', fvals)|
                      map(attribute='key') }}"

gives

  filtered_hosts: [Host_A, Host_C]

---

• Example of a complete playbook for testing

- hosts: all

  vars:

    filters:
      os_type: linux
      datacenter: REM
      location: IND

    fkeys: "{{ filters.keys() }}"
    fvals: "{{ filters.values()|join(',') }}"
    filtered_hosts: "{{ hostvars|dict2items|
                        selectattr('value.my_fvals', '==', fvals)|
                        map(attribute='key') }}"

  tasks:

    - debug:
        var: fvals
      run_once: true

    - set_fact:
        my_fvals: "{{ lookup('vars', *fkeys, default='UNDEF') }}"
    - debug:
        var: my_fvals

    - debug:
        var: filtered_hosts|to_yaml
      run_once: true

• You can create a role if you want to

shell> tree roles
roles/
└── filtered_hosts
    ├── defaults
    │   └── main.yml
    └── tasks
        └── main.yml

3 directories, 2 files

shell> cat roles/filtered_hosts/defaults/main.yml 
fkeys: "{{ filters.keys() }}"
fvals: "{{ filters.values()|join(',') }}"

shell> cat roles/filtered_hosts/tasks/main.yml 
- set_fact:
    my_fvals: "{{ lookup('vars', *fkeys, default='UNDEF') }}"
- set_fact:
    filtered_hosts: "{{ hostvars|dict2items|
                        selectattr('value.my_fvals', '==', fvals)|
                        map(attribute='key') }}"
  run_once: true

The playbook

shell> cat pb.yml
- hosts: all

  vars:

    filters:
      os_type: linux
      datacenter: REM
      location: IND

  roles:
    - filtered_hosts

  tasks:

    - debug:
        var: filtered_hosts|to_yaml
      run_once: true

gives

shell> ansible-playbook pb.yml

PLAY [all] ***********************************************************************************

TASK [filtered_hosts : set_fact] *************************************************************
ok: [Host_A]
ok: [Host_C]
ok: [Host_B]

TASK [filtered_hosts : set_fact] *************************************************************
ok: [Host_A]

TASK [debug] *********************************************************************************
ok: [Host_A] => 
  filtered_hosts|to_yaml: |-
    [Host_A, Host_C]

PLAY RECAP ***********************************************************************************
Host_A: ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
Host_B: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
Host_C: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

• It is not possible to use the inventory plugin ansible.builtin.constructed here

shell> tree inventory
inventory/
├── 01-hosts
└── 02-constructed.yml

0 directories, 2 files

shell> cat inventory/01-hosts 
Host_A
Host_B
Host_C

shell> cat inventory/02-constructed.yml 
plugin: ansible.builtin.constructed
use_extra_vars: true
use_vars_plugins: true
strict: true
compose:
  fkeys: filters.keys()
  fvals: filters.values()|join(',')
  my_fvals: lookup('vars', *filters.keys(), default='UNDEF')
groups:
  filtered_hosts: my_fvals == fvals

because in this inventory plugin lookups were disabled from templating

shell> ansible-inventory -i inventory -e @filters.yml --list --yaml

[WARNING]: * Failed to parse /export/scratch/tmp7/test-477/inventory/02-constructed.yml with auto plugin: failed to parse /export/scratch/tmp7/test-477/inventory/02-constructed.yml: Could not set my_fvals for host Host_A: The lookup vars was found, however lookups were disabled from templating . Could not set my_fvals for host Host_A: The lookup vars was found, however lookups were disabled from templating

[WARNING]: * Failed to parse /export/scratch/tmp7/test-477/inventory/02-constructed.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory

[WARNING]: * Failed to parse /export/scratch/tmp7/test-477/inventory/02-constructed.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.

[WARNING]: Unable to parse /export/scratch/tmp7/test-477/inventory/02-constructed.yml as an inventory source

all:
  children:
    ungrouped:
      hosts:
        Host_A:
          datacenter: REM
          filters: &id001
            datacenter: REM
            location: IND
            os_type: linux
          fkeys:
          - os_type
          - datacenter
          - location
          fvals: linux,REM,IND
          location: IND
          os_type: linux
        Host_B:
          datacenter: REM
          filters: *id001
          location: USA
          os_type: linux
          status: success
        Host_C:
          datacenter: REM
          filters: *id001
          location: IND
          os_type: linux
          status: success

Answer 2

This is your current inventory (as I understand it) "dumped" into a static inventory. This can be replaced with your current dynamic inventory. I placed this in inventories/default/0-hosts.yml

---
all:
  hosts:
    host_a: {
      "os_type": "linux",
      "datacenter": "REM",
      "location": "IND"
    }
    host_b: {
      "os_type": "linux",
      "datacenter": "REM",
      "location": "USA",
      "status": "success"
    }
    host_c: {
      "os_type": "linux",
      "datacenter": "REM",
      "location": "IND",
      "status": "success"
    }

Lets add a new ansible.builtin.constructed source where we create dynamically all the necessary groups from your variables. This goes in inventories/default/1-constructed.yml

---
plugin: ansible.builtin.constructed
strict: false
keyed_groups:
  - prefix: os_type
    key: os_type
  - prefix: datacenter
    key: datacenter
  - prefix: location
    key: location

We can already explore the result from theansible-inventory -i inventories/default/ --list command:

{
    "_meta": {
        "hostvars": {
            "host_a": {
                "datacenter": "REM",
                "location": "IND",
                "os_type": "linux"
            },
            "host_b": {
                "datacenter": "REM",
                "location": "USA",
                "os_type": "linux",
                "status": "success"
            },
            "host_c": {
                "datacenter": "REM",
                "location": "IND",
                "os_type": "linux",
                "status": "success"
            }
        }
    },
    "all": {
        "children": [
            "ungrouped",
            "os_type_linux",
            "datacenter_REM",
            "location_IND",
            "location_USA"
        ]
    },
    "datacenter_REM": {
        "hosts": [
            "host_a",
            "host_b",
            "host_c"
        ]
    },
    "location_IND": {
        "hosts": [
            "host_a",
            "host_c"
        ]
    },
    "location_USA": {
        "hosts": [
            "host_b"
        ]
    },
    "os_type_linux": {
        "hosts": [
            "host_a",
            "host_b",
            "host_c"
        ]
    }
}

From there it is quite easy to write a playbook which will take advantage of this multi-source inventory taking into account your user variables. As a pure example, this is what I tried in playbook.yml. I passed your user filters as extra vars for the example but they can come from any source.

---
- name: Process user vars
  hosts: localhost
  gather_facts: false

  tasks:
    - name: Create a pattern from filter (defaulting to all if filter is empty)
      vars:
        filter_data: "{{ user_filter | d({}) }}"
      ansible.builtin.set_fact:
        filter_pattern: "{{
          filter_data.keys() | zip(filter_data.values())
          | map('join', '_') | join(':&') | d('all', true)
        }}"
    
    - name: Show the constructed pattern
      ansible.builtin.debug:
        var: hostvars.localhost.filter_pattern

    - name: We can now use that pattern in a lookup in any play to get a list
      ansible.builtin.debug:
        var: "q(
          'ansible.builtin.inventory_hostnames',
          hostvars.localhost.filter_pattern
        )"

- name: We can also use the pattern to target hosts in an other play
  hosts: "{{ hostvars.localhost.filter_pattern }}"
  gather_facts: false

  # Note that you could play a role here

  tasks:
    - name: Show inventory hostname we would run on
      ansible.builtin.debug:
        var: inventory_hostname

Demo runs:

• With your original filter

$ ansible-playbook -i inventories/default/ -e '{"user_filter":{"os_type":"linux", "datacenter":"REM", "location":"IND"}}' playbook.yml 

PLAY [Process user vars] ******************************************************************************************************************************************************************************************

TASK [Create a pattern from filter (defaulting to all if filter is empty)] ****************************************************************************************************************************************
ok: [localhost]

TASK [Show the constructed pattern] *******************************************************************************************************************************************************************************
ok: [localhost] => {
    "hostvars.localhost.filter_pattern": "os_type_linux:&datacenter_REM:&location_IND"
}

TASK [We can now use that pattern in a lookup in any play to get a list] ******************************************************************************************************************************************
ok: [localhost] => {
    "q( 'ansible.builtin.inventory_hostnames', hostvars.localhost.filter_pattern )": [
        "host_a",
        "host_c"
    ]
}

PLAY [We can also use the pattern to target hosts in an other play] ***********************************************************************************************************************************************

TASK [Show inventory hostname we would run on] ********************************************************************************************************************************************************************
ok: [host_a] => {
    "inventory_hostname": "host_a"
}
ok: [host_c] => {
    "inventory_hostname": "host_c"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
host_a                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_c                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

• Change location

$ ansible-playbook -i inventories/default/ -e '{"user_filter":{"os_type":"linux", "datacenter":"REM", "location":"USA"}}' playbook.yml 

PLAY [Process user vars] ******************************************************************************************************************************************************************************************

TASK [Create a pattern from filter (defaulting to all if filter is empty)] ****************************************************************************************************************************************
ok: [localhost]

TASK [Show the constructed pattern] *******************************************************************************************************************************************************************************
ok: [localhost] => {
    "hostvars.localhost.filter_pattern": "os_type_linux:&datacenter_REM:&location_USA"
}

TASK [We can now use that pattern in a lookup in any play to get a list] ******************************************************************************************************************************************
ok: [localhost] => {
    "q( 'ansible.builtin.inventory_hostnames', hostvars.localhost.filter_pattern )": [
        "host_b"
    ]
}

PLAY [We can also use the pattern to target hosts in an other play] ***********************************************************************************************************************************************

TASK [Show inventory hostname we would run on] ********************************************************************************************************************************************************************
ok: [host_b] => {
    "inventory_hostname": "host_b"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
host_b                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

• Defaults to all hosts if no filter is provided

$ ansible-playbook -i inventories/default/ playbook.yml 

PLAY [Process user vars] ******************************************************************************************************************************************************************************************

TASK [Create a pattern from filter (defaulting to all if filter is empty)] ****************************************************************************************************************************************
ok: [localhost]

TASK [Show the constructed pattern] *******************************************************************************************************************************************************************************
ok: [localhost] => {
    "hostvars.localhost.filter_pattern": "all"
}

TASK [We can now use that pattern in a lookup in any play to get a list] ******************************************************************************************************************************************
ok: [localhost] => {
    "q( 'ansible.builtin.inventory_hostnames', hostvars.localhost.filter_pattern )": [
        "host_a",
        "host_b",
        "host_c"
    ]
}

PLAY [We can also use the pattern to target hosts in an other play] ***********************************************************************************************************************************************

TASK [Show inventory hostname we would run on] ********************************************************************************************************************************************************************
ok: [host_a] => {
    "inventory_hostname": "host_a"
}
ok: [host_b] => {
    "inventory_hostname": "host_b"
}
ok: [host_c] => {
    "inventory_hostname": "host_c"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
host_a                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_b                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_c                     : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0