How can I install community.general collection in Ansible, using Ansible? With task run only when it is missing?

Question

Right now I have

---
- name: Install community.general collection
  command: ansible-galaxy collection install community.general
  changed_when: True

which works, but will report that something was changed on every run (changed_when: False has opposite problem) and I suspect that error handling will be poor.

Answer 1

When trying to install an existing collection from command line I get:

$ ansible-galaxy collection install community.general
Starting galaxy collection install process
Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.
$ echo $?
0

Based on this output and on the fact this does not raise an error, the following should meet your requirement as a first quick and dirty solution. You can enhance it for your specific needs:

- name: Install community.general collection
  command: ansible-galaxy collection install community.general
  register: install_general
  changed_when: '"Nothing to do." not in install_general.stdout'

Answer 2

Q: "How can I install community.general collection ... only when it is missing?"

A: What collections are available depends on the configuration COLLECTIONS_PATHS. By default,

shell> ansible-config dump | grep coll
COLLECTIONS_PATHS(default) = ['/home/admin/.ansible/collections', '/usr/share/ansible/collections']

You can configure paths to fit your needs. For example,

shell> cat ansible.cfg
[defaults]
collections_paths = /home/admin/.ansible/ansible_collections:/usr/lib/python3/dist-packages/ansible_collections

In this case, you'll see something similar to

shell> ansible-config dump | grep coll
COLLECTIONS_PATHS(/export/scratch/tmp1/test-02/ansible.cfg) = ['/home/admin/.ansible/ansible_collections', '/usr/lib/python3/dist-packages/ansible_collections']

In a play, you can use the lookup config and get the paths

  collection_paths: "{{ lookup('config', 'COLLECTIONS_PATHS') }}"

gives

  collection_paths:
  - /home/admin/.ansible/ansible_collections
  - /usr/lib/python3/dist-packages/ansible_collections

Find all directories

    - find:
        paths: "{{ collection_paths }}"
        file_type: directory
        use_regex: true
        excludes: '^_.*'
      register: out1

and declare the list

  collection_dirs: "{{ out1.files|map(attribute='path') }}"

gives

  collection_dirs:
  - /usr/lib/python3/dist-packages/ansible_collections/mellanox
  - /usr/lib/python3/dist-packages/ansible_collections/openstack
  - /usr/lib/python3/dist-packages/ansible_collections/microsoft
  - /usr/lib/python3/dist-packages/ansible_collections/awx
    ...

Get the subdirectories

    - find:
        paths: "{{ item }}"
        file_type: directory
        use_regex: true
        excludes: '^.*\.info'
      loop: "{{ collection_dirs }}"
      register: out2

and declare the list of all collections

  collection_all: "{{ out2.results|json_query('[].files[].path') }}"

gives

  collection_all:
  - /usr/lib/python3/dist-packages/ansible_collections/mellanox/onyx
  - /usr/lib/python3/dist-packages/ansible_collections/openstack/cloud
  - /usr/lib/python3/dist-packages/ansible_collections/microsoft/ad
  - /usr/lib/python3/dist-packages/ansible_collections/awx/awx
    ...

Group the collections and create a dictionary

  collection_groups: "{{ collection_all|map('dirname')|map('basename')|
                         zip(collection_all|map('basename'))|groupby('0') }}"
  colls: |
    {% filter from_yaml %}
    {% for i in collection_groups %}
    {{ i.0 }}: {{ i.1|map('last') }}
    {% endfor %}
    {% endfilter %}

gives

  colls:
    amazon: [aws]
    ansible: [windows, posix, utils, netcommon]
    arista: [eos]
    awx: [awx]
    azure: [azcollection]
    check_point: [mgmt]
    chocolatey: [chocolatey]
    cisco: [ucs, intersight, meraki, ios, mso, iosxr, dnac, ise, nxos, asa, nso, aci]
    cloud: [common]
    cloudscale_ch: [cloud]
    community: [windows, hashi_vault, sap_libs, general, sap, ciscosmb, digitalocean,
      hrobot, aws, network, zabbix, okd, sops, google, fortios, mongodb, libvirt, azure,
      crypto, grafana, dns, postgresql, skydive, docker, mysql, proxysql, vmware, routeros,
      rabbitmq]
    containers: [podman]
    cyberark: [pas, conjur]
    dellemc: [os10, unity, openmanage, os9, os6, enterprise_sonic, powerflex]
    f5networks: [f5_modules]
    fortinet: [fortimanager, fortios]
    frr: [frr]
    gluster: [gluster]
    google: [cloud]
    grafana: [grafana]
    hetzner: [hcloud]
    hpe: [nimble]
    ibm: [spectrum_virtualize, qradar]
    infinidat: [infinibox]
    infoblox: [nios_modules]
    inspur: [ispim, sm]
    junipernetworks: [junos]
    kubernetes: [core]
    lowlydba: [sqlserver]
    mellanox: [onyx]
    microsoft: [ad]
    netapp: [storagegrid, ontap, aws, cloudmanager, elementsw, um_info, azure]
    netapp_eseries: [santricity]
    netbox: [netbox]
    ngine_io: [cloudstack, exoscale, vultr]
    openstack: [cloud]
    openvswitch: [openvswitch]
    ovirt: [ovirt]
    purestorage: [flashblade, fusion, flasharray]
    sensu: [sensu_go]
    splunk: [es]
    t_systems_mms: [icinga_director]
    theforeman: [foreman]
    vmware: [vmware_rest]
    vultr: [cloud]
    vyos: [vyos]
    wti: [remote]

Then, the testing is trivial. For example,

    - debug:
        msg: "{{ col }} is installed."
      when: arr.1 in colls[arr.0]
      vars:
        col: community.general
        arr: "{{ col|split('.') }}"

---

Optionally, create a simple list

  colls: "{{ collection_all|map('dirname')|map('basename')|
             zip(collection_all|map('basename'))|
             map('join', '.')|sort }}"

gives

  colls:
  - amazon.aws
  - ansible.netcommon
  - ansible.posix
  - ansible.utils
  - ansible.windows
  - arista.eos
    ...

and use it in the condition

    - debug:
        msg: "{{ collection }} is installed."
      when: collection in colls
      vars:
        collection: community.general

---

If missing, install the collection

    - community.general.ansible_galaxy_install:
        type: collection
        name: "{{ col }}"
      when: arr.1 not in colls[arr.0]
      vars:
        col: community.general
        arr: "{{ col|split('.') }}"

---

Example of a complete playbook for testing

- hosts: localhost

  vars:

    collection_paths: "{{ lookup('config', 'COLLECTIONS_PATHS') }}"
    collection_dirs: "{{ out1.files|map(attribute='path') }}"
    collection_all: "{{ out2.results|json_query('[].files[].path') }}"
    collection_groups: "{{ collection_all|map('dirname')|map('basename')|
                           zip(collection_all|map('basename'))|groupby('0') }}"
    colls: |
      {% filter from_yaml %}
      {% for i in collection_groups %}
      {{ i.0 }}: {{ i.1|map('last') }}
      {% endfor %}
      {% endfilter %}

  tasks:

    - debug:
        var: collection_paths

    - find:
        paths: "{{ collection_paths }}"
        file_type: directory
        use_regex: true
        excludes: '^_.*'
      register: out1

    - debug:
        var: collection_dirs

    - find:
        paths: "{{ item }}"
        file_type: directory
        use_regex: true
        excludes: '^.*\.info'
      loop: "{{ collection_dirs }}"
      register: out2

    - debug:
        var: collection_all
    - debug:
        var: collection_groups
    - debug:
        var: colls|to_yaml

    - debug:
        msg: "{{ col }} is installed."
      when: arr.1 in colls[arr.0]
      vars:
        col: community.general
        arr: "{{ col|split('.') }}"

---

Note: If you unconditionally install the collections

    - community.general.ansible_galaxy_install:
        type: collection
        name: "{{ item.0.key }}.{{ item.1 }}"
      loop: "{{ colls|dict2items|subelements('value') }}"
      register: out

the latest versions will be installed in the first path found. The information about already installed collections '*.info' will be stored in this path too. For example, the collection ansible.netcommon version 6.0.0 will be installed in /home/admin/.ansible/ansible_collections in addition to already installed ansible.netcommon version 4.1.0 in /usr/lib/python3/dist-packages/ansible_collections

shell> ls -1 /home/admin/.ansible/ansible_collections/ | sort
amazon
amazon.aws-7.2.0.info
ansible
ansible.netcommon-4.1.0.info
ansible.netcommon-6.0.0.info
ansible.posix-1.5.4.info
ansible.utils-3.0.0.info
ansible.windows-2.2.0.info
arista
arista.eos-7.0.0.info
...

Declare the list of new collections installed

  result: "{{ out.results|json_query('[].new_collections')|select }}"

gives

  result:
  - amazon.aws: 7.2.0
  - ansible.windows: 2.2.0
  - ansible.posix: 1.5.4
  - ansible.utils: 3.0.0
  - ansible.netcommon: 6.0.0
  - arista.eos: 7.0.0
  - awx.awx: 23.6.0
    ...