Overriding list entry in Hydra, from another YAML

Question

In Hydra config YAML files, I have the following structure in the file inner.yaml:

key_a:
    - entry_a_1: xxxx
      entry_a_2: xxxxx
    - entry_a_3: xxxx
      entry_a_4: xxxxx

What I want is to be able to override the entry_a_M from another YAML file, for example at the file outer.yaml:

defaults:
    - inner_config@outer_inner_config: inner
outter_inner_config:
    entry_a_1: YYYY

I have tried with key_a.0.entry_a_1: WWWW and other combinations but it doesn't work.

Please note:

• I don't want to override it from the CLI
• If there are keys in each list item, e.g. - key: [entry_a_1...] then it can be done, as shown in question here. But it is not my case and it would not work with having that key in the list entry, in my case.

Any answers on that?

Answer 1

This is not supported directly. Lists are replaced by OmegaConf.merge(), which is what Hydra is using to merge the config objects.

My recommendation is to convert your list to a dictionary. This will allow you to override it by the key. If your code needs to access the values as a list, you can use the built in OmegaConf resolver oc.dict.values.

From the docs:

>>> cfg = OmegaConf.create(
...     {
...         "workers": {
...             "node3": "10.0.0.2",
...             "node7": "10.0.0.9",
...         },
...         "ips": "${oc.dict.values: workers}",
...     }
... )
>>> # Values are dynamically fetched through interpolations:
>>> show(cfg.ips)
type: ListConfig, value: ['${workers.node3}', '${workers.node7}']
>>> assert cfg.ips == ["10.0.0.2", "10.0.0.9"]

This can be utilized with instantiate by placing the source dictionary outside of the object being instantiated, e.g:

objects:
  obj1:
    _target_: __main__.Foo
  obj2:
    _target_: __main__.Foo

main:
  _target_: __main__.Main
  objects: "${oc.dict.values: ..objects}"

import hydra
from hydra.utils import instantiate
from omegaconf import OmegaConf
from typing import List


class Foo:
    def __init__(self):
        pass


class Main:
    objects: List[Foo];

    def __init__(self, objects):
        self.objects = objects

    def __str__(self):
        return f"Main with {len(self.objects)} objects"



@hydra.main(version_base=None, config_path="conf", config_name="cfg")
def main(cfg):
    print(OmegaConf.to_yaml(cfg, resolve=True))
    obj = instantiate(cfg.main)
    print(obj)


if __name__ == '__main__':
    main()

To complete the solution, you can use the pattern described here to select multiple configs from the same config group. Refactoring the objects in the config above into two separate config files and using the defaults list:

objects/obj1.yaml:

obj1:
  _target_: __main__.Foo

objects/obj2.yaml:

obj2:
  _target_: __main__.Foo

config.yaml:

defaults:
  - _self_
  - objects:
      - obj1
      - obj2
main:
  _target_: __main__.Main
  objects: "${oc.dict.values: ..objects}"