How to merge elasticsearch indices

Question

In Elasticsearch v7.12.0 I have several indices named as payload.YYYY-MM. With YYYY = year and MM = month.

In order to reduce the amount of indices, I would like to merge all the indices that are from the same year in a new indice named payload.YYYY

How can I do it using the Elasticsearch API ?

(I don't want to use the aliases as it doesn't reduce the amount of indices)

Answer 1

You can do this very simply using the _reindex API and running the following command for each year you want to support:

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "payload.2022-*"
  },
  "dest": {
    "index": "payload.2022"
  }
}

All indexes named payload.2022-* will be reindexed into the yearly index payload.2022. Make sure to create the yearly index with the same mapping as the monthly indexes prior to running the command.

Note: adding wait_for_completion=false makes the command create a task in the background and return immediately with a task ID. You can then use that task ID to monitor the process in the background using

GET _tasks/<task_id>