How to ignore error in celery chord and allows the process to continue

445 views Asked by At

I have a celery workflow with multiple tasks, chain and chords. Some of theses tasks need to be executed sequentially, some others can be executed in parallel.

The problem is when an error is raised in a chord, all the chain is interrupted. I need to continue the process even when there is an error.

This is a minimal example to reproduce my use case

from celery import chain, chord

from celery_app import celery_app


@celery_app.task(name="TASK_1")
def task_1():
    print("task_1")


@celery_app.task(name="TASK_2")
def task_2():
    print("task_2")


@celery_app.task(name="PARALLEL_TASK")
def parallel_task():
    # HEAVY HTTP Call
    print("parallel task")


@celery_app.task(name="ERROR_PARALLEL_TASK")
def error_parallel_task():
    print("task error")
    raise ValueError()


@celery_app.task(name="TASK_3")
def task_3():
    print("task_3")


@celery_app.task(name="TASK_4")
def task_4():
    print("task_4")


chain(
    task_1.si(),
    task_2.si(),
    chord(
        [
            parallel_task.si(),
            parallel_task.si(),
            error_parallel_task.si(),
            error_parallel_task.si(),
            parallel_task.si(),
            parallel_task.si()
        ],
        task_3.si(),
    ),
    task_4.si(),
).delay()

This example produce a celery.exceptions.ChordError: Dependency 10719ad2-cd61-4596-bb9b-92d42d1ea967 raised ValueError()

I need the process to continue executing task_3 and task_4 even if the group face an error.

What i tried so far:

  • Various celery config variables to ignore errors
ignore_result=True
throws=(ValueError,)
acks_on_failure_or_timeout=True

  • Using .on_error() to trigger the next task whatever append
chain(
    group(
        parallel_task.si(),
        error_parallel_task.si(),
    ).on_error(task_3.si()),
    task_3.si()
)

This one works but the task_3 is called multiple times and i don't want that.

  • Make a task that never fail. With try / except I can make my task to never fail. It works but I don't like it. The task will appeared as a success when it is actually failed. And i can't use celery retry system easily this way.

Is there a way to simply ignore error in a chord / chain to allow the process to continue ? Is celery the right tool to achieve what I'm trying to do ?

0

There are 0 answers