dask-distributed: how to cancel tasks submitted with fire_and_forget?

Question

When submitting task graphs using fire_and_forget, is it possible to later (in a new process/client) cancel those tasks (e.g. by key name)? And will that also cancel all dependent tasks, or are these also 'fire_and_forget'-like?

Answer 1

Yes, you can create a new future with the keyname

from dask.distributed import Future

future = Future(key_name, my_client)
future.cancel()

Forcing cancellation even in the face of multiple clients seems reasonable (fire-and-forget is considered its own client). Implemented here: https://github.com/dask/distributed/pull/1408 . In version > 1.18.3 you will be able to use the force=True keyword

future.cancel(force=True)

This will cancel the future, even if other clients desire it.