I'm attempting to create a long-running (minutes) Celery task that'll be durable enough to withstand power loss, application crashes, etc. I have read the docs and know why this might be a bad idea but, in my case re-runs are acceptable and I'm more concerned that the action runs to completion at least once.

Here's my task:

@celery.task(
    acks_late=True,
    bind=True,
    reject_on_worker_lost=True,
    retry_kwargs={"max_retries": 5},
)
def do_thing(self, demo_id=None):    
    wrapped_thing()

Celery config:

celery = Celery(
    __name__,
    backend=redis_backend_url,
    broker=redis_broker_url,
    include=["application.background_jobs", "application.scheduled_tasks"],
    redbeat_redis_url=redis_redbeat_url,
)

I've tried every variation of this that I can think of (setting acks_late and reject_on_worker_lost globally, etc.) and, no matter what 1.) the task is immediately marked as "started" in Flower and 2.) not re-run if I bring my app down and then back up again while the operation is in-progress. The task is also not present in the unacknowledged list of tasks if I poke into the application using app.control.inspect().reserved().

Environment:

• Celery 4.4.7
• Python 3.7
• Redis 6.2