Skip to content

Worker child is not restarted after --max-tasks-per-child is reached #646

Description

@aysanorhun

Taskiq version

0.12.4

Python version

Python 3.12

OS

Linux

What happened?

Description

With Taskiq 0.12.4 and taskiq-redis 1.2.3, when --max-tasks-per-child is reached, the worker stops prefetching, waits for running tasks, and starts broker shutdown:

Max number of tasks executed.
Stopping prefetching messages...
No more tasks to wait for. Shutting down.
Shutting down the broker.

However, the child process remains alive. Since ProcessManager only restarts a worker after detecting that it is dead, this worker slot is never respawned and stops consuming tasks.

The only workaround I found was forcing the child to exit using os._exit(...), after which ProcessManager correctly respawned that specific worker.

Expected behavior

The child should exit after completing its admitted tasks, allowing ProcessManager to respawn it.

The documentation also describes this as a restart:

--max-tasks-per-child - maximum number of tasks to be executed by a single worker process before restart.

https://taskiq-python.github.io/guide/cli.html#other-parameters

The option was originally requested in #309 and implemented by #314. Issue #556 also shows the expected worker is dead → scheduling reload → restarted sequence.

Is the intended behavior for the child to exit and be respawned after reaching --max-tasks-per-child? If so, should the worker guarantee termination when graceful receiver or broker shutdown does not complete?

I previously created a discussion about this behavior:

https://github.com/orgs/taskiq-python/discussions/608

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions