Skip to content

fix(sched): count NUM_TASK_IN_QUEUE at queue push and pop - #792

Merged
ytakano merged 2 commits into
mainfrom
fix/sched-num-task-in-queue
Nov 17, 2026
Merged

ytakano merged 2 commits into
mainfrom
fix/sched-num-task-in-queue

Conversation

@atsushi421

@atsushi421 atsushi421 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR fixes the underflow of NUM_TASK_IN_QUEUE. #789 stacks on this PR, because it needs an exact counter.

On QEMU, NUM_TASK_IN_QUEUE underflowed to u32::MAX. ArcWake::wake incremented it, and get_next_task() decremented it when a non-clustered scheduler returned a task. Other paths did not match:

  • The re-wakes in do_preemption() and run_main(), and the re-wake of a preempted task, call wake_task() without ArcWake::wake. Their tasks were decremented but never incremented.
  • A task that invoke_preemption() pushed to a preemption-pending heap was incremented, but it runs without get_next_task().
  • get_next() skips terminated and panicked tasks without a decrement.

With the counter at u32::MAX, wake_workers() woke every sleeping worker on every loop of the primary CPU. This bug is older than #787.

Changes:

  • The PrioritizedFIFO, PrioritizedRR, and Panicked schedulers increment the counter when they push a task to their queue and decrement it when they pop one. So the counter equals the number of queued non-clustered tasks on every path. ArcWake::wake and get_next_task() no longer change it.
  • These schedulers wake the primary CPU after they enqueue a task, as the EDF scheduler does since feat(sched): merge GEDF into ClusteredEDF and rename it to EDF #787. The re-wakes do not go through ArcWake::wake, and before this PR the extra wakes hid the missing notification.
  • ArcWake::wake no longer wakes the primary CPU, because every scheduler now does it after an enqueue.
  • The primary CPU calls wake_workers() before it sleeps, not after. wake_cpu(0) on CPU 0 does nothing, so a task that a timer wakes on CPU 0 waited until the sleep ended (up to 1 second) when all workers slept. Before this PR the saturated counter hid this delay.
  • The two target-specific definitions of the counter become one.

Related links

How was this PR tested?

  • scripts/ci.sh passes on my machine.
  • With temporary counters during 30 seconds of test_dag on QEMU (aarch64_virt, 4 CPUs), the maximum of NUM_TASK_IN_QUEUE was 3, and wake_workers() woke 55 and 433 workers for non-clustered tasks in two runs. On the base branch, the counter was u32::MAX, and wake_workers() woke 90697 workers.
  • On QEMU, test_prioritized_fifo, test_prioritized_rr, test_voluntary_preemption, test_async_mutex, test_pubsub, test_multi_prioritized_scheduler, test_dag, and test_edf give the same results as on the base branch. The panic in test_pubsub is the intended panic test on both branches.

Notes for reviewers

@atsushi421
atsushi421 added this pull request to stack #793 November 13, 2026 05:53
@romancardenas
romancardenas force-pushed the fix/sched-num-task-in-queue branch from e2ae9d6 to d417ea8 Compare November 16, 2026 00:20
atsushi421 added a commit that referenced this pull request Nov 16, 2026
A waker reads `RUNNING` of a CPU and pins a task to the
preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But
the CPU clears `RUNNING` and empties the heap without that mutex. If
the CPU finishes its task and empties the heap between the read and the
pin, it finds no queued task and sleeps. While the CPU runs no task,
the preemption IPI can leave the task pinned, and `wake_workers()` does
not see the heap. Then the task waits until an unrelated interrupt
wakes the CPU.

Now `run_main()` checks the heap again before the CPU sleeps. The check
comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So
the check sees the pin of every waker that read the CPU as busy.

On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes
every sleeping worker on each loop of the primary CPU. This hides the
race. PR #792 fixes the underflow, so this fix must come first. With
the previous commit, the EDF scheduler also pins a task while its set
has an idle core. So the race can leave those idle cores asleep too.

Signed-off-by: atsushi421 <[email protected]>
Base automatically changed from feat/edf-idle-core-preemption to main November 16, 2026 05:11
romancardenas pushed a commit that referenced this pull request Nov 16, 2026
A waker reads `RUNNING` of a CPU and pins a task to the
preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But
the CPU clears `RUNNING` and empties the heap without that mutex. If
the CPU finishes its task and empties the heap between the read and the
pin, it finds no queued task and sleeps. While the CPU runs no task,
the preemption IPI can leave the task pinned, and `wake_workers()` does
not see the heap. Then the task waits until an unrelated interrupt
wakes the CPU.

Now `run_main()` checks the heap again before the CPU sleeps. The check
comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So
the check sees the pin of every waker that read the CPU as busy.

On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes
every sleeping worker on each loop of the primary CPU. This hides the
race. PR #792 fixes the underflow, so this fix must come first. With
the previous commit, the EDF scheduler also pins a task while its set
has an idle core. So the race can leave those idle cores asleep too.

Signed-off-by: atsushi421 <[email protected]>
NUM_TASK_IN_QUEUE underflowed to u32::MAX. ArcWake::wake incremented
it, and get_next_task() decremented it when a non-clustered scheduler
returned a task. Other paths did not match:

- The re-wakes in do_preemption() and run_main(), and the re-wake of a
  preempted task, call wake_task() without ArcWake::wake, so their
  tasks were decremented but never incremented.
- A task that invoke_preemption() pushed to a preemption-pending heap
  was incremented but runs without get_next_task().
- get_next() skips terminated and panicked tasks without a decrement.

With the counter at u32::MAX, wake_workers() woke every sleeping worker
on every loop of the primary CPU.

Now the PrioritizedFIFO, PrioritizedRR, and Panicked schedulers
increment the counter when they push a task to their queue and
decrement it when they pop one. So the counter equals the number of
queued non-clustered tasks on every path. These schedulers also wake
the primary CPU after they enqueue a task, as the EDF scheduler does,
because the re-wakes do not go through ArcWake::wake. The two
target-specific definitions of the counter become one.

Signed-off-by: atsushi421 <[email protected]>
@romancardenas
romancardenas force-pushed the fix/sched-num-task-in-queue branch from d417ea8 to 47b6f5d Compare November 16, 2026 05:11
- Call wake_workers() before the primary CPU sleeps, so a task that a timer wakes on CPU 0 reaches a worker without waiting for the sleep to end
- Remove the duplicate wake_cpu(0) in ArcWake::wake and reword the notify comments in the schedulers

Signed-off-by: atsushi421 <[email protected]>
@atsushi421
atsushi421 marked this pull request as ready for review November 16, 2026 08:29
@atsushi421
atsushi421 requested a review from ytakano November 16, 2026 08:29
@ytakano

ytakano commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@ytakano
ytakano merged commit 08ff0d3 into main Nov 17, 2026
1 check passed
@ytakano
ytakano deleted the fix/sched-num-task-in-queue branch November 17, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants