Repository navigation
fix(sched): count NUM_TASK_IN_QUEUE at queue push and pop - #792
Merged
Merged
Conversation
This was referenced Nov 13, 2026
romancardenas
force-pushed
the
fix/sched-num-task-in-queue
branch
from
November 16, 2026 00:20
e2ae9d6 to
d417ea8
Compare
atsushi421
added a commit
that referenced
this pull request
Nov 16, 2026
A waker reads `RUNNING` of a CPU and pins a task to the preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But the CPU clears `RUNNING` and empties the heap without that mutex. If the CPU finishes its task and empties the heap between the read and the pin, it finds no queued task and sleeps. While the CPU runs no task, the preemption IPI can leave the task pinned, and `wake_workers()` does not see the heap. Then the task waits until an unrelated interrupt wakes the CPU. Now `run_main()` checks the heap again before the CPU sleeps. The check comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So the check sees the pin of every waker that read the CPU as busy. On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes every sleeping worker on each loop of the primary CPU. This hides the race. PR #792 fixes the underflow, so this fix must come first. With the previous commit, the EDF scheduler also pins a task while its set has an idle core. So the race can leave those idle cores asleep too. Signed-off-by: atsushi421 <[email protected]>
3 tasks done
romancardenas
pushed a commit
that referenced
this pull request
Nov 16, 2026
A waker reads `RUNNING` of a CPU and pins a task to the preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But the CPU clears `RUNNING` and empties the heap without that mutex. If the CPU finishes its task and empties the heap between the read and the pin, it finds no queued task and sleeps. While the CPU runs no task, the preemption IPI can leave the task pinned, and `wake_workers()` does not see the heap. Then the task waits until an unrelated interrupt wakes the CPU. Now `run_main()` checks the heap again before the CPU sleeps. The check comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So the check sees the pin of every waker that read the CPU as busy. On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes every sleeping worker on each loop of the primary CPU. This hides the race. PR #792 fixes the underflow, so this fix must come first. With the previous commit, the EDF scheduler also pins a task while its set has an idle core. So the race can leave those idle cores asleep too. Signed-off-by: atsushi421 <[email protected]>
NUM_TASK_IN_QUEUE underflowed to u32::MAX. ArcWake::wake incremented it, and get_next_task() decremented it when a non-clustered scheduler returned a task. Other paths did not match: - The re-wakes in do_preemption() and run_main(), and the re-wake of a preempted task, call wake_task() without ArcWake::wake, so their tasks were decremented but never incremented. - A task that invoke_preemption() pushed to a preemption-pending heap was incremented but runs without get_next_task(). - get_next() skips terminated and panicked tasks without a decrement. With the counter at u32::MAX, wake_workers() woke every sleeping worker on every loop of the primary CPU. Now the PrioritizedFIFO, PrioritizedRR, and Panicked schedulers increment the counter when they push a task to their queue and decrement it when they pop one. So the counter equals the number of queued non-clustered tasks on every path. These schedulers also wake the primary CPU after they enqueue a task, as the EDF scheduler does, because the re-wakes do not go through ArcWake::wake. The two target-specific definitions of the counter become one. Signed-off-by: atsushi421 <[email protected]>
romancardenas
force-pushed
the
fix/sched-num-task-in-queue
branch
from
November 16, 2026 05:11
d417ea8 to
47b6f5d
Compare
- Call wake_workers() before the primary CPU sleeps, so a task that a timer wakes on CPU 0 reaches a worker without waiting for the sleep to end - Remove the duplicate wake_cpu(0) in ArcWake::wake and reword the notify comments in the schedulers Signed-off-by: atsushi421 <[email protected]>
atsushi421
marked this pull request as ready for review
November 16, 2026 08:29
Contributor
|
The change looks correct to me, and I think it can be merged. I created the following issues related to this PR. |
ytakano
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes the underflow of
NUM_TASK_IN_QUEUE. #789 stacks on this PR, because it needs an exact counter.On QEMU,
NUM_TASK_IN_QUEUEunderflowed tou32::MAX.ArcWake::wakeincremented it, andget_next_task()decremented it when a non-clustered scheduler returned a task. Other paths did not match:do_preemption()andrun_main(), and the re-wake of a preempted task, callwake_task()withoutArcWake::wake. Their tasks were decremented but never incremented.invoke_preemption()pushed to a preemption-pending heap was incremented, but it runs withoutget_next_task().get_next()skips terminated and panicked tasks without a decrement.With the counter at
u32::MAX,wake_workers()woke every sleeping worker on every loop of the primary CPU. This bug is older than #787.Changes:
ArcWake::wakeandget_next_task()no longer change it.ArcWake::wake, and before this PR the extra wakes hid the missing notification.ArcWake::wakeno longer wakes the primary CPU, because every scheduler now does it after an enqueue.wake_workers()before it sleeps, not after.wake_cpu(0)on CPU 0 does nothing, so a task that a timer wakes on CPU 0 waited until the sleep ended (up to 1 second) when all workers slept. Before this PR the saturated counter hid this delay.Related links
How was this PR tested?
scripts/ci.shpasses on my machine.test_dagon QEMU (aarch64_virt, 4 CPUs), the maximum ofNUM_TASK_IN_QUEUEwas 3, andwake_workers()woke 55 and 433 workers for non-clustered tasks in two runs. On the base branch, the counter wasu32::MAX, andwake_workers()woke 90697 workers.test_prioritized_fifo,test_prioritized_rr,test_voluntary_preemption,test_async_mutex,test_pubsub,test_multi_prioritized_scheduler,test_dag, andtest_edfgive the same results as on the base branch. The panic intest_pubsubis the intendedpanic teston both branches.Notes for reviewers