-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
On servers whose isolcpus cores are held by SCHED_FIFO tasks, DNE lock cancels can hang for many minutes after a large unlink:
Lustre: ldlm_cn00_007: service thread pid 37083 was inactive for 3660.157 seconds. __flush_work+0x26b/0x380 tgt_sync+0x127/0x150 [ptlrpc] tgt_blocking_ast+0x2ac/0x750 [ptlrpc]
Delayed work queued without naming a CPU runs on whichever CPU its timer expires on. On Linux 6.9 to 6.18 that can be an isolated CPU, where the work gets only a few percent of the CPU (fixed in 6.19 by 7dec062cfcf2, not in RHEL 10). tgt_sync() waits for the lu_site_flush pass, so ldlm_cn threads wait too. On master, the PM QoS work, ldlm pool recalc and service watchdog also run on isolated CPUs.
Reproduced on a single-node DNE VM with isolcpus=2,3 and busy loops on those CPUs: during an rm -rf of 60000 files, a flush pass took 140s and a cross-MDT mkdir 23s.
Fix: queue delayed work that does not name a CPU on unbound workqueues, which stay off isolated CPUs. With that, the same test ran no Lustre work on the isolated CPUs, and nothing took over 2s.
- is related to
-
LU-18938 lu object free eats so much time
-
- Open
-