[LU-8714] too many update logs during soak-test.

Related — Sun, 30 Jan 2022 09:49:31 +0000

In the last DNE soak test, we found the recovery stuck there for very long time, (> 4 hours). It looks like there are too much update log being left during recovery.

Each MDT has around 80k-100k records, which seems too much,

wangdi-mac01:~ wangdi$ grep -r "mdt_index 1" /tmp/records  | wc
   76104  913248 15353624
wangdi-mac01:~ wangdi$ grep -r "mdt_index 0" /tmp/records  | wc
   91589 1099068 19376239
wangdi-mac01:~ wangdi$ grep -r "mdt_index 2" /tmp/records  | wc
  102798 1233576 21763151
wangdi-mac01:~ wangdi$ grep -r "mdt_index 3" /tmp/records  | wc
   98332 1179984 20821847

Unfortunately, there are not much logs to help me understanding why there are so much logs being left.

But it seems we can make cancellation smarter. In current implementation, when one batchid is committed, it only cancel the update records for this batchid, but we actually can cancel all of update records, whose batchid < current committed batchid. Then even if some update recordss might be left for some reasons, these recordss can still be deleted by later batchid commitment.

Whamcloud Community JIRA

[LU-8714] too many update logs during soak-test.