[LU-2965] MDS evicted OST after attempting recovery.

• Type: Bug
• Resolution: Cannot Reproduce
• Priority: Critical
• Affects Version/s: Lustre 2.4.0
• Labels: LB
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: James A Simmons

Comments

[Peter Jones]

Thanks James - that is good news!

[James A Simmons]

For our last test shot IR worked flawlessly. I think we can close this ticket. If we encounter the bug again this ticket can be reopened.

[Jodi Levi (Inactive)]

Reducing priority until more information is available to understand the issue.

[Li Wei (Inactive)]

From spare.log (why is it called "spare", by the way?):

Mar 8 21:06:01 widow-spare06 kernel: [ 7478.823574] Lustre: routed1-OST00eb-osc-ffff8804319c7000: Connection restored to routed1-OST00eb (at 10.36.227.92@o2ib)
Mar 8 21:06:01 widow-spare06 kernel: [ 7478.845592] Lustre: routed1-OST01ab-osc-ffff8804319c7000: Connection restored to routed1-OST01ab (at 10.36.227.92@o2ib)
Mar 8 21:12:33 widow-spare06 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Mar 8 21:12:33 widow-spare06 kernel: [ 0.000000] Initializing cgroup subsys cpuset
[...]
Mar 8 21:13:34 widow-spare06 kernel: [ 120.446824] ipmi device interface
[EOF]

The client completed recoveries at least with OST00eb and OST01ab, but restarted for some reason. OST00eb last heard from the client at around 21:08:53---two to three minutes after the recovery finished. If this was not something else, the timing was close to the 150s ping interval, suggesting the request heard was probably a ping. The next ping should happen 150s to 300s later, which correspond to 21:11:23 and 21:13:53. Could this eviction merely because the client was simply not remounted after the restart? Any log available after 21:13:34?

[Oleg Drokin]

I guess the assumption clients stopped pinging is because they were evicted for not pinging

[James A Simmons]

10.36.227.198 was our 2.4 non cray client. I attached the log (spare.log) here.