https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-15353 Lustre <p>Due to a hardware fault, we lost power on a disk array that was part of an OSS. The disk groups in the RAID report as recovered, and an fsck was run, but we are now getting errors, e.g.,:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Dec  9 12:32:20 coss5 kernel: LustreError: 23370:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a6dc07:0x0]: -30 Dec  9 12:32:20 coss5 kernel: LustreError: 23370:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 15 previous similar messages Dec  9 12:44:39 coss5 kernel: LustreError: 23479:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x59ebd23:0x0]: -30 Dec  9 12:44:39 coss5 kernel: LustreError: 23479:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 26 previous similar messages Dec  9 12:54:54 coss5 kernel: LustreError: 21968:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a10caf:0x0]: -30 Dec  9 12:54:54 coss5 kernel: LustreError: 21968:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 113 previous similar messages Dec  9 13:02:16 coss5 kernel: LustreError: 22016:0:(tgt_lastrcvd.c:1133:tgt_client_del()) charlie-OST0054: failed to update server data, skip client 40af1999-fab1-c3b9-87e0-a070dbf6eb50 zeroing, rc -30 Dec  9 13:05:10 coss5 kernel: LustreError: 22500:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a6f63d:0x0]: -30 Dec  9 13:05:10 coss5 kernel: LustreError: 22500:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 4 previous similar messages Dec  9 13:08:20 coss5 kernel: LustreError: 12705:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:08:58 coss5 kernel: LustreError: 22537:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:09:23 coss5 kernel: LustreError: 22015:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:09:48 coss5 kernel: LustreError: 22535:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:10:13 coss5 kernel: LustreError: 19790:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:10:38 coss5 kernel: LustreError: 22014:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:11:03 coss5 kernel: LustreError: 12704:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:11:53 coss5 kernel: LustreError: 22004:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:11:53 coss5 kernel: LustreError: 22004:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 1 previous similar message Dec  9 13:13:08 coss5 kernel: LustreError: 22048:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:13:08 coss5 kernel: LustreError: 22048:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 2 previous similar messages Dec  9 13:15:39 coss5 kernel: LustreError: 22014:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:15:39 coss5 kernel: LustreError: 22014:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 5 previous similar messages Dec  9 13:18:45 coss5 kernel: LustreError: 12704:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a0c6ae:0x0]: -30 Dec  9 13:18:45 coss5 kernel: LustreError: 12704:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 14 previous similar messages Dec  9 13:20:15 coss5 kernel: LustreError: 12704:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:20:15 coss5 kernel: LustreError: 12704:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 10 previous similar messages Dec  9 13:29:02 coss5 kernel: LustreError: 22001:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:29:02 coss5 kernel: LustreError: 22001:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 20 previous similar messages Dec  9 13:29:03 coss5 kernel: LustreError: 22490:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a1cc60:0x0]: -30 Dec  9 13:29:03 coss5 kernel: LustreError: 22490:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 660 previous similar messages Dec  9 13:39:04 coss5 kernel: LustreError: 12699:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:39:04 coss5 kernel: LustreError: 12699:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 23 previous similar messages Dec  9 13:40:09 coss5 kernel: LustreError: 23370:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x5a0c9dd:0x0]: -30 Dec  9 13:40:09 coss5 kernel: LustreError: 23370:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 39 previous similar messages Dec  9 13:49:06 coss5 kernel: LustreError: 22492:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:49:06 coss5 kernel: LustreError: 22492:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 23 previous similar messages Dec  9 13:51:10 coss5 kernel: LustreError: 12702:0:(ofd_dev.c:1804:ofd_destroy_hdl()) charlie-OST0054: error destroying object [0x100540000:0x3a33c87:0x0]: -30 Dec  9 13:51:10 coss5 kernel: LustreError: 12702:0:(ofd_dev.c:1804:ofd_destroy_hdl()) Skipped 2 previous similar messages Dec  9 13:59:08 coss5 kernel: LustreError: 12693:0:(tgt_lastrcvd.c:1027:tgt_client_new()) charlie-OST0054: Failed to write client lcd at idx 3, rc -30 Dec  9 13:59:08 coss5 kernel: LustreError: 12693:0:(tgt_lastrcvd.c:1027:tgt_client_new()) Skipped 23 previous similar messages</pre> </div></div> <p>Can you please tell me what the steps are to resolve this issue</p> <p>Campbell</p> LU-15353

OST: error destroying object, Failed to write client lcd

Bug Major Resolved Not a Bug Andreas Dilger Dneg Thu, 9 Dec 2021 14:36:49 +0000 Fri, 10 Dec 2021 13:53:40 +0000 Fri, 10 Dec 2021 13:53:40 +0000 0 3 <p>-30 = -EROFS, so it looks like the filesystem was mounted read-only due to the previous error. That would have been posted in the logs before the filesystem turned read-only. </p> <p>The OST needs to be unmounted, run e2fsck, and remounted. </p> <p>Thanks Andreas. I think I made the mistake of running -p rather than -y on my fscks before I opened this ticket. All good now</p> Development Rank 1|i02c4n: Rank (Obsolete) 9223372036854775807 Severity