https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-6712 Lustre <p>Attempting to setup for SWL test - two clients out of the total have this issue:<br/> Tests fail due to IO error when creating or listing directories<br/> Example:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"># ls /p/l_wham/white215/ ls: cannot access /p/l_wham/white215/mybob: Input/output error ls: cannot access /p/l_wham/white215/SWL: Input/output error SWL foo mybob </pre> </div></div> <p>Client errors:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">LustreError: 47371:0:(file.c:3081:ll_inode_revalidate_fini()) lustre: revalidate FID [0x20000040c:0x1f:0x0] error: rc = -5 LustreError: 47376:0:(file.c:3081:ll_inode_revalidate_fini()) lustre: revalidate FID [0x20000040c:0x1f:0x0] error: rc = -5 LustreError: 47376:0:(file.c:3081:ll_inode_revalidate_fini()) Skipped 1 previous similar message </pre> </div></div> <p>Lustre dump from one client attached. Does not appear to be errors on MDS<br/> Remounting the client appears to clear the issue. </p> Hyerion SWL test LU-6712

Hyperion IO error revalidate FID [0x20000040c:0x1f:0x0] error: rc = -5

Bug Critical Resolved Cannot Reproduce Di Wang Cliff White Thu, 11 Jun 2015 21:28:11 +0000 Sun, 10 Oct 2021 21:25:26 +0000 Sun, 10 Oct 2021 21:25:26 +0000 Lustre 2.8.0 0 3 <p>looks like FLD cache on the client has some problem</p> <p>Client allocate the FID at MDT000a </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>40000000:00000040:3.0:1434056723.090873:0:55019:0:(fid_request.c:382:seq_client_alloc_fid()) cli-cli-lustre-MDT000a-mdc-ffff88085864d400: Allocated FID [0x114000040e:0x14:0x0] </pre> </div></div> <p>Then in the following, it lookup this FID in FLD cache can get the FID in MDT000b</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>00800000:00000002:3.0:1434056723.090888:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #b for fid=[0x114000040e:0x14:0x0] </pre> </div></div> <p>Cliff: Could you please do</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lctl get_param fld.*MDT0000.fldb </pre> </div></div> <p>on MDT0000 and post the result here. Thanks.</p> <p>results of lctl get_param fld.*MDT0000.fldb” on Hyperion MDT0</p> <p>According to the debug log and FLDB (MDT0), it is clearly the fldb cache on the client side is corrupted. </p> <p>Client side cache</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>iwc34.fini.txt:00800000:00000002:3.0:1434056723.090390:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #0 for fid=[0x200000400:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090402:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #1 for fid=[0x1180000409:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090412:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #3 for fid=[0x1040000403:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090421:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #4 for fid=[0xf80000406:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090430:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #5 for fid=[0x1100000409:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090440:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #a for fid=[0xfc0000405:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090449:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #6 for fid=[0xf40000401:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090458:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #7 for fid=[0x11c0000402:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090467:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #9 for fid=[0x10c0000406:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090476:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #2 for fid=[0x1000000404:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090485:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #b for fid=[0x1140000407:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090494:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #8 for fid=[0x1080000402:0x84f:0x0] iwc34.fini.txt:00800000:00000002:3.0:1434056723.090888:0:55019:0:(lmv_fld.c:79:lmv_fld_lookup()) FLD lookup got mds #b for fid=[0x114000040e:0x14:0x0] </pre> </div></div> <p>On the server side</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[0x0000000f40000400-0x0000000f80000400):6:mdt [0x0000000f80000400-0x0000000fc0000400):1:mdt [0x0000000fc0000400-0x0000001000000400):7:mdt [0x0000001000000400-0x0000001040000400):2:mdt [0x0000001040000400-0x0000001080000400):8:mdt [0x0000001080000400-0x00000010c0000400):3:mdt [0x00000010c0000400-0x0000001100000400):9:mdt [0x0000001100000400-0x0000001140000400):4:mdt [0x0000001140000400-0x0000001180000400):a:mdt [0x0000001180000400-0x00000011c0000400):5:mdt [0x00000011c0000400-0x0000001200000400):b:mdt </pre> </div></div> <p>Unfortunately, I can not find the problem by checking the code. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Cliff: Could you please tell me<br/> 1. Anything special for hyperion clients? Only 1 filesystem right?<br/> 2. Could you please do "lctl get_param fld.*.fldb" on all of MDS and post the result here?<br/> 3. Is there recovery during SWL tests? or if SWL tests is not running? Could you please tell what did you run before this happens?</p> <p>Same data from all MDS</p> Development Rank 1|hzxflb: Rank (Obsolete) 9223372036854775807 Severity