https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11970 Lustre <p>We are evaluating Starfish for file system usage detail and have successfully scanned numerous lustre and NFS file systems. However, when we start ingesting data via changelogs we are running into a condition where it will hang and the only resolution is to power cycle the client we are testing with. </p> <p>In trying to identify the problem, we found when we tried to unmount the file system that process would also hang and could not be aborted. </p> <p>lsof of the file system showed the hung process was stuck on  /&lt;file system&gt;/.lustre/fid. Up until this point, we didn't even know that the hidden directory even existed nor its purpose. In scanning Jira, it is involve in lustre rsync and lfsck operations but not a lot of information regarding other roles it plays.</p> <p>One thing is certain: Starfish uses FIDs in there monitoring tools and we can see that .lustre/fid is being identified by the Starfish process.</p> <p>We're hoping we can get some additional information on what's going on with changelogs/.lustre.</p> Dell servers running TOSS (RHEL 7.5) IB connected to DDN SFA hardware LU-11970

Using changelog reader causes fid2path process to lockup in kernel space

Question/Request Minor Resolved Duplicate Mikhail Pershin Joe Mervini Thu, 14 Feb 2019 18:45:26 +0000 Sat, 23 Mar 2019 14:49:09 +0000 Sat, 23 Mar 2019 14:48:38 +0000 Lustre 2.8.0 0 3 <p>Mike</p> <p>Can you advise please?</p> <p>Thanks</p> <p>Peter</p> <p>Joe</p> <p>Is this the Astra system?</p> <p>Peter</p> <p>Peter</p> <p>No - this is on our regular production clusters. We're seeing this behavior on all three of the file systems. </p> <p>One thing we're curious about is why lustre is spitting out FIDs to a directory that is essentially unreadable. One clue is we had the system locked up on a FID that was not in the .lustre directory (although the .lustre directory was still being held by a process identified with lsof) but once the system was rebooted, using fid2path on that FID produced a no such file or directory message.</p> <p>When a file is deleted does it have any interaction with the .lustre directory? If so could it be a race condition? Another question is; is there any client side read operation that would cause a changelog change?  </p> <p>Peter,</p> <p>Has there been any activity on this ticket?</p> <p>Regards,</p> <p>Joe</p> <p>Joe, I am looking into this. Considering this is about Lustre 2.8.0 I am checking tickets which can be related to this problem, probably it is addressed already.</p> <p>This can be resolved by <a href="https://jira.whamcloud.com/browse/LU-8821" title="double find in mdt_path_current()" class="issue-link" data-issue-key="LU-8821"><del>LU-8821</del></a>, there was potential deadlock case in the code. The patch in that ticket can be updated for 2.8 if needed.</p> <p>As for other questions above - /.lustre/fid directory allows to get access to a file by its FID and is used often to get paths to that file from its LinkEA attribute what is fid2path exactly does. If you know FID of object you may access it and modify it but cannot delete it.<br/> The only interaction with deleted files I can think of is that unlinked files which are still opened cannot be found by FID, you'll get 'no such file ..' message while file is being still used by some process. See <a href="https://jira.whamcloud.com/browse/LU-11638" title="lfs fid2path should list open-unlinked files" class="issue-link" data-issue-key="LU-11638">LU-11638</a> for details.<br/> As for changelog logged read operation I assume you meant 'non-modification' operation, yes, we have CL_GETXATTR at least and also CL_DN_OPEN which are non-modification operations but can be recorded in changelog. Also CL_OPEN can be enabled to track OPENs.</p> <p>Believed to be a duplicate of <a href="https://jira.whamcloud.com/browse/LU-8821" title="double find in mdt_path_current()" class="issue-link" data-issue-key="LU-8821"><del>LU-8821</del></a></p> Duplicate LU-8821 Related LU-11501 Development Epic/Theme Lustre-2.8.0 Rank 1|i00bon: Rank (Obsolete) 9223372036854775807