https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-31 Lustre <p>eeb, after some further investigation for readdir+, I found there some issues for my original lockless mode readdir+, I have described them in the HLD, they are difficult to be resolved. So I have made the HLD for simplified lock mode readdir+, and also hope to implement the lustre special traversing directory tool.</p> <p>Sir, please inspect the HLD when you have time, and give your suggestion. Thanks!</p> <p>Happy Christmas~</p> LU-31

Please inspect the HLD for readdir+

Review task LU-23 Minor Resolved Won't Do Eric Barton nasf Thu, 23 Dec 2010 23:38:15 +0000 Wed, 6 Jun 2018 00:50:18 +0000 Wed, 6 Jun 2018 00:50:18 +0000 0 1 <p>updated at 2010-12-29</p> <p>updated at 2010-12-29</p> <p>Hi, I got a chance to read through the document for interest, and have a question:</p> <ul> <li>I think we don't pass parent fid over wire for setattr and setxattr, 0.3.4 says we want to take CW LIST lock of parent for these operations, does this mean that we have to change RPC format for setattr and setxattr, Which may have compatible issue with old version clients like 1.8.* or 2.0.*?</li> </ul> <p>In fact, as current master implementation, when create, the pfid is recorded as some kind of xattr by child, such information can be used by readdir+.</p> <p>I have some concerns:</p> <ul> <li>size glimpse RPC comprises most of the RPCs in readdir+, compared with aggregate<br/> glimpse, SOM can avoid most of such RPCs. We'd better evaluate the current progress<br/> and remaining work of SOM, if there is no roadblocks, SOM should be a better option<br/> than aggregate glimpse.</li> <li>I prefer STL (Sub Tree Lock) than the new LIST lock, because the first one has two<br/> advantages: <ul> <li>it can avoid access conflicts (both pseudo and real ones) because it's weak, while<br/> LIST lock looks more like strong STL.</li> <li>STL can be used in the future for WBC.</li> </ul> </li> <li>no need to introduce a new READDIRPLUS RPC, we can expand current MDS_READPAGE in a<br/> backward compatible way: in the request to MDS client specifies what attributes it<br/> wants along with entry names. In this way readdir() will issue normal MDS_READPAGE<br/> and store directory entries in directory page cache; while statahead thread issue<br/> MDS_READPAGE to fetch file attributes.</li> </ul> <p>I sort of agree and disagree on this point from Lai:<br/> &gt; size glimpse RPC comprises most of the RPCs in readdir+, compared with aggregate<br/> &gt; glimpse, SOM can avoid most of such RPCs. We'd better evaluate the current progress<br/> &gt; and remaining work of SOM, if there is no roadblocks, SOM should be a better option<br/> &gt; than aggregate glimpse.</p> <p>SOM will definitely bring in a big boost here, but it's a long work to make SOM really happen with all the cases.</p> <p>As for glimpse aggregation, I sort of agree that it won't bring in many of the benefits. But what will bring in many benefits, almost at SOM level is async "glimpse ahead" just kind of like what we currently do with statahead. As long as we can get the data faster than app processes it, of course.</p> <p>Considering that file may have multiple stripes (max 160), async glimpse may not help much. Besides, if the system is WAN based, async glimpse may not help at all.</p> <p>IMHO current SOM design is too complicated, because it tries to ensure client update SOM in all cases (server and client recovery), if it's allowed to miss SOM update and let MDS update it later itself (when MDS finds SOM not set upon handling a getattr from client), SOM can be much simplified.</p> <p>Related documents became inactive for a long time, need more consideration for readdir+ feature.</p> Bugzilla ID 17845.0 Development Rank 1|hzw07z: Rank (Obsolete) 10115