[LU-11367] integrate LSOM with lfs find Created: 12/Sep/18 Updated: 28/Jul/20 Resolved: 20/Sep/19 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.12.0 |
| Fix Version/s: | Lustre 2.13.0, Lustre 2.12.4 |
| Type: | Improvement | Priority: | Major |
| Reporter: | Andreas Dilger | Assignee: | Qian Yingjin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | LSOM, performance | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||||||||||||||||||||||||||
| Description |
|
The LSOM functionality should be integrated with lfs find so that it is possible to use this functionality directly on the client. The MDS can fill in the mbo_size and mbo_blocks fields from the LSOM xattr, if the actual size/blocks are not available, and then set new OBD_MD_FLLSIZE and OBD_MD_FLLBLOCKS flags in the reply so that the client knows these fields are valid. The lfs find command can add a "--lazy" option (or maybe "--lsize" and "--lblocks"?) to allow the use of LSOM data from the MDS. We might consider to also use this for "--size" when it is specified in some large range, like "--size -1G" or "--blocks +1M" since we don't need accurate sizes to match those requests. That could probably be done as a separate patch after the initial functionality is implemented and we have some idea how well LSOM is keeping the size uptodate. Please include updates to the command usage and man page. |
| Comments |
| Comment by Gerrit Updater [ 01/Nov/18 ] |
|
Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/33545 |
| Comment by Gerrit Updater [ 30/Apr/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33545/ |
| Comment by Peter Jones [ 30/Apr/19 ] |
|
Landed for 2.13 |
| Comment by Gerrit Updater [ 30/Apr/19 ] |
|
Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34780 |
| Comment by Gerrit Updater [ 30/Apr/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34780/ |
| Comment by Oleg Drokin [ 30/Apr/19 ] |
|
reopening because of revert. |
| Comment by Andreas Dilger [ 06/May/19 ] |
|
We need a new version of patch https://review.whamcloud.com/33545 pushed with the fix for |
| Comment by Gerrit Updater [ 11/Jun/19 ] |
|
Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35167 |
| Comment by Nathan Rutman [ 29/Aug/19 ] |
|
A related ticket added llapi accessor functions for lum components, but I can't seem to find it now - if someone knows, please add an issue link. |
| Comment by Andreas Dilger [ 29/Aug/19 ] |
|
It would be useful, but not required before the patch is landed, to run a performance comparison of "lfs find $MOUNT --lazy -size +N" with "find $MOUNT -size +N", clearing the cache before the two runs, especially if the OSTs are disk based and the MDT is flash based (or cached in RAM). That would be useful to include in the commit message (if ready in time), and for presentations about this feature. |
| Comment by Gerrit Updater [ 20/Sep/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35167/ |
| Comment by Peter Jones [ 20/Sep/19 ] |
|
Landed for 2.13 |
| Comment by Gerrit Updater [ 22/Oct/19 ] |
|
Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36553 |
| Comment by Gerrit Updater [ 21/Nov/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36553/ |
| Comment by John Hammond [ 30/Jun/20 ] |
|
Renaming ioctls in this way is a bad idea and will only cause more problems than it fixes. Please do not do this. |