[LU-11994] Add support for LSOM in LFSCK Created: 23/Feb/19 Updated: 13/Oct/21 |
|
| Status: | Open |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | Lustre 2.16.0 |
| Type: | Improvement | Priority: | Major |
| Reporter: | Li Xi | Assignee: | Qian Yingjin |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||
| Description |
|
From Andreas: It makes sense to enhance LFSCK to update the LSOM size during a scan so that it is available and accurate for all existing files. |
| Comments |
| Comment by Li Xi [ 13/Mar/19 ] |
|
Agreed on the importance of this ticket. @Joeseph Gmitter Anyone can help Yingjin on the LFSCK development? It would reduce the time spending on learning the codes. |
| Comment by Patrick Farrell (Inactive) [ 14/Mar/19 ] |
|
So how would we do this? The client has a mechanism for getting size, sending glimpse requests to the OSTs, but is that available from the MDS/from lfsck? I assume not, since it uses the whole CLIO stack to calculate file size. Is there an equivalent mechanism available to the MDS? The MDS has the layout, of course, but the problem is using it to get the size. Next question is, where would we put this? A new kind of lfsck scan seems like the right place for this, rather than in one of the existing scans, but I don't have that much detail on lfsck. A little more input on the high level design would help make this implementable for someone like Yingjin or I, who don't know lfsck well. |
| Comment by Alex Zhuravlev [ 14/Mar/19 ] |
|
LFSCK can get attributes for OST objects using OUT. not sure one dedicated scan is required. |
| Comment by Andreas Dilger [ 22/Feb/21 ] |
|
I don't think that the whole CLIO stack is needed for this. There are functions available in the LOV code to calculate the file size from the file layout and the object sizes, essentially just calling lov_stripe_size() (or an equivalent function) for each stripe the last initialized component and pick the largest size would be enough for 99% of cases. In some rare cases, the file was truncated smaller than the start of the last component, in which case lov_stripe_size() should be called on stripes of the previous component, repeat as necessary until an object with a size >= start of component or first component is hit. I also don't think a dedicated scan is needed. The layout scan is already checking every OST object in the filesystem in order to check for object existence, fix ownership for quota, etc., so that is enough to get the object size to the MDS as needed. |