[LU-5223] generate DNE stripe directory LMV EA dynamically via readdir Created: 18/Jun/14 Updated: 10/Jul/14 Resolved: 10/Jul/14 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.6.0 |
| Fix Version/s: | Lustre 2.6.0 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Andreas Dilger | Assignee: | nasf (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | HB, dne2, lfsck | ||
| Issue Links: |
|
||||||||
| Severity: | 3 | ||||||||
| Rank (Obsolete): | 14557 | ||||||||
| Description |
|
During the LFSCK 3 design review, we reached a conclusion that the DNE 2 striped directory implementation and the LFSCK 3 checking of them could be simplified by having the MDT store only the LMV header (struct lmv_mds_md_v1) and dynamically generate the list of stripe FIDs lmv_stripe_fids[] via readdir for sending to the client instead of storing it in the xattr. This simplifies the LFSCK 3 implementation by reducing the number of potential sources of inconsistent information. There is already enough redundancy for the layout of a striped directory because the master/stripe relationship is also encoded in the ".." entry and linkEA of each stripe. In addition, this simplifies the DNE 2 async update implementation since the MDT does not need to log the large master LMV xattr on all of the slave MDTs. For the 2.6 release this would keep the network protocol the same for the client, but would avoid storing a potentially large xattr on the MDS. The master LMV would no longer need to store the lmv_master_fid, and this field can be replaced by two __u64 padding fields. It is desirable to include this change into the 2.6.0 release, since this is the first release with striped directory support, and this would avoid the need for on-disk format compatibility between upgrading and downgrading the MDS. In the future, it would be possible to optimize the handling of very widely striped directories by having the client read the slave directory entries directly from the master directory FID instead of having the master MDT synthesize the LMV xattr. For small striped directories, or small regular directories there could be an optimization to return the readdir data as part of the initial lookup. However, that is out of scope for this initial patch. |
| Comments |
| Comment by nasf (Inactive) [ 19/Jun/14 ] |
|
Here is the patch: |
| Comment by nasf (Inactive) [ 20/Jun/14 ] |
|
Another patch: |
| Comment by Jodi Levi (Inactive) [ 08/Jul/14 ] |
|
Patches landed to Master. |
| Comment by Andreas Dilger [ 10/Jul/14 ] |
|
Another patch that fixes a bug in the original patch also needs to land: |
| Comment by James A Simmons [ 10/Jul/14 ] |
|
The 11040 has been abandoned. New patch at http://review.whamcloud.com/#/c/11035. |
| Comment by Jodi Levi (Inactive) [ 10/Jul/14 ] |
|
Patch landed to Master. |