[LU-10288] LFSCK support for mirrored files

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.12.0
• Type: Improvement
• Priority: Critical
• Reporter: Jinshan Xiong (Inactive)
• Assignee: nasf (Inactive)
• Resolution: Fixed
• Labels: FLR2

Description

PFID EA should already have enough information to restore mirrored files in a catastrophic failure where the metadata on the MDT is lost. After FLR is introduced, the filter_fid contains the following information:

struct ost_layout {                                                             
        __u32   ol_stripe_size;                                                 
        __u32   ol_stripe_count;                                                
        __u64   ol_comp_start;                                                  
        __u64   ol_comp_end;                                                    
        __u32   ol_comp_id;                                                     
} __attribute__((packed)); 

struct filter_fid {                                                             
        struct lu_fid           ff_parent;                                      
        struct ost_layout       ff_layout;                                      
        __u32                   ff_layout_version;                              
        __u32                   ff_range; /* range of layout version that       
                                           * write are allowed */               
} __attribute__((packed)); 

And component ID is composed of SEQ_ID and MIRROR_ID as follows:

#define SEQ_ID_MAX              0x0000FFFF                                      
#define SEQ_ID_MASK             SEQ_ID_MAX                                      
/* bit 30:16 of lcme_id is used to store mirror id */                           
#define MIRROR_ID_MASK          0x7FFF0000                                      
#define MIRROR_ID_SHIFT         16  

With the above information, the LFSCK just needs to use FID to identify the OST objects that belong to the same file, and use mirror ID, the upper 16 bit in component ID, to identify the components that belong to the same mirror, and the use SEQ ID and ol_comp_start, ol_comp_end in ost_layout to compose components in one mirror.

The problem is how to identify and restore stale components. By checking the information of ff_layout_version and ff_range, it should be easy to know whether the file was being written at the time of failure; but it seems to be difficult to identify if a previous resync was complete. Therefore, we probably need more information for this purpose.

Comments

[Joseph Gmitter (Inactive)]

Moved out from under LU-9771 as it belongs to FLR2

[nasf (Inactive)]

Consider the DoM case, things will be more complex. The DoM data is directly attached to the MDT-object, if the LOV EA corrupted, we have to re-calculate the DoM boundary from its subsequent component range. But the component ID may be discontinuous, so during the LOV EA rebuilding, we have to guess the end boundary of DoM, and even if after the LFSCK, we may still not know exactly which one is the second component for the mirror. So we may have no way to recover the LOV EA exactly the same as before the corruption.

[nasf (Inactive)]

Another issue is about how to detect whether a file is DoM case or not if its LOV EA corrupted. Currently, the LFSCK can check the MDT-object's size on the MDT, if it is non-zero, then it will DoM case; otherwise, it can be handled as any case. But such way will be broken once SoM related feature is introduced. So we some more reliable mechanism.

[Andreas Dilger]

Another issue is about how to detect whether a file is DoM case or not if its LOV EA corrupted. Currently, the LFSCK can check the MDT-object's size on the MDT, if it is zero, then it will DoM case; otherwise, it can be handled as any case.

Wouldn't it be true that a nonzero size and non-zero blocks means it is DoM? (Note there may be upgraded 1.8 MDTs that still store total file size in inode>i_size that we should probably fix)

But such way will be broken once SoM related feature is introduced. So we some more reliable mechanism.

The SOM feature will store the file size/blocks in a separate "som" xattr, so it should not interfere with DoM detection.

[nasf (Inactive)]

Yes, the DoM is for non-zero case. I am not sure whether we still have customer to use 1.8 formatted file which MDT-object's size is not zero. If yes, then its LOV EA is non-PFL mode, so we can know it is not DoM case; if the 1.8 file lost its LOV EA, then its OST-object will declare as orphan, but 1.8 OST-object has no component ID, so the LFSCK should know it is for non-DoM file.

[nasf (Inactive)]

The DoM related issue will be handled via another independent ticket LU-11081. This ticket will only process mirrored file without DoM component.

[Gerrit Updater]

Fan Yong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32705
Subject: LU-10288 lfsck: layout LFSCK for mirrored file
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 276b919a8b5f23c6e8d75c5907c58b760c87657c

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32705/
Subject: LU-10288 lfsck: layout LFSCK for mirrored file
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 36ba989752c62cc76b06089373fcd6cec6da9008

[Peter Jones]

Landed for 2.12