[LU-4788] LFSCK 3: MDT-MDT consistency verification Created: 20/Mar/14  Updated: 26/Feb/15  Resolved: 05/Nov/14

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.7.0

Type: New Feature Priority: Critical
Reporter: nasf (Inactive) Assignee: nasf (Inactive)
Resolution: Fixed Votes: 0
Labels: LFSCK

Sub-Tasks:
Key
Summary
Type
Status
Assignee
LU-4836 LFSCK 3 Test Plan Technical task Resolved nasf  
LU-5506 LFSCK 3: Skip orphan objects handling... Technical task Resolved nasf  
LU-5508 LFSCK 3: RPC adjustment for remote tr... Technical task Resolved nasf  
LU-5509 LFSCK 3: get PFID from linkEA for rem... Technical task Resolved nasf  
LU-5511 LFSCK 3: repair unmatched distributed... Technical task Resolved nasf  
LU-5512 LFSCK 3: repair dangling name entry Technical task Resolved nasf  
LU-5513 LFSCK 3: repair multiple referenced n... Technical task Resolved nasf  
LU-5515 LFSCK 3: repair bad file type in name... Technical task Resolved nasf  
LU-5516 LFSCK 3: repair the lost name entry Technical task Resolved nasf  
LU-5517 LFSCK 3: repair invalid nlink count Technical task Resolved nasf  
LU-5518 LFSCK 3: recover client visible objec... Technical task Resolved nasf  
LU-5519 LFSCK 3: verify striped directory Technical task Resolved nasf  
Epic: LFSCK
Project: OpenSFS SFS-DEV-001
Rank (Obsolete): 13167

 Description   

In LFSCK 3, we will not only extend the FID-in-dirent and linkEA verification from single MDT to cross-MDTs cases, but also some additional inconsistent check/repair for remote object and striped directory, includes:
1) Dangling referenced name entry: The name entry exists, but related MDT-object does not exist.
2) Orphan MDT-object: The MDT-object exists, but there is no name entry to reference it.
3) Multiple-referenced name entry: More than one MDT-objects point back to the same name entry, but the name entry only references one of them.
4) Unmatched name entry and MDT-object pairs: The name entry references the MDT-object which has no linkEA for back-reference or it points back to another name entry that does not exists or does not reference the MDT-object.
5) Redundant or invalid linkEA entries: Repeated linkEA entries, or bad parent FID, or the parent FID is not for MDT-object, or not allocated yet.
6) Invalid nlink count: The MDT-object's nlink count does not matches the name entries (which reference such MDT-object) count.



 Comments   
Comment by nasf (Inactive) [ 20/Aug/14 ]

The patches have been landed to master for LFSCK 3:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/

Comment by nasf (Inactive) [ 20/Aug/14 ]

Other patches that are related with LFSCK and needs to be landed to master for LFSCK 3:
1) http://review.whamcloud.com/#/c/11275/
2) http://review.whamcloud.com/#/c/11276/
3) http://review.whamcloud.com/#/c/11288/
4) http://review.whamcloud.com/#/c/11407/
5) http://review.whamcloud.com/#/c/11304/
6) http://review.whamcloud.com/#/c/11373/

Comment by nasf (Inactive) [ 20/Aug/14 ]

The LFSCK 3 patches for LFSCK framework adjustment to be landed to master:
1) http://review.whamcloud.com/#/c/10986/
2) http://review.whamcloud.com/#/c/10987/
3) http://review.whamcloud.com/#/c/10602/
4) http://review.whamcloud.com/#/c/10493/
5) http://review.whamcloud.com/#/c/10603/
6) http://review.whamcloud.com/#/c/11317/

Comment by nasf (Inactive) [ 22/Aug/14 ]

The total patches list for all LFSCK 3 related patches:

Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/

To be landed:

1) http://review.whamcloud.com/#/c/11486/
2) http://review.whamcloud.com/#/c/11330/
3) http://review.whamcloud.com/#/c/11383/
4) http://review.whamcloud.com/#/c/11384/
5) http://review.whamcloud.com/#/c/11390/
6) http://review.whamcloud.com/#/c/11391/
7) http://review.whamcloud.com/#/c/11444/
8) http://review.whamcloud.com/#/c/11516/
9) http://review.whamcloud.com/#/c/11536/
10) http://review.whamcloud.com/#/c/11845/
11) http://review.whamcloud.com/#/c/11846/
12) http://review.whamcloud.com/#/c/11847/
13) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 30/Sep/14 ]

The total patches list for all LFSCK 3 related patches:
Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/
23) http://review.whamcloud.com/#/c/11486/
24) http://review.whamcloud.com/#/c/11330/

To be landed:
1) http://review.whamcloud.com/#/c/11383/
2) http://review.whamcloud.com/#/c/11384/
3) http://review.whamcloud.com/#/c/11390/
4) http://review.whamcloud.com/#/c/11391/
5) http://review.whamcloud.com/#/c/11444/
6) http://review.whamcloud.com/#/c/11516/
7) http://review.whamcloud.com/#/c/11536/
8) http://review.whamcloud.com/#/c/11845/
9) http://review.whamcloud.com/#/c/11846/
10) http://review.whamcloud.com/#/c/11847/
11) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 17/Oct/14 ]

The total patches list for all LFSCK 3 related patches:
Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/
23) http://review.whamcloud.com/#/c/11486/
24) http://review.whamcloud.com/#/c/11330/
25) http://review.whamcloud.com/#/c/12247/
26) http://review.whamcloud.com/#/c/12248/
27) http://review.whamcloud.com/#/c/12249/

To be landed:
1) http://review.whamcloud.com/#/c/11391/
2) http://review.whamcloud.com/#/c/11444/
3) http://review.whamcloud.com/#/c/11516/
4) http://review.whamcloud.com/#/c/11536/
5) http://review.whamcloud.com/#/c/11845/
6) http://review.whamcloud.com/#/c/11846/
7) http://review.whamcloud.com/#/c/11847/
8) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 22/Oct/14 ]

The total patches list for all LFSCK 3 related patches:
Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/
23) http://review.whamcloud.com/#/c/11486/
24) http://review.whamcloud.com/#/c/11330/
25) http://review.whamcloud.com/#/c/12247/
26) http://review.whamcloud.com/#/c/12248/
27) http://review.whamcloud.com/#/c/12249/
28) http://review.whamcloud.com/#/c/11391/
29) http://review.whamcloud.com/#/c/11444/

To be landed:
1) http://review.whamcloud.com/#/c/11516/
2) http://review.whamcloud.com/#/c/11536/
3) http://review.whamcloud.com/#/c/11845/
4) http://review.whamcloud.com/#/c/11846/
5) http://review.whamcloud.com/#/c/11847/
6) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 30/Oct/14 ]

The total patches list for all LFSCK 3 related patches:
Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/
23) http://review.whamcloud.com/#/c/11486/
24) http://review.whamcloud.com/#/c/11330/
25) http://review.whamcloud.com/#/c/12247/
26) http://review.whamcloud.com/#/c/12248/
27) http://review.whamcloud.com/#/c/12249/
28) http://review.whamcloud.com/#/c/11391/
29) http://review.whamcloud.com/#/c/11444/
30) http://review.whamcloud.com/#/c/11516/
31) http://review.whamcloud.com/#/c/11536/
32) http://review.whamcloud.com/#/c/11845/

To be landed:
1) http://review.whamcloud.com/#/c/11846/
2) http://review.whamcloud.com/#/c/11847/
3) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 31/Oct/14 ]

The total patches list for all LFSCK 3 related patches:
Landed:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/
7) http://review.whamcloud.com/#/c/11275/
8) http://review.whamcloud.com/#/c/11276/
9) http://review.whamcloud.com/#/c/11288/
10) http://review.whamcloud.com/#/c/11407/
11) http://review.whamcloud.com/#/c/11590/
12) http://review.whamcloud.com/#/c/10602/
13) http://review.whamcloud.com/#/c/10493/
14) http://review.whamcloud.com/#/c/11304/
15) http://review.whamcloud.com/#/c/11373/
16) http://review.whamcloud.com/#/c/10986/
17) http://review.whamcloud.com/#/c/11382/
18) http://review.whamcloud.com/#/c/11485/
19) http://review.whamcloud.com/#/c/10987/
20) http://review.whamcloud.com/#/c/10603/
21) http://review.whamcloud.com/#/c/10996/
22) http://review.whamcloud.com/#/c/11317/
23) http://review.whamcloud.com/#/c/11486/
24) http://review.whamcloud.com/#/c/11330/
25) http://review.whamcloud.com/#/c/12247/
26) http://review.whamcloud.com/#/c/12248/
27) http://review.whamcloud.com/#/c/12249/
28) http://review.whamcloud.com/#/c/11391/
29) http://review.whamcloud.com/#/c/11444/
30) http://review.whamcloud.com/#/c/11516/
31) http://review.whamcloud.com/#/c/11536/
32) http://review.whamcloud.com/#/c/11845/
33) http://review.whamcloud.com/#/c/11846/
34) http://review.whamcloud.com/#/c/11847/

To be landed:
1) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 31/Oct/14 ]

The total patches list for all LFSCK 3 related patches:

Landed to Lustre-2.6:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/

Landed to Lustre-2.7:
1) http://review.whamcloud.com/#/c/11275/
2) http://review.whamcloud.com/#/c/11276/
3) http://review.whamcloud.com/#/c/11288/
4) http://review.whamcloud.com/#/c/11407/
5) http://review.whamcloud.com/#/c/11590/
6) http://review.whamcloud.com/#/c/10602/
7) http://review.whamcloud.com/#/c/10493/
8) http://review.whamcloud.com/#/c/11304/
9) http://review.whamcloud.com/#/c/11373/
10) http://review.whamcloud.com/#/c/10986/
11) http://review.whamcloud.com/#/c/11382/
12) http://review.whamcloud.com/#/c/11485/
13) http://review.whamcloud.com/#/c/10987/
14) http://review.whamcloud.com/#/c/10603/
15) http://review.whamcloud.com/#/c/10996/
16) http://review.whamcloud.com/#/c/11317/
17) http://review.whamcloud.com/#/c/11486/
18) http://review.whamcloud.com/#/c/11330/
19) http://review.whamcloud.com/#/c/12247/
20) http://review.whamcloud.com/#/c/12248/
21) http://review.whamcloud.com/#/c/12249/
22) http://review.whamcloud.com/#/c/11391/
23) http://review.whamcloud.com/#/c/11444/
24) http://review.whamcloud.com/#/c/11516/
25) http://review.whamcloud.com/#/c/11536/
26) http://review.whamcloud.com/#/c/11845/
27) http://review.whamcloud.com/#/c/11846/
28) http://review.whamcloud.com/#/c/11847/
29) http://review.whamcloud.com/#/c/11848/

Comment by nasf (Inactive) [ 05/Nov/14 ]

All the patches have been landed to master.

Comment by Gerrit Updater [ 26/Feb/15 ]

Fan Yong (fan.yong@intel.com) uploaded a new patch: http://review.whamcloud.com/13897
Subject: LU-4788 lfsck: measure namespace LFSCK performance
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: be5eb5afda0571aed9b6c01d461be5f9922dc276

Generated at Sat Feb 10 01:45:51 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.