[LU-4788] LFSCK 3: MDT-MDT consistency verification - Whamcloud Community JIRA

Details

Type: New Feature
Resolution: Fixed
Priority: Critical
Fix Version/s: Lustre 2.7.0
Affects Version/s: None
Labels:
- LFSCK

Epic:
- LFSCK
Project:
OpenSFS SFS-DEV-001
Rank (Obsolete):
13167

Description

In LFSCK 3, we will not only extend the FID-in-dirent and linkEA verification from single MDT to cross-MDTs cases, but also some additional inconsistent check/repair for remote object and striped directory, includes:
1) Dangling referenced name entry: The name entry exists, but related MDT-object does not exist.
2) Orphan MDT-object: The MDT-object exists, but there is no name entry to reference it.
3) Multiple-referenced name entry: More than one MDT-objects point back to the same name entry, but the name entry only references one of them.
4) Unmatched name entry and MDT-object pairs: The name entry references the MDT-object which has no linkEA for back-reference or it points back to another name entry that does not exists or does not reference the MDT-object.
5) Redundant or invalid linkEA entries: Repeated linkEA entries, or bad parent FID, or the parent FID is not for MDT-object, or not allocated yet.
6) Invalid nlink count: The MDT-object's nlink count does not matches the name entries (which reference such MDT-object) count.

Attachments

Sub-Tasks

Progress

1.	LFSCK 3 Test Plan	Resolved	nasf (Inactive)
2.	LFSCK 3: Skip orphan objects handling for failed servers	Resolved	nasf (Inactive)
3.	LFSCK 3: RPC adjustment for remote transaction	Resolved	nasf (Inactive)
4.	LFSCK 3: get PFID from linkEA for remote directory on ldiskfs	Resolved	nasf (Inactive)
5.	LFSCK 3: repair unmatched distributed parent-child pairs	Resolved	nasf (Inactive)
6.	LFSCK 3: repair dangling name entry	Resolved	nasf (Inactive)
7.	LFSCK 3: repair multiple referenced name entry	Resolved	nasf (Inactive)
8.	LFSCK 3: repair bad file type in name entry	Resolved	nasf (Inactive)
9.	LFSCK 3: repair the lost name entry	Resolved	nasf (Inactive)
10.	LFSCK 3: repair invalid nlink count	Resolved	nasf (Inactive)
11.	LFSCK 3: recover client visible objects from the backend /lost+found directory	Resolved	nasf (Inactive)
12.	LFSCK 3: verify striped directory	Resolved	nasf (Inactive)

Activity

[LU-4788] LFSCK 3: MDT-MDT consistency verification

Gerrit Updater added a comment - 26/Feb/15 12:37 PM

Fan Yong (fan.yong@intel.com) uploaded a new patch: http://review.whamcloud.com/13897
Subject: ~~LU-4788~~ lfsck: measure namespace LFSCK performance
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: be5eb5afda0571aed9b6c01d461be5f9922dc276

Gerrit Updater added a comment - 26/Feb/15 12:37 PM Fan Yong (fan.yong@intel.com) uploaded a new patch: http://review.whamcloud.com/13897 Subject: LU-4788 lfsck: measure namespace LFSCK performance Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: be5eb5afda0571aed9b6c01d461be5f9922dc276

nasf (Inactive) added a comment - 05/Nov/14 4:49 AM

All the patches have been landed to master.

nasf (Inactive) added a comment - 05/Nov/14 4:49 AM All the patches have been landed to master.

nasf (Inactive) added a comment - 31/Oct/14 11:46 PM

The total patches list for all LFSCK 3 related patches:

Landed to Lustre-2.6:
1) http://review.whamcloud.com/#/c/10030/
2) http://review.whamcloud.com/#/c/10143/
3) http://review.whamcloud.com/#/c/10447/
4) http://review.whamcloud.com/#/c/10733/
5) http://review.whamcloud.com/#/c/10751/
6) http://review.whamcloud.com/#/c/10765/

nasf (Inactive) added a comment - 31/Oct/14 11:46 PM The total patches list for all LFSCK 3 related patches: Landed to Lustre-2.6: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ Landed to Lustre-2.7: 1) http://review.whamcloud.com/#/c/11275/ 2) http://review.whamcloud.com/#/c/11276/ 3) http://review.whamcloud.com/#/c/11288/ 4) http://review.whamcloud.com/#/c/11407/ 5) http://review.whamcloud.com/#/c/11590/ 6) http://review.whamcloud.com/#/c/10602/ 7) http://review.whamcloud.com/#/c/10493/ 8) http://review.whamcloud.com/#/c/11304/ 9) http://review.whamcloud.com/#/c/11373/ 10) http://review.whamcloud.com/#/c/10986/ 11) http://review.whamcloud.com/#/c/11382/ 12) http://review.whamcloud.com/#/c/11485/ 13) http://review.whamcloud.com/#/c/10987/ 14) http://review.whamcloud.com/#/c/10603/ 15) http://review.whamcloud.com/#/c/10996/ 16) http://review.whamcloud.com/#/c/11317/ 17) http://review.whamcloud.com/#/c/11486/ 18) http://review.whamcloud.com/#/c/11330/ 19) http://review.whamcloud.com/#/c/12247/ 20) http://review.whamcloud.com/#/c/12248/ 21) http://review.whamcloud.com/#/c/12249/ 22) http://review.whamcloud.com/#/c/11391/ 23) http://review.whamcloud.com/#/c/11444/ 24) http://review.whamcloud.com/#/c/11516/ 25) http://review.whamcloud.com/#/c/11536/ 26) http://review.whamcloud.com/#/c/11845/ 27) http://review.whamcloud.com/#/c/11846/ 28) http://review.whamcloud.com/#/c/11847/ 29) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 31/Oct/14 2:32 AM

To be landed:
1) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 31/Oct/14 2:32 AM The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ 23) http://review.whamcloud.com/#/c/11486/ 24) http://review.whamcloud.com/#/c/11330/ 25) http://review.whamcloud.com/#/c/12247/ 26) http://review.whamcloud.com/#/c/12248/ 27) http://review.whamcloud.com/#/c/12249/ 28) http://review.whamcloud.com/#/c/11391/ 29) http://review.whamcloud.com/#/c/11444/ 30) http://review.whamcloud.com/#/c/11516/ 31) http://review.whamcloud.com/#/c/11536/ 32) http://review.whamcloud.com/#/c/11845/ 33) http://review.whamcloud.com/#/c/11846/ 34) http://review.whamcloud.com/#/c/11847/ To be landed: 1) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 30/Oct/14 2:18 AM

To be landed:
1) http://review.whamcloud.com/#/c/11846/
2) http://review.whamcloud.com/#/c/11847/
3) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 30/Oct/14 2:18 AM The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ 23) http://review.whamcloud.com/#/c/11486/ 24) http://review.whamcloud.com/#/c/11330/ 25) http://review.whamcloud.com/#/c/12247/ 26) http://review.whamcloud.com/#/c/12248/ 27) http://review.whamcloud.com/#/c/12249/ 28) http://review.whamcloud.com/#/c/11391/ 29) http://review.whamcloud.com/#/c/11444/ 30) http://review.whamcloud.com/#/c/11516/ 31) http://review.whamcloud.com/#/c/11536/ 32) http://review.whamcloud.com/#/c/11845/ To be landed: 1) http://review.whamcloud.com/#/c/11846/ 2) http://review.whamcloud.com/#/c/11847/ 3) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 22/Oct/14 5:06 AM

To be landed:
1) http://review.whamcloud.com/#/c/11516/
2) http://review.whamcloud.com/#/c/11536/
3) http://review.whamcloud.com/#/c/11845/
4) http://review.whamcloud.com/#/c/11846/
5) http://review.whamcloud.com/#/c/11847/
6) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 22/Oct/14 5:06 AM The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ 23) http://review.whamcloud.com/#/c/11486/ 24) http://review.whamcloud.com/#/c/11330/ 25) http://review.whamcloud.com/#/c/12247/ 26) http://review.whamcloud.com/#/c/12248/ 27) http://review.whamcloud.com/#/c/12249/ 28) http://review.whamcloud.com/#/c/11391/ 29) http://review.whamcloud.com/#/c/11444/ To be landed: 1) http://review.whamcloud.com/#/c/11516/ 2) http://review.whamcloud.com/#/c/11536/ 3) http://review.whamcloud.com/#/c/11845/ 4) http://review.whamcloud.com/#/c/11846/ 5) http://review.whamcloud.com/#/c/11847/ 6) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 17/Oct/14 12:11 AM

To be landed:
1) http://review.whamcloud.com/#/c/11391/
2) http://review.whamcloud.com/#/c/11444/
3) http://review.whamcloud.com/#/c/11516/
4) http://review.whamcloud.com/#/c/11536/
5) http://review.whamcloud.com/#/c/11845/
6) http://review.whamcloud.com/#/c/11846/
7) http://review.whamcloud.com/#/c/11847/
8) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 17/Oct/14 12:11 AM The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ 23) http://review.whamcloud.com/#/c/11486/ 24) http://review.whamcloud.com/#/c/11330/ 25) http://review.whamcloud.com/#/c/12247/ 26) http://review.whamcloud.com/#/c/12248/ 27) http://review.whamcloud.com/#/c/12249/ To be landed: 1) http://review.whamcloud.com/#/c/11391/ 2) http://review.whamcloud.com/#/c/11444/ 3) http://review.whamcloud.com/#/c/11516/ 4) http://review.whamcloud.com/#/c/11536/ 5) http://review.whamcloud.com/#/c/11845/ 6) http://review.whamcloud.com/#/c/11846/ 7) http://review.whamcloud.com/#/c/11847/ 8) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 30/Sep/14 5:16 AM

To be landed:
1) http://review.whamcloud.com/#/c/11383/
2) http://review.whamcloud.com/#/c/11384/
3) http://review.whamcloud.com/#/c/11390/
4) http://review.whamcloud.com/#/c/11391/
5) http://review.whamcloud.com/#/c/11444/
6) http://review.whamcloud.com/#/c/11516/
7) http://review.whamcloud.com/#/c/11536/
8) http://review.whamcloud.com/#/c/11845/
9) http://review.whamcloud.com/#/c/11846/
10) http://review.whamcloud.com/#/c/11847/
11) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 30/Sep/14 5:16 AM The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ 23) http://review.whamcloud.com/#/c/11486/ 24) http://review.whamcloud.com/#/c/11330/ To be landed: 1) http://review.whamcloud.com/#/c/11383/ 2) http://review.whamcloud.com/#/c/11384/ 3) http://review.whamcloud.com/#/c/11390/ 4) http://review.whamcloud.com/#/c/11391/ 5) http://review.whamcloud.com/#/c/11444/ 6) http://review.whamcloud.com/#/c/11516/ 7) http://review.whamcloud.com/#/c/11536/ 8) http://review.whamcloud.com/#/c/11845/ 9) http://review.whamcloud.com/#/c/11846/ 10) http://review.whamcloud.com/#/c/11847/ 11) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 22/Aug/14 2:04 AM - edited

The total patches list for all LFSCK 3 related patches:

To be landed:

nasf (Inactive) added a comment - 22/Aug/14 2:04 AM - edited The total patches list for all LFSCK 3 related patches: Landed: 1) http://review.whamcloud.com/#/c/10030/ 2) http://review.whamcloud.com/#/c/10143/ 3) http://review.whamcloud.com/#/c/10447/ 4) http://review.whamcloud.com/#/c/10733/ 5) http://review.whamcloud.com/#/c/10751/ 6) http://review.whamcloud.com/#/c/10765/ 7) http://review.whamcloud.com/#/c/11275/ 8) http://review.whamcloud.com/#/c/11276/ 9) http://review.whamcloud.com/#/c/11288/ 10) http://review.whamcloud.com/#/c/11407/ 11) http://review.whamcloud.com/#/c/11590/ 12) http://review.whamcloud.com/#/c/10602/ 13) http://review.whamcloud.com/#/c/10493/ 14) http://review.whamcloud.com/#/c/11304/ 15) http://review.whamcloud.com/#/c/11373/ 16) http://review.whamcloud.com/#/c/10986/ 17) http://review.whamcloud.com/#/c/11382/ 18) http://review.whamcloud.com/#/c/11485/ 19) http://review.whamcloud.com/#/c/10987/ 20) http://review.whamcloud.com/#/c/10603/ 21) http://review.whamcloud.com/#/c/10996/ 22) http://review.whamcloud.com/#/c/11317/ To be landed: 1) http://review.whamcloud.com/#/c/11486/ 2) http://review.whamcloud.com/#/c/11330/ 3) http://review.whamcloud.com/#/c/11383/ 4) http://review.whamcloud.com/#/c/11384/ 5) http://review.whamcloud.com/#/c/11390/ 6) http://review.whamcloud.com/#/c/11391/ 7) http://review.whamcloud.com/#/c/11444/ 8) http://review.whamcloud.com/#/c/11516/ 9) http://review.whamcloud.com/#/c/11536/ 10) http://review.whamcloud.com/#/c/11845/ 11) http://review.whamcloud.com/#/c/11846/ 12) http://review.whamcloud.com/#/c/11847/ 13) http://review.whamcloud.com/#/c/11848/

nasf (Inactive) added a comment - 20/Aug/14 7:37 AM

The LFSCK 3 patches for LFSCK framework adjustment to be landed to master:
1) http://review.whamcloud.com/#/c/10986/
2) http://review.whamcloud.com/#/c/10987/
3) http://review.whamcloud.com/#/c/10602/
4) http://review.whamcloud.com/#/c/10493/
5) http://review.whamcloud.com/#/c/10603/
6) http://review.whamcloud.com/#/c/11317/

nasf (Inactive) added a comment - 20/Aug/14 7:37 AM The LFSCK 3 patches for LFSCK framework adjustment to be landed to master: 1) http://review.whamcloud.com/#/c/10986/ 2) http://review.whamcloud.com/#/c/10987/ 3) http://review.whamcloud.com/#/c/10602/ 4) http://review.whamcloud.com/#/c/10493/ 5) http://review.whamcloud.com/#/c/10603/ 6) http://review.whamcloud.com/#/c/11317/

People

Assignee:: nasf (Inactive)

Reporter:: nasf (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 8 Start watching this issue

Dates

Created:: 20/Mar/14 2:11 PM

Updated:: 26/Feb/15 12:37 PM

Resolved:: 05/Nov/14 4:49 AM