[LU-6361] LFSCK 4: improve LFSCK performance - Whamcloud Community JIRA

Details

Type: Improvement
Resolution: Fixed
Priority: Major
Fix Version/s: Lustre 2.8.0
Affects Version/s: None
Labels:
None

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

It is used for tracing all LFSCK 4 tasks.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

LFSCK_4_test_results_201505.xlsx
62 kB
05/Feb/19 12:37 AM

Sub-Tasks

Progress

1.	LFSCK 4: Improve inode iteration for uninitialized groups	Resolved	nasf (Inactive)
2.	LFSCK 4: Improve OI scrub trigger strategy: do not trigger urgent OI scrub on the whole system if only few inconsistent OI entries are found	Resolved	nasf (Inactive)
3.	LFSCK 4: Record linkEA verification history in RAM	Resolved	nasf (Inactive)
4.	LFSCK 4: optimize ldlm lock used by LFSCK	Resolved	nasf (Inactive)
5.	LFSCK 4: namespace LFSCK can skip dot entry	Resolved	nasf (Inactive)
6.	LFSCK 4: namespace LFSCK counts the objects repeatedly	Resolved	nasf (Inactive)
7.	LFSCK 4: locate object only when necessary	Resolved	nasf (Inactive)
8.	LFSCK 4: lock the MDT-object based on prediction for preparing bad linkEA	Resolved	nasf (Inactive)
9.	LFSCK 4: show LFSCK started/completed time directly via LFSCK lproc interface	Resolved	nasf (Inactive)
10.	LFSCK 4: namespace LFSCK scalability	Resolved	nasf (Inactive)
11.	LFSCK MDS crash: unable to handle kernel NULL pointer dereference	Resolved	nasf (Inactive)

Activity

[LU-6361] LFSCK 4: improve LFSCK performance

Andreas Dilger added a comment - 05/Feb/19 12:39 AM

Attached are the test results comparing LFSCK phase 3 (DNE support) with LFSCK phase 4 (performance tuning), measured when this work was competed in 2015.

Andreas Dilger added a comment - 05/Feb/19 12:39 AM Attached are the test results comparing LFSCK phase 3 (DNE support) with LFSCK phase 4 (performance tuning), measured when this work was competed in 2015.

Andreas Dilger made changes - 05/Feb/19 12:37 AM

Attachment

New: LFSCK_4_test_results_201505.xlsx [ 31938 ]

John Bent (Inactive) added a comment - 04/Feb/19 11:59 PM

Hello, has anyone actually used this awesome sounding patch to measure performance? We have an RFP that wants 1B namespace entries to be fsck'd in 12 hours. Anyone have any ideas what a reasonable lfsck rate is? Thanks!

John Bent (Inactive) added a comment - 04/Feb/19 11:59 PM Hello, has anyone actually used this awesome sounding patch to measure performance? We have an RFP that wants 1B namespace entries to be fsck'd in 12 hours. Anyone have any ideas what a reasonable lfsck rate is? Thanks!

Peter Jones made changes - 30/May/15 4:27 AM

Resolution		New: Fixed [ 1 ]
Status	Original: In Progress [ 3 ]	New: Resolved [ 5 ]

Peter Jones added a comment - 30/May/15 4:27 AM

Landed for 2.8

Peter Jones added a comment - 30/May/15 4:27 AM Landed for 2.8

Gerrit Updater added a comment - 30/May/15 2:39 AM

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/12109/
Subject: ~~LU-6361~~ lfsck: measure LFSCK 4 performance
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: db2110a7af734f42a3a0b425feade8498c309895

Gerrit Updater added a comment - 30/May/15 2:39 AM Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/12109/ Subject: LU-6361 lfsck: measure LFSCK 4 performance Project: fs/lustre-release Branch: master Current Patch Set: Commit: db2110a7af734f42a3a0b425feade8498c309895

nasf (Inactive) added a comment - 01/May/15 4:12 AM

The patch for measure LFSCK 4 performance:
http://review.whamcloud.com/#/c/12109/

nasf (Inactive) added a comment - 01/May/15 4:12 AM The patch for measure LFSCK 4 performance: http://review.whamcloud.com/#/c/12109/

Jodi Levi (Inactive) made changes - 13/Mar/15 3:38 PM

Issue Type

Original: Bug [ 1 ]

New: Improvement [ 4 ]

nasf (Inactive) made changes - 13/Mar/15 12:01 AM

Status

Original: Open [ 1 ]

New: In Progress [ 3 ]

nasf (Inactive) created issue - 13/Mar/15 12:01 AM

People

Assignee:: nasf (Inactive)

Reporter:: nasf (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 13/Mar/15 12:01 AM

Updated:: 05/Feb/19 12:39 AM

Resolved:: 30/May/15 4:27 AM