Implement OI Scrub for ZFS (LU-7585)

[LU-10193] Index object on-disk layout compatibility Created: 02/Nov/17  Updated: 18/Jun/21  Resolved: 06/Feb/18

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.11.0

Type: Technical task Priority: Minor
Reporter: nasf (Inactive) Assignee: nasf (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Related
is related to LU-12038 conf-sanity test 59 memory leaks Resolved
Rank (Obsolete): 9223372036854775807

 Description   

Lustre uses ZAP to implement index object for ZFS backend. When tar the index object via backend ZPL for backup, it is explained as regular file, then when untar it, it is not ZAP formatted again, then the Lustre cannot recognize the 'bad' formatted index object.

On the other hand, each backend FS has its own special format for index object. Then we cannot migrate the index files from one backend to another directly.

To resolve such issue, the patch will backup the index object with plain format to the local '/index_backup' directory with the name of soruce index's FID string and ".lbx" postfix when
umount the device.

The format of the backup is as following:
1) header: 512 bytes, including:
magic: 4 bytes
count: 4 bytes
keysize: 4 bytes
recsize: 4 bytes
owner_fid: 16 bytes
padding: 480 bytes

2) body: after the header, <key, rec> pairs one by one.

The backup behavior is controlled via new OSD lproc interface: index_backup. By default, it is off. You can turn it on to enable backup when server umount via writing non-zero value to such lproc interface.



 Comments   
Comment by Gerrit Updater [ 15/Nov/17 ]

Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/30106
Subject: LU-10193 tests: test migration between ldiskfs and zfs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6c9440bad4d1f3590555ca4f8dc3d8e50e85e7f8

Comment by nasf (Inactive) [ 15/Nov/17 ]

Another two patches:
https://review.whamcloud.com/#/c/28834/
https://review.whamcloud.com/#/c/28847/

Comment by Gerrit Updater [ 19/Nov/17 ]

Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/30167
Subject: LU-10193 tests: debug patch for ZFS OI scrub
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a5b3b17aa3947cd56377a4af0c213c72d0b1166a

Comment by Gerrit Updater [ 18/Jan/18 ]

Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/30910
Subject: LU-10193 osd-zfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4347fccb03046d09be9b5be92238f4ca1c0619e0

Comment by Gerrit Updater [ 18/Jan/18 ]

Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/30911
Subject: LU-10193 osd-ldiskfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 43a80c1dca566d3d7709090bef6f96baadd623c4

Comment by Gerrit Updater [ 06/Feb/18 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/30910/
Subject: LU-10193 osd-zfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6ff16309a006788cc91a7adf561aaebd76dd3b38

Comment by Gerrit Updater [ 06/Feb/18 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/30911/
Subject: LU-10193 osd-ldiskfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a48853cf3dc51b24cc276fcf5f2e3e25a32c0d25

Comment by Gerrit Updater [ 06/Feb/18 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/30106/
Subject: LU-10193 tests: test migration between ldiskfs and zfs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a6fb2bbe71efcb07265ced337d2792626e2d9bbe

Comment by Peter Jones [ 06/Feb/18 ]

Landed for 2.11

Generated at Sat Feb 10 02:32:53 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.