[LU-10193] Index object on-disk layout compatibility

• Type: Technical task
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.11.0
• Assignee: nasf (Inactive)
• Reporter: nasf (Inactive)

Description

Lustre uses ZAP to implement index object for ZFS backend. When tar the index object via backend ZPL for backup, it is explained as regular file, then when untar it, it is not ZAP formatted again, then the Lustre cannot recognize the 'bad' formatted index object.

On the other hand, each backend FS has its own special format for index object. Then we cannot migrate the index files from one backend to another directly.

To resolve such issue, the patch will backup the index object with plain format to the local '/index_backup' directory with the name of soruce index's FID string and ".lbx" postfix when
umount the device.

The format of the backup is as following:
1) header: 512 bytes, including:
magic: 4 bytes
count: 4 bytes
keysize: 4 bytes
recsize: 4 bytes
owner_fid: 16 bytes
padding: 480 bytes

2) body: after the header, <key, rec> pairs one by one.

The backup behavior is controlled via new OSD lproc interface: index_backup. By default, it is off. You can turn it on to enable backup when server umount via writing non-zero value to such lproc interface.

Issue Links

• is related to: LU-12038 conf-sanity test 59 memory leaks

Comments

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/30106/
Subject: LU-10193 tests: test migration between ldiskfs and zfs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a6fb2bbe71efcb07265ced337d2792626e2d9bbe

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/30911/
Subject: LU-10193 osd-ldiskfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a48853cf3dc51b24cc276fcf5f2e3e25a32c0d25

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/30910/
Subject: LU-10193 osd-zfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6ff16309a006788cc91a7adf561aaebd76dd3b38

[Gerrit Updater]

Fan Yong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30911
Subject: LU-10193 osd-ldiskfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 43a80c1dca566d3d7709090bef6f96baadd623c4

[Gerrit Updater]

Fan Yong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30910
Subject: LU-10193 osd-zfs: backup index object with plain format
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4347fccb03046d09be9b5be92238f4ca1c0619e0

[Gerrit Updater]

Fan Yong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30167
Subject: LU-10193 tests: debug patch for ZFS OI scrub
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a5b3b17aa3947cd56377a4af0c213c72d0b1166a

[nasf (Inactive)]

Another two patches:
https://review.whamcloud.com/#/c/28834/
https://review.whamcloud.com/#/c/28847/