[LU-7585] Implement OI Scrub for ZFS Created: 20/Dec/15 Updated: 25/Nov/20 Resolved: 06/Feb/18 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.9.0 |
| Fix Version/s: | Lustre 2.11.0 |
| Type: | Improvement | Priority: | Critical |
| Reporter: | Andreas Dilger | Assignee: | nasf (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | lfsck | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||
| Sub-Tasks: |
|
||||||||||||||||||||||||||||||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||||||||||||||||||||||||||||||
| Description |
|
There at occasions where the OI file can become corrupted (e.g. double mount of a filesystem), or otherwise need to be rebuilt (e.g. upgrade of normal ZFS filesystem to Lustre MDT in the future). It would be useful to be able to verify and/or rebuild the ZFS OI files during dnode iteration. |
| Comments |
| Comment by Andreas Dilger [ 27/Apr/16 ] |
|
There have been a number of requests to be able to use tar or rsync or similar tools to do file level backup/restore for ZFS MDTs, for example upgrading the MDT from ldiskfs to ZFS. The only currently supported mechanism for ZFS backup/restore is zfs send and zfs recv which ties the data to always be in a ZFS filesystem. In addition to implementing ZFS Scrub to repair the OI files in case of corruption (for whatever reason), we should consider whether this would allow us to use tar to restore an ldiskfs MDT into a new ZFS filesystem and/or restore a ZFS tar backup? We would need to support IGIF FIDs for ZFS which I think is relatively low complexity, as well as the ability to rebuild FID-in-dirent which may be considerably more complex. With Data-on-MDT it might also be possible to take an existing regular ZFS (or ext4) filesystem and convert it to a Lustre MDT (mkdir ROOT/; mv "*" ROOT/; umount; mount -t lustre; run LFSCK), but this is not a primary goal at this point. |
| Comment by Gerrit Updater [ 19/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28607 |
| Comment by Gerrit Updater [ 19/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28608 |
| Comment by Gerrit Updater [ 19/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28609 |
| Comment by Gerrit Updater [ 19/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28610 |
| Comment by Gerrit Updater [ 24/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28678 |
| Comment by Gerrit Updater [ 24/Aug/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28679 |
| Comment by Gerrit Updater [ 01/Sep/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28833 |
| Comment by Gerrit Updater [ 01/Sep/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28834 |
| Comment by Gerrit Updater [ 04/Sep/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28847 |
| Comment by Gerrit Updater [ 05/Sep/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/28855 |
| Comment by Gerrit Updater [ 28/Sep/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/29241 |
| Comment by Gerrit Updater [ 14/Oct/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/29617 |
| Comment by Gerrit Updater [ 20/Oct/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/29696 |
| Comment by Gerrit Updater [ 23/Oct/17 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/29710 |
| Comment by nasf (Inactive) [ 15/Nov/17 ] |
|
All the patches for this ticket: |
| Comment by Gerrit Updater [ 17/Dec/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/28607/ |
| Comment by Gerrit Updater [ 17/Dec/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/29710/ |
| Comment by Gerrit Updater [ 18/Jan/18 ] |
|
Fan Yong (fan.yong@intel.com) uploaded a new patch: https://review.whamcloud.com/30909 |
| Comment by Gerrit Updater [ 31/Jan/18 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/30909/ |
| Comment by Peter Jones [ 31/Jan/18 ] |
|
Can this work now be marked as resolved? |
| Comment by nasf (Inactive) [ 31/Jan/18 ] |
|
No, there are still three patches to be landed: |
| Comment by Peter Jones [ 31/Jan/18 ] |
|
Isn't that distinct functionality tracked under a different JIRA ticket ( |
| Comment by nasf (Inactive) [ 31/Jan/18 ] |
|
|
| Comment by Peter Jones [ 31/Jan/18 ] |
|
Ah. Got it. Thanks |
| Comment by nasf (Inactive) [ 31/Jan/18 ] |
|
All the remaining three patches have been approved by reviewers, just waiting gate keeper for landing. |
| Comment by Peter Jones [ 31/Jan/18 ] |
|
so should https://review.whamcloud.com/#/c/28834/ and https://review.whamcloud.com/#/c/28847/ be abandoned? |
| Comment by nasf (Inactive) [ 31/Jan/18 ] |
|
Right, abandoned already. |
| Comment by Peter Jones [ 06/Feb/18 ] |
|
ok. So now everything is landed AFAICT |
| Comment by Gerrit Updater [ 25/Nov/20 ] |
|
|