[LU-18162] Improve lu_device/obd_device handling and Lustre mount state machine

• Type: Task
• Resolution: Unresolved
• Priority: Minor
• Severity: 3
• Assignee: Tim Day
• Reporter: Tim Day

Description

Lustre device handling is split between the legacy OBD infrastructure and the newer LU device infrastructure. Each subsystem in Lustre is a strange mix of these different paradigms. With very little standardization across Lustre, it's difficult to understand how each component of Lustre is started and stopped during mount and umounts. As a result, the mount state machine is difficult to debug and error prone. Further, the process of writing extensions to Lustre (notably OSDs) is frustrated by the lack of clear and consistent documentation and mechanism around create Lustre devices.

 

In no particular order, I've documented a number of tasks that can meaningfully improve Lustre's device handling:

• The methods `>o_process_config` and `>ldo_process_config` need to be deprecated in favor of proper generics. Explicit callers of these functions need to avoid crafting their own `llog` to run operations on devices.
• `lu_device` needs to have a proper `kref`.
• `lu_device_init()` and `lu_device_fini()` should be consolidated. The device init should probably happen immediately after `ldto_device_alloc()` and the device fini immediately before `ldto_device_free()`
• OBD and other child devices need to align their life cycles to LU. `>o_setup` and `>o_cleanup` should be removed in favor of `lu_device` methods. The child devices should be allocated and free'd alongside their parent `lu_device`.
• `class_` and `obd_` methods need to be de-duplicated (or at least reviewed).
• Userspace setup/cleanup should be disabled. It isn’t safe, often tested, or useful. Often times, manipulating the devices from userspace can be dangerous.
• `lu_site` should probably be tied to the actual Lustre super-block.
• OBD health check could be more useful (i.e. return the reason for the device failing).
• The client and server-side mount paths should be split entirely. The `lustre` and `lustre_tgt` difference could be handled in userspace.
• Fix the `lockdep` issues during the Lustre mount
• Fix naming. I'm pretty sure bottom device for lu_site is always an OSD - never anything else.

Issue Links

• is related to: LU-16763 Improve Lustre kunit testing and OBD life cycle testing
• is related to: LU-17995 In-memory OSD device
• is related to: LU-17848 OSD and dt cleanups / deduplication
• is related to: LU-8802 Dynamically allocate obd_devices.

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58824/
Subject: LU-18162 oss: convert Object Storage Service to LU device
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e444862a62f81696cf6ede14c6f373d01e27b834

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58810/
Subject: LU-18162 llite: consolidate cl_stack_fini()/lu_stack_fini()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 16ffcf7f06ae04ebcfe2debabebe36265cb44dae

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58804/
Subject: LU-18162 kunit: convert LDLM extent test to LU device
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 224ecf9db7fce9424adb2ad1a6cf83415774f456

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58785/
Subject: LU-18162 kunit: convert OBD test to LU device
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d253156f43caed3438e3b42656766c02561e2b2c

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58784/
Subject: LU-18162 obdclass: init LU device after alloc
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 973b5f1fed3d69b29ef5dcaa4fc283b07e85d13d

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58802/
Subject: LU-18162 obdecho: remove LBUG() from echo_device_init()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8ec47c389dfdaf323d4ab558432cf3c0d10b00aa

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58787/
Subject: LU-18162 osd-ldiskfs: leave osd_device_init() unimplemented
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5b6a76e282ab149a53802d47981c7e3e890be138

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58783/
Subject: LU-18162 obdclass: create ldto wrappers and default LU allocator
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d04233bbdcb04780b10cae9de5e4d78c23cfca73

[Gerrit Updater]

"Timothy Day <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58831
Subject: LU-18162 obdclass: remove o_cleanup/setup and friends
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3bc8443596c2c417fe44c9b7f8272fd02568a777

[Gerrit Updater]

"Timothy Day <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58830
Subject: LU-18162 ofd: convert Object Filter Device to LU device
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 675df0cac800313ec9db195b64e85f48ab12fd40

[Gerrit Updater]

"Timothy Day <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58829
Subject: LU-18162 lmv: convert Logical Metadata Volume to LU device
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5da01918a5244fb3bd43a30ec9a7999ef52d19a6