[LU-10360] use Imperative Recovery logs for client->MDT/OST connections

• Type: New Feature
• Resolution: Unresolved
• Priority: Minor
• Assignee: Mikhail Pershin
• Reporter: Andreas Dilger

Description

The Imperative Recovery (IR) feature landed in LU-19 created a dynamic list of active server NIDs on the MGS for purposes of speeding up client recovery when a target failed over to another server node. A server failure triggered a notification from the MGS to the client to update its target NIDs to reconnect to the recovered server more quickly.

It would be possible to extend this mechanism to also use the MGS IR log to do initial client mount, so that the MGS did not need to store the OST/MDT NIDs statically in the config log, but rather get the current NIDs directly from the dynamic MGS log. This would facilitate Lustre running in configurations where the server NIDs are not static (e.g. cloud, DHCP, etc). The initial connection to the MGS node(s) can already be done using the MGS hostname, since mount.lustre will do DNS name resolution.

Some care would be needed when OSTs are being registered with the MGS, especially in testing environments where OSTs are reformatted regularly and often use the same fsname, since this may allow OSTs to register with the MGS that do not actually belong to the same filesystem.

Issue Links

• is duplicated by: LU-18164 MGS should dynamically track server NIDs
• is related to: LU-14090 lctl replace_nids and starting target with local copy of logs
• is related to: LU-18280 Advertise dynamically configured server NIDs to MGS
• is related to: LU-18417 Finish IPv6 support
• is related to: LU-10359 remove NIDs from config llogs
• is related to: LU-16086 add generic LNet network number support
• is related to: LU-16722 MGS config log restructuring
• is related to: LU-18819 allow direct IB GUID addressing without IPoIB
• is related to: LU-18975 Nodemap-specific tunable parameter configuration
• is related to: LU-13306 allow clients to accept mgs_nidtbl_entry with IPv6 NIDs
• is related to: LU-11077 Client-specific tunable parameter configuration
• is related to: LU-13340 add LCFG_ADD_UUIDv6 and related commands
• is related to: LU-14668 LNet: do discovery in the background
• is related to: LU-14608 Adding second network to filesystem
• is related to: LU-19 imperative recovery
• is related to: LU-10391 LNET: Support IPv6
• is related to: LU-5881 Allow hostnames in NID
• links to: RED-1552 depends on LU-10360

Comments

[Gerrit Updater]

"Mikhail Pershin <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58595
Subject: LU-10360 mgc: client to mount without NIDs in config
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5434b0582c2ffb46de2883e2fd882d89dc9f79bd

[Andreas Dilger]

It seems possible to transition systems to using dynamic NIDs by putting "lctl set_param mgc.*.dynamic_nids=1" as one of the first records in the config llog, so that clients will allow IR to determine where the MDTs and OSTs are located. After that is done, it would just be a matter of how long to create config llog records with the NIDs in them for backward compatibility. The original patch landed as v2_13_55-106-g37be05eca3, so it is in all 2.14.0 and later releases.

[Andreas Dilger]

I think in conjunction with LU-10359 it should be possible to test a configuration that doesn't have server NIDs in the configuration at all, or totally incorrect NIDs in the config llog, to confirm that this is working properly.

A conf-sanity test case should be added to confirm that this is working properly.

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50000/
Subject: LU-10360 ldlm: remove client_import_find_conn()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 14544bdca5cc42a3ea80fe665e332fe4c88b081a

[Gerrit Updater]

"Neil Brown <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50000
Subject: LU-10360 ldlm: remove client_import_find_conn()
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 12cbeaf1fb7bc83d7a842b71d6e8a33601e085ce

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/39911/
Subject: LU-10360 tests: test dynamic NIDs feature
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2553f2fc8630061a8b6dbc5504d3f5277cb1cecf

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/45905
Subject: LU-10360 mgc: Use IR for client->MDS/OST connections
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: b1c09656513f3198adf849182617e6eafef76954

[Gerrit Updater]

Amir Shehata ([e-mail]) uploaded a new patch: https://review.whamcloud.com/40736
Subject: LU-10360 mgs: Mount to dynamically added networks
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4c68340088f2f56d16f6b1392de5ad7f7d139ff4

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39613/
Subject: LU-10360 mgc: Use IR for client->MDS/OST connections
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 37be05eca3f4aee15c946656a77f56967c15253d

[Gerrit Updater]

Amir Shehata ([e-mail]) uploaded a new patch: https://review.whamcloud.com/39911
Subject: LU-10360 tests: test dynamic NIDs feature
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d0bfbcb3bb643ce6dc33590bd937cb3c935ac88a