[LU-10825] Configuring multi-rail with a large number of nodes

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.12.0, Lustre 2.10.5
• Affects Version/s: Lustre 2.10.3
• Assignee: Amir Shehata (Inactive)
• Reporter: Taizeng Wu

Description

Recently, we prepare to deployment a lustre with multi-rail, but i don't known how to enable dynamic discovery.

We use lustre-2.10.3, it seems dynamic discovery is implementd in version 2.11.

We have about 2 mgs/mds, 6 oss and 512 client nodes, how to configure static multi-rail with a large number of nodes ?

Comments

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/32250/
Subject: LU-10825 lnet: add ip2nets syntax handling for peer
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: e124f39b6b4dd56780ba4490b81dca32ab08575c

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/32249/
Subject: LU-10825 libcfs: generate ip addresses
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: a48dc3fd0f738b545571a6d2cfdeb337f2d3243b

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32250
Subject: LU-10825 lnet: add ip2nets syntax handling for peer
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: fc78b0ee95a2ee85121e84e5d104f5c268aae26f

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32249
Subject: LU-10825 libcfs: generate ip addresses
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: a22011c6d2cd804413b5f7e8353b687fb742a495

[Peter Jones]

Landed for 2.12

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31786/
Subject: LU-10825 lnet: add ip2nets syntax handling for peer
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 70c95457f6836a9c0a9e95ae0c4bdd20f99a8747

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31785/
Subject: LU-10825 libcfs: generate ip addresses
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4c5f788397213aa41356df1f96f7ade58653973a

[Gerrit Updater]

Amir Shehata ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31786
Subject: LU-10825 lnet: add ip2nets syntax handling for peer
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3edcd387af4028e93f5d4df9caed9a36539ccbf6

[Gerrit Updater]

Amir Shehata ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31785
Subject: LU-10825 libcfs: generate ip addresses
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: eb88bfc56451ad790445daf1d5be303915b596b9

[Amir Shehata (Inactive)]

I'm currently working on a patch to make it easier to configure large systems without Dynamic Discovery.

But for now you'll need to configure the servers to know about the client's interfaces and you'll need to configure the clients to know about the server's interfaces. And since the OSS/MGS communicate you'll need to configure these to know about each other's interfaces.

MR doesn't handle interface down cases. If you intentionally (or unintentionally) bring down an interface, it will interfere with the file system operations as you've seen. We're currently working on a feature, LNet Health, which will be able to handle this particular interface failures.