[LU-14058] Create tests for large number of MDTs

• Status: Reopened
• Project: Lustre
• Affects Version/s: Lustre 2.14.0
• Fix Version/s: Lustre 2.16.0
• Type: Improvement
• Priority: Minor
• Reporter: James Nunez (Inactive)
• Assignee: Sarah Liu
• Resolution: Unresolved
• Labels: failing_tests, tests

Description

Patch https://review.whamcloud.com/37759 "LU-12506 changelog: support large number of MDT" allows clients to mount file systems with large numbers of MDTs. We need to create a, probably conf-sanity, test(s) that creates file systems with 16 and 32 MDTs and confirms that a client can mount this file system. The basic mount is a required test. Other testing with a large number of MDTs should be listed in this ticket.

Issue Links

• is related to: LU-12506 Client unable to mount filesystem wit...

Comments

[Olaf Faaland]

According to the commit message, the patch fixes the case where MDTs > 64.  So It would be really good to have tests for 16 or 32 MDTs, but your mount test should have >64 MDTs.

[Oleg Drokin]

this probably makes sense to do as part of conf-sanity and since out VMs have relatively little memory we might not be able to run 32-64 MDTs on a single node so perhaps the test would need to spread out MDTs (off loopback mounted storage in tmp perhaps) across multiple nodes

[Alex Kulyavtsev]

confirm, you need > 64 MDT to be sure.

You can trip the max in /proc/misc with fewer MDTs as there are ~18 devices created by linux. This  number vary from system to system. Lustre creates two devices (obd,lnet) common for all lustre mounts in /proc/misc besides changelog devices changelog-<fsname>-MDT00NN (one per MDT).

Probably Nmdt > 64-(18+2), that is  Nmdt > 44 will be enough but you may want to use >64 to be sure.

Note,

• the total MDT count is not necessarily for the same FS, it is total limit on number of entries in /proc/misc . You can mount two FS with 40 MDT each on the same client.
• mounting the same fs multiple times on the same client does not trigger the error (creates extra entries in /proc/misc)

[Andreas Dilger]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/41485
Subject: LU-14058 tests: handle more MDTs in sanity.sh
Project: fs/lustre-release
Branch: master
Current Patch Set: 3
Commit: 5d146e9452e3d6875fd674cd168595557f331ed5

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/41485/
Subject: LU-14058 tests: handle more MDTs in sanity.sh
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 173bccd140adf69ce08c20810a69e783c8c12595

[Peter Jones]

Landed for 2.15

[Andreas Dilger]

There is still an unlanded patch for this ticket (it was originally pushed under LU-12506 so didn't show up here):

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38058
Subject: LU-14058 tests: clean up MDT name generation
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2f8cc17c0f13cb805e4905d8e10f8d4536f0efce

[Gerrit Updater]

"Wei Liu <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/46268
Subject: LU-14058 tests: run conf-sanity with 16 MDTs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: bed894f273c56b743188a9ae7d1a259072f99026