[LU-11191] Striped DIR accross 2 MDTs very slow

• Type: Bug
• Resolution: Duplicate
• Priority: Major
• Affects Version/s: Lustre 2.10.3
• Environment: Client: sles12 SP3 Lustre2.10.3 ; servers: cent7.4 Lustre 2.10.3
• Severity: 3
• Assignee: Andreas Dilger
• Reporter: Mahmoud Hanafi

Description

May be dup of LU-8118

We have 2 MDSs each with 1 MDT. Running mdtest  on mdt0 or mdt1 directory is fast but a striped (using -D) dir across both is very slow.

 $ lfs getdirstripe ../MDT0
lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none

$ lfs getdirstripe ../MDT0_and_1/
lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: fnv_1a_64
mdtidx		 FID[seq:oid:ver]
     0		 [0x200002340:0x3:0x0]		
     1		 [0x240002340:0x3:0x0]		

STRIPE DIR

-- started at 07/29/2018 22:47:01 --

mdtest-1.9.3 was launched with 8 total task(s) on 8 node(s)
Command line used: /home/mhanafi/bin/mdtest -z 4 -b 3 -n 500 -i 5 -u -d MDT0_and_1/
Path: /nobackuptest/mhanafi/MDTESTS/MDT0_and_1
FS: 3250.9 TiB   Used FS: 0.3%   Inodes: 102.4 Mi   Used Inodes: 0.0%

8 tasks, 3872 files/directories

SUMMARY: (of 5 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   Directory creation:      11555.863      11347.736      11457.891         85.542
   Directory stat    :      27105.495      25926.197      26639.416        389.884
   Directory removal :       1101.478        932.399        986.495         65.291
   File creation     :      26333.527      22289.699      25075.189       1451.802
   File stat         :      32253.499      28994.953      30613.853       1247.751
   File read         :      41577.227      37023.290      39056.222       1766.014
   File removal      :      50925.204      46150.431      47627.634       1694.602
   Tree creation     :       1215.416       1194.374       1205.364          8.445
   Tree removal      :        142.877        101.763        130.532         15.266

-- finished at 07/29/2018 22:47:31 --

SINGLE STRIPE DIR

-- started at 07/29/2018 22:47:37 --

mdtest-1.9.3 was launched with 8 total task(s) on 8 node(s)
Command line used: /home/mhanafi/bin/mdtest -z 4 -b 3 -n 500 -i 5 -u -d MDT0/
Path: /nobackuptest/mhanafi/MDTESTS/MDT0
FS: 3250.9 TiB   Used FS: 0.3%   Inodes: 102.4 Mi   Used Inodes: 0.0%

8 tasks, 3872 files/directories

SUMMARY: (of 5 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   Directory creation:      57733.445      50424.700      53351.848       2550.582
   Directory stat    :      79237.751      68950.262      72220.457       3684.412
   Directory removal :      60426.946      56294.687      57780.897       1518.306
   File creation     :      29322.719      28476.241      28922.159        336.506
   File stat         :      32544.144      29101.692      31061.575       1346.628
   File read         :      39504.498      36896.728      38229.842       1128.910
   File removal      :      50177.351      43709.006      46399.363       2125.337
   Tree creation     :       6880.380       5797.328       6223.052        392.540
   Tree removal      :       8109.055       7364.743       7683.243        243.272

-- finished at 07/29/2018 22:47:40 --

Issue Links

• duplicates: LU-4684 DNE3: allow migrating DNE striped directory