Details
-
Bug
-
Resolution: Fixed
-
Major
-
None
-
None
-
b2_10
-
3
-
9223372036854775807
Description
There is a MDS and two MDTs. both MDT's hardware setup is symmetric.
This is DNE1 setup and static MDT allocation to each directory below.
[root@c01 ~]# lfs mkdir -i 0 /scratch0/dir0 [root@c01 ~]# lfs mkdir -i 1 /scratch0/dir1
If it run mdtest to each MDT separately, File creation to slave MDT (MDT0001) is much slower than primary MDT (MDT0000).
Here is quick summary
1. MDT0000 on MDT0 : 154K ops/sec
2. MDT0001 on MDT1 : 94K ops/sec
Also tested MDT1 device as MDT0000. reformated MDT1 device as MDT0000 and also reformated MDT0 device as MDT0001. (which means swapped MDT0 and MDT1 device)
3. MDT0000 on original MDT1 devcide : 151K ops/sec
4. MDT0001 on original MDT0 devcide : 106K ops/sec
From those benchmark results, MDT device and backend storage are no problem and it doesn't master. In any case, file creation to MDT0001 is slower than MDT0000.
Here is full mutest results.
mpirun -np 128 /work/tools/bin/mdtest -n 10000 -v -d /scratch0/dir1 -i 3 -p 60 -F -u
Format MDT0 device with MDT0000 SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 155589.552 152790.159 154009.399 1170.995 File stat : 454932.894 444775.351 449009.216 4315.516 File read : 233628.858 230038.744 232081.775 1507.029 File removal : 188460.588 184435.235 186712.008 1685.251 Tree creation : 551.714 444.141 493.856 44.292 Tree removal : 19.593 18.601 18.984 0.436 V-1: Entering timestamp... Format MDT1 device with MDT0001 SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 97428.133 92734.086 94657.494 2007.797 File stat : 463844.746 439627.133 450037.946 10174.240 File read : 234910.249 232565.024 233533.717 999.923 File removal : 186289.259 181171.423 184208.010 2195.839 Tree creation : 476.266 32.866 325.249 206.784 Tree removal : 19.429 14.144 17.055 2.191 V-1: Entering timestamp... Reformat MDT1 device as MDT0000 SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 155432.973 145656.215 151697.151 4311.335 File stat : 436363.906 420914.320 428509.377 6309.935 File read : 231848.424 229879.273 230823.486 805.926 File removal : 189856.501 186441.697 187710.599 1525.794 Tree creation : 564.044 432.872 504.217 54.166 Tree removal : 18.839 17.053 17.802 0.757 V-1: Entering timestamp... Reformat MDT0 device as MDT0001 SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 110312.440 103512.106 106042.905 3036.285 File stat : 443284.493 425246.521 435923.695 7728.341 File read : 226239.692 225898.388 226067.629 139.351 File removal : 186702.519 181944.612 184773.293 2043.883 Tree creation : 533.233 28.863 342.123 223.290 Tree removal : 17.901 17.260 17.650 0.280 V-1: Entering timestamp...
