[LU-2043] sgpdd survey enhancements

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.4.0
• Type: Improvement
• Priority: Minor
• Reporter: Gregoire Pichon
• Assignee: Cliff White (Inactive)
• Resolution: Fixed
• Epic: sgpdd

Description

I would like to submit some patches related to sgpdd survey.

The main one is to enhance sgpdd-survey to support multiple servers sharing a single storage array.

Also some ideas, I would like to discuss with other users concerned:

• replace the use of raw device (which is deprecated) by the use of O_DIRECT option
• allow optional placement of sgp_dd commands according to accessed device's NUMA node

Comments

[Gregoire Pichon]

I have posted the patch for multiple servers support.
http://review.whamcloud.com/#change,4122

[Peter Jones]

Thanks for the patch Gergoire. Cliff could you please review this change?

[Gregoire Pichon]

As requested by Minh Diep, here is the output of the sgpdd-survey (patch set 3).

scsidevs="mo88:/dev/sdc mo88:/dev/sdbk mo88:/dev/sdaa mo88:/dev/sdcg mo88:/dev/sdcm mo88:/dev/sdak mo88:/dev/sdbg mo88:/dev/sddo mo88:/dev/sdg mo88:/dev/sdbo mo88:/dev/sdbc mo88:/dev/sddk mo88:/dev/sdae mo88:/dev/sdcq mo88:/dev/sdai mo89:/dev/sde mo89:/dev/sdbm mo89:/dev/sdac mo89:/dev/sdci mo89:/dev/sdco mo89:/dev/sdam mo89:/dev/sdbi mo89:/dev/sddq mo89:/dev/sdi mo89:/dev/sdbq mo89:/dev/sdbe mo89:/dev/sddm mo89:/dev/sdag mo89:/dev/sdcs mo89:/dev/sdck" size=4096 rszlo=512 rszhi=1024 crglo=1 crghi=2 boundary=2048 thrlo=8 thrhi=16 actions="write read" rslt_loc=/tmp/sgpdd/20121119_094748  ./sgpdd-survey
Mon Nov 19 09:48:15 CET 2012 sgpdd-survey on mo88:/dev/sdc mo88:/dev/sdbk mo88:/dev/sdaa mo88:/dev/sdcg mo88:/dev/sdcm mo88:/dev/sdak mo88:/dev/sdbg mo88:/dev/sddo mo88:/dev/sdg mo88:/dev/sdbo mo88:/dev/sdbc mo88:/dev/sddk mo88:/dev/sdae mo88:/dev/sdcq mo88:/dev/sdai mo89:/dev/sde mo89:/dev/sdbm mo89:/dev/sdac mo89:/dev/sdci mo89:/dev/sdco mo89:/dev/sdam mo89:/dev/sdbi mo89:/dev/sddq mo89:/dev/sdi mo89:/dev/sdbq mo89:/dev/sdbe mo89:/dev/sddm mo89:/dev/sdag mo89:/dev/sdcs mo89:/dev/sdck from mo88
dev 30 sz 125829120K rsz  512K crg    30 thr   240 write 3419.26 MB/s    30 x 114.19 = 3425.79 MB/s read 10191.50 MB/s    30 x 342.15 = 10264.49 MB/s 
dev 30 sz 125829120K rsz  512K crg    30 thr   480 write 5310.29 MB/s    30 x 177.64 = 5329.23 MB/s read 10293.14 MB/s    30 x 345.89 = 10376.64 MB/s 
dev 30 sz 125829120K rsz  512K crg    60 thr   240 write 3373.46 MB/s    60 x  56.60 = 3396.03 MB/s read 6110.08 MB/s    60 x 102.29 = 6137.47 MB/s 
dev 30 sz 125829120K rsz  512K crg    60 thr   480 write 5286.39 MB/s    60 x  88.42 = 5305.48 MB/s read 6277.94 MB/s    60 x 105.12 = 6307.41 MB/s 
dev 30 sz 125829120K rsz 1024K crg    30 thr   240 write 8298.65 MB/s    30 x 278.06 = 8341.88 MB/s read 10271.83 MB/s    30 x 344.75 = 10342.60 MB/s 
dev 30 sz 125829120K rsz 1024K crg    30 thr   480 write 8399.27 MB/s    30 x 281.72 = 8451.75 MB/s read 10344.08 MB/s    30 x 347.34 = 10420.13 MB/s 
dev 30 sz 125829120K rsz 1024K crg    60 thr   240 write 8183.27 MB/s    60 x 137.20 = 8231.74 MB/s read 7607.72 MB/s    60 x 127.45 = 7646.94 MB/s 
dev 30 sz 125829120K rsz 1024K crg    60 thr   480 write 8291.63 MB/s    60 x 138.27 = 8296.39 MB/s read 7630.64 MB/s    60 x 127.84 = 7670.40 MB/s 



rawdevs="mo88:/dev/raw/raw1 mo88:/dev/raw/raw2 mo88:/dev/raw/raw3 mo88:/dev/raw/raw4 mo88:/dev/raw/raw5 mo88:/dev/raw/raw6 mo88:/dev/raw/raw7 mo88:/dev/raw/raw8 mo88:/dev/raw/raw9 mo88:/dev/raw/raw10 mo88:/dev/raw/raw11 mo88:/dev/raw/raw12 mo88:/dev/raw/raw13 mo88:/dev/raw/raw14 mo88:/dev/raw/raw15 mo89:/dev/raw/raw16 mo89:/dev/raw/raw17 mo89:/dev/raw/raw18 mo89:/dev/raw/raw19 mo89:/dev/raw/raw20 mo89:/dev/raw/raw21 mo89:/dev/raw/raw22 mo89:/dev/raw/raw23 mo89:/dev/raw/raw24 mo89:/dev/raw/raw25 mo89:/dev/raw/raw26 mo89:/dev/raw/raw27 mo89:/dev/raw/raw28 mo89:/dev/raw/raw29 mo89:/dev/raw/raw30" size=4096 rszlo=512 rszhi=1024 crglo=1 crghi=2 boundary=2048 thrlo=8 thrhi=16 actions="write read" rslt_loc=/tmp/sgpdd/20121119_095621 ./sgpdd-survey
Mon Nov 19 09:56:34 CET 2012 sgpdd-survey on mo88:/dev/raw/raw1 mo88:/dev/raw/raw2 mo88:/dev/raw/raw3 mo88:/dev/raw/raw4 mo88:/dev/raw/raw5 mo88:/dev/raw/raw6 mo88:/dev/raw/raw7 mo88:/dev/raw/raw8 mo88:/dev/raw/raw9 mo88:/dev/raw/raw10 mo88:/dev/raw/raw11 mo88:/dev/raw/raw12 mo88:/dev/raw/raw13 mo88:/dev/raw/raw14 mo88:/dev/raw/raw15 mo89:/dev/raw/raw16 mo89:/dev/raw/raw17 mo89:/dev/raw/raw18 mo89:/dev/raw/raw19 mo89:/dev/raw/raw20 mo89:/dev/raw/raw21 mo89:/dev/raw/raw22 mo89:/dev/raw/raw23 mo89:/dev/raw/raw24 mo89:/dev/raw/raw25 mo89:/dev/raw/raw26 mo89:/dev/raw/raw27 mo89:/dev/raw/raw28 mo89:/dev/raw/raw29 mo89:/dev/raw/raw30 from mo88
dev 30 sz 125829120K rsz  512K crg    30 thr   240 write 5051.62 MB/s    30 x 168.43 = 5052.85 MB/s read 7009.16 MB/s    30 x 234.79 = 7043.55 MB/s 
dev 30 sz 125829120K rsz  512K crg    30 thr   480 write 5048.04 MB/s    30 x 168.80 = 5064.01 MB/s read 7038.77 MB/s    30 x 235.89 = 7076.74 MB/s 
dev 30 sz 125829120K rsz  512K crg    60 thr   240 write 4552.82 MB/s    60 x  76.11 = 4566.76 MB/s read 6368.97 MB/s    60 x 106.66 = 6399.54 MB/s 
dev 30 sz 125829120K rsz  512K crg    60 thr   480 write 4609.79 MB/s    60 x  77.06 = 4623.41 MB/s read 6366.94 MB/s    60 x 106.64 = 6398.39 MB/s 
dev 30 sz 125829120K rsz 1024K crg    30 thr   240 write 2196.15 MB/s    30 x  73.76 = 2212.72 MB/s read 9262.66 MB/s    30 x 310.83 = 9324.93 MB/s 
dev 30 sz 125829120K rsz 1024K crg    30 thr   480 write 2228.72 MB/s    30 x  74.39 = 2231.60 MB/s read 9104.59 MB/s    30 x 305.45 = 9163.57 MB/s 
dev 30 sz 125829120K rsz 1024K crg    60 thr   240 write 2786.97 MB/s    60 x  46.36 = 2781.49 MB/s read 7779.28 MB/s    60 x 130.41 = 7824.90 MB/s 
dev 30 sz 125829120K rsz 1024K crg    60 thr   480 write 3099.02 MB/s    60 x  51.75 = 3104.78 MB/s read 7768.62 MB/s    60 x 130.21 = 7812.88 MB/s 

[Cliff White (Inactive)]

Change has been merged