[LU-9348] brw_size is not always dynamically changeable

Sun, 26 Nov 2017 22:17:04 +0000

On one OSS I could do:
[root@wolf-3 combined]# lctl set_param obdfilter.lsdraid-OST0000.brw_size=1
obdfilter.lsdraid-OST0000.brw_size=1
[root@wolf-3 combined]# lctl set_param obdfilter.*.brw_size=1
obdfilter.lsdraid-OST0000.brw_size=1

But on another OSS I could not do the same thing:

lctl set_param obdfilter.*.brw_size=1
error: set_param: setting /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size=1: Numerical result out of range
[root@wolf-4 combined]# cat /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
16
[root@wolf-4 combined]# echo "0" > /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
-bash: echo: write error: Numerical result out of range
[root@wolf-4 combined]# echo "1" > /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
-bash: echo: write error: Numerical result out of range
[root@wolf-4 combined]# echo "16" > /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
[root@wolf-4 combined]# echo "17" > /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
-bash: echo: write error: Numerical result out of range
[root@wolf-4 combined]# echo "10" > /proc/fs/lustre/obdfilter/lsdraid-OST0001/brw_size
-bash: echo: write error: Numerical result out of range

dmesg:
[ 91.466648] SPL: using hostid 0x61303830
[ 99.368278] LNet: HW CPU cores: 72, npartitions: 2
[ 99.375528] alg: No test for adler32 (adler32-zlib)
[ 99.381114] alg: No test for crc32 (crc32-table)
[ 107.466028] Lustre: Lustre: Build Version: 2.9.0_dirty
[ 107.583440] LNet: Added LNI 192.168.1.4@o2ib [8/256/0/180]
[ 107.912856] LustreError: 11-0: lsdraid-MDT0000-lwp-OST0001: operation mds_connect to node 192.168.1.5@o2ib failed: rc = -114
[ 107.977949] Lustre: lsdraid-OST0001: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-450
[ 113.824128] Lustre: lsdraid-OST0001: Will be in recovery for at least 2:30, or until 1 client reconnects
[ 113.834861] Lustre: lsdraid-OST0001: Connection restored to lsdraid-MDT0000-mdtlov_UUID (at 192.168.1.5@o2ib)
[ 114.012466] Lustre: lsdraid-OST0001: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 114.105679] Lustre: lsdraid-OST0001: deleting orphan objects from 0x0:82221 to 0x0:83201
[ 132.876656] LustreError: 11-0: lsdraid-MDT0000-lwp-OST0001: operation mds_connect to node 192.168.1.5@o2ib failed: rc = -114
[ 157.849631] LustreError: 11-0: lsdraid-MDT0000-lwp-OST0001: operation mds_connect to node 192.168.1.5@o2ib failed: rc = -114
[ 699.046036] Lustre: Failing over lsdraid-OST0001
[ 699.058930] Lustre: server umount lsdraid-OST0001 complete
[ 754.958545] Lustre: lsdraid-OST0001: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-450
[ 756.682541] Lustre: lsdraid-OST0001: Will be in recovery for at least 2:30, or until 1 client reconnects
[ 756.693277] Lustre: lsdraid-OST0001: Connection restored to lsdraid-MDT0000-mdtlov_UUID (at 192.168.1.5@o2ib)
[ 756.867379] Lustre: lsdraid-OST0001: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 756.940023] Lustre: lsdraid-OST0001: deleting orphan objects from 0x0:82221 to 0x0:83233

The impact is that this pool has to be unmounted and destroyed before more testing with different brw_size can continue. This only happens with OSTs that have been used with existing data, on a newly formatted OST I seem to be able to change the size every time.

Whamcloud Community JIRA

[LU-9348] brw_size is not always dynamically changeable