[LU-7919] Buffer overflow in mount_lustre: parse_ldd(),append_option() Created: 25/Mar/16  Updated: 27/Oct/22  Resolved: 08/Oct/16

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.8.0
Fix Version/s: Lustre 2.9.0

Type: Bug Priority: Major
Reporter: Lokesh Nagappa Jaliminche (Inactive) Assignee: WC Triage
Resolution: Fixed Votes: 0
Labels: mount

Issue Links:
Related
is related to LU-7965 mkfs.lustre sometimes fails with a se... Resolved
is related to LU-11785 conf-sanity test_98 fails with 'Buffe... Resolved
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

buffer overflow in mount_lustre: parse_ldd(),append_option()

Reproduction Steps:
================
[root@dev-1 tests]# sh llmount.sh
[root@dev-1 tests]# rm -f /tmp/A2000; for x in $(seq 1 4000); do echo -n A >> /tmp/A2000 ; done
[root@dev-1 tests]# umount /mnt/mds1
[root@dev-1 tests]# losetup /dev/loop0 /tmp/lustre-mdt1
[root@dev-1 tests]# mount -t lustre /dev/loop0 /mnt/mds1 -o option4kplus=$(cat /tmp/A2000 )

failure logs:
=========

[root@dev-1 tests]# mount -t lustre /dev/loop0 /mnt/mds1 -o option4kplus=$(cat /tmp/A2000 )
mount.lustre: mount /dev/loop0 at /mnt/mds1 failed: Invalid argument
This may have multiple causes.
Are the mount options correct?
Check the syslog for more info.
*** glibc detected *** /sbin/mount.lustre: free(): invalid next size (normal): 0x000000000234f050 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3bbc075e66]
/lib64/libc.so.6[0x3bbc0789b3]
/sbin/mount.lustre(main+0x411)[0x402f11]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3bbc01ed5d]
/sbin/mount.lustre[0x4024a9]



 Comments   
Comment by Gerrit Updater [ 25/Mar/16 ]

lokesh.jaliminche (lokesh.jaliminche@seagate.com) uploaded a new patch: http://review.whamcloud.com/19158
Subject: LU-7919 mount: Buffer overflow issue while parsing mount
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8fae5045d8f92ca8924dc33684903d032b06dd39

Comment by Lokesh Nagappa Jaliminche (Inactive) [ 09/Aug/16 ]

Regarding failure on test-board,
https://testing.hpdd.intel.com/test_sets/bb6c7ec6-5de3-11e6-906c-5254006e85c2

I have checked on my local quartet setup, it is working properly.
My test case is looking for the error string when mount options exceeds the size limit ,

here are the logs on my local quartet setup
==================================

ogs on Local quartet setup intel master
== conf-sanity test 98: Buffer-overflow check while parsing mount_opts == 14:56:11 (1470734771)
start mds service on 192.168.56.147
Loading modules from /root/mrp/intel/lustre-wc/lustre
detected 1 online CPUs by sysfs
libcfs will create CPU partition based on online CPUs
debug=-1
subsystem_debug=all -lnet -lnd -pinger
../lnet/lnet/lnet options: 'networks=tcp0(eth0) accept=all'
gss/krb5 is not supported
quota/lquota options: 'hash_lqs_cur_bits=3'
pdsh@dev-8: 192.168.56.147: ssh exited with exit code 1
Starting mds1:   -o loop /tmp/lustre-mdt1 /mnt/lustre-mds1
pdsh@dev-8: 192.168.56.147: ssh exited with exit code 1
pdsh@dev-8: 192.168.56.147: ssh exited with exit code 1
Started lustre-MDT0000
start ost1 service on 192.168.56.145
pdsh@dev-8: 192.168.56.145: ssh exited with exit code 1
Starting ost1:   -o loop /tmp/lustre-ost1 /mnt/lustre-ost1
pdsh@dev-8: 192.168.56.145: ssh exited with exit code 1
pdsh@dev-8: 192.168.56.145: ssh exited with exit code 1
Started lustre-OST0000
mount lustre on /mnt/lustre.....
Starting client: dev-8:  -o user_xattr,flock 192.168.56.147@tcp:/lustre /mnt/lustre
setup single mount lustre success
stop mds service on 192.168.56.147
Stopping /mnt/lustre-mds1 (opts:-f) on 192.168.56.147
pdsh@dev-8: 192.168.56.147: ssh exited with exit code 1
start mds service on 192.168.56.147 pdsh@dev-8: 192.168.56.147: ssh exited with exit code 1 Starting mds1: -o user_xattr,user_xattr,user_xattr,user_xattr,loop /tmp/lustre-mdt1 /mnt/lustre-mds1 192.168.56.147: error: mount options exceeds page size of kernel pdsh@dev-8: 192.168.56.147: ssh exited with exit code 7 Start of /tmp/lustre-mdt1 on mds1 failed 7

Test case checks for this string error: mount options exceeds page size of kernel but instead I am getting different error on Maloo test-board

Logs on Maloo Test-Board
=====================

== conf-sanity test 98: Buffer-overflow check while parsing mount_opts =============================== 02:11:19 (1470708679)
start mds service on trevis-57vm7
CMD: trevis-57vm7 mkdir -p /mnt/lustre-mds1
Loading modules from /usr/lib64/lustre
detected 1 online CPUs by sysfs
libcfs will create CPU partition based on online CPUs
debug=-1
subsystem_debug=all -lnet -lnd -pinger
CMD: trevis-57vm7 test -b /dev/lvm-Role_MDS/P1
CMD: trevis-57vm7 e2label /dev/lvm-Role_MDS/P1
Starting mds1:   /dev/lvm-Role_MDS/P1 /mnt/lustre-mds1
CMD: trevis-57vm7 mkdir -p /mnt/lustre-mds1; mount -t lustre   		                   /dev/lvm-Role_MDS/P1 /mnt/lustre-mds1
CMD: trevis-57vm7 /usr/sbin/lctl get_param -n health_check
CMD: trevis-57vm7 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests//usr/lib64/lustre/tests:/usr/lib64/lustre/tests:/usr/lib64/lustre/tests/../utils:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/qt-3.3/bin:/usr/lib64/compat-openmpi16/bin:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh set_default_debug \"-1\" \"all -lnet -lnd -pinger\" 4 
CMD: trevis-57vm7 e2label /dev/lvm-Role_MDS/P1 				2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}'
CMD: trevis-57vm7 e2label /dev/lvm-Role_MDS/P1 				2>/dev/null | grep -E ':[a-zA-Z]{3}[0-9]{4}'
CMD: trevis-57vm7 e2label /dev/lvm-Role_MDS/P1 2>/dev/null
.
.
.
CMD: trevis-57vm7 umount -d -f /mnt/lustre-mds3
CMD: trevis-57vm7 lsmod | grep lnet > /dev/null && lctl dl | grep ' ST '
stop mds service on trevis-57vm3
CMD: trevis-57vm3 grep -c /mnt/lustre-mds4' ' /proc/mounts
Stopping /mnt/lustre-mds4 (opts:-f) on trevis-57vm3
CMD: trevis-57vm3 umount -d -f /mnt/lustre-mds4
CMD: trevis-57vm3 lsmod | grep lnet > /dev/null && lctl dl | grep ' ST '
start mds service on trevis-57vm7 CMD: trevis-57vm7 mkdir -p /mnt/lustre-mds1 CMD: trevis-57vm7 test -b /dev/lvm-Role_MDS/P1 CMD: trevis-57vm7 e2label /dev/lvm-Role_MDS/P1 Starting mds1: -o user_xattr,user_xattr,user_xattr,user_xattr,user_xattr,user_xattr /dev/lvm-Role_MDS/P1 /mnt/lustre-mds1 pdsh@trevis-57vm1: trevis-57vm7: mcmd: Bad read of expected verification number off of stderr socket: Success Start of /dev/lvm-Role_MDS/P1 on mds1 failed 254
 conf-sanity test_98: @@@@@@ FAIL: Buffer overflow check failed 
  Trace dump:

In above logs I am getting this error
"mcmd: Bad read of expected verification number off of stderr socket: Success Start of /dev/lvm-Role_MDS/P1 on mds1 failed 254"
which is unexpected. I am not sure if this is happening because of my patch, can someone please help me get more info on this ?

Comment by Gerrit Updater [ 08/Oct/16 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/19158/
Subject: LU-7919 mount: Buffer overflow issue while parsing mount
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 00049e341a1e978c635b44f1d3ae474d0eb75f10

Comment by Peter Jones [ 08/Oct/16 ]

Landed for 2.9

Generated at Sat Feb 10 02:13:03 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.