[LU-7353] Segfault on lctl commands 'lctl network peer_list' 'lctl network conn_list' Created: 27/Oct/15  Updated: 01/Jul/16  Resolved: 11/Nov/15

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.8.0
Fix Version/s: Lustre 2.8.0

Type: Bug Priority: Minor
Reporter: Jeff Johnson (Inactive) Assignee: Dmitry Eremin (Inactive)
Resolution: Fixed Votes: 1
Labels: None
Environment:

CentOS 6.6, 2.6.32-504.el6.x86_64, Lustre 2.7.56 / ZFS
(used Lustre 2.8 version as is closest to actual version)

Attachments: File peer_list_core.tar.gz    
Issue Links:
Related
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

LFS performing normally. Admin issues lctl commands which result in segfaults.

'lctl network peer_list'
'lctl network conn_list'
each result in segfaults.

Attached core file resulting from 'lctl network peer_list' executed on an OSS node.

 Comments   
Comment by Jeff Johnson (Inactive) [ 28/Oct/15 ]

Trace:

[root@ls15-mds-00.i ~]# lctl network peer_list

      • Segmentation fault
        Register dump:

RAX: 000000000063a620 RBX: 000000000063a620 RCX: 0000000000000000
RDX: 000000000000000e RSI: 000000000043280b RDI: 0000000000000000
RBP: 00007fffade0c00c R8 : 0000000000000000 R9 : 0000000000000001
R10: 00000030d5329f00 R11: 0000000000000003 R12: 000000000000000f
R13: 0000000000000000 R14: 0000000000000003 R15: 00007fffade0c7e6
RSP: 00007fffade0bfa8

RIP: 00000030d533372f EFLAGS: 00010246

CS: 0033 FS: 0000 GS: 0000

Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000000

FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000

ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1f80
XMM0: 000000000000000000000000ff000000 XMM1: 000000000000000000000000ff000000
XMM2: 000000000000000000000000ff000000 XMM3: 000000000000000000000000ff000000
XMM4: 000000000000000000000000ff000000 XMM5: 000000000000000000000000ff000000
XMM6: 000000000000000000000000ff000000 XMM7: 000000000000000000000000ff000000
XMM8: 000000000000000000000000ff000000 XMM9: 000000000000000000000000ff000000
XMM10: 000000000000000000000000ff000000 XMM11: 000000000000000000000000ff000000
XMM12: 000000000000000000000000ff000000 XMM13: 000000000000000000000000ff000000
XMM14: 000000000000000000000000ff000000 XMM15: 000000000000000000000000ff000000

Backtrace:
/lib64/libc.so.6[0x30d533372f]
lctl[0x428a35]
lctl[0x428b0e]
lctl[0x422081]
lctl[0x427698]
lctl[0x416517]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x30d521ed5d]
lctl[0x402f59]

Memory map:

00400000-00438000 r-xp 00000000 09:02 1201235 /usr/sbin/lctl
00637000-0063b000 rw-p 00037000 09:02 1201235 /usr/sbin/lctl
0063b000-00658000 rw-p 00000000 00:00 0
0083a000-0083b000 rw-p 0003a000 09:02 1201235 /usr/sbin/lctl
01dbd000-01dde000 rw-p 00000000 00:00 0 [heap]
30d4e00000-30d4e20000 r-xp 00000000 09:02 131102 /lib64/ld-2.12.so
30d501f000-30d5020000 r--p 0001f000 09:02 131102 /lib64/ld-2.12.so
30d5020000-30d5021000 rw-p 00020000 09:02 131102 /lib64/ld-2.12.so
30d5021000-30d5022000 rw-p 00000000 00:00 0
30d5200000-30d538a000 r-xp 00000000 09:02 131128 /lib64/libc-2.12.so
30d538a000-30d558a000 ---p 0018a000 09:02 131128 /lib64/libc-2.12.so
30d558a000-30d558e000 r--p 0018a000 09:02 131128 /lib64/libc-2.12.so
30d558e000-30d558f000 rw-p 0018e000 09:02 131128 /lib64/libc-2.12.so
30d558f000-30d5594000 rw-p 00000000 00:00 0
30d5600000-30d5602000 r-xp 00000000 09:02 131321 /lib64/libdl-2.12.so
30d5602000-30d5802000 ---p 00002000 09:02 131321 /lib64/libdl-2.12.so
30d5802000-30d5803000 r--p 00002000 09:02 131321 /lib64/libdl-2.12.so
30d5803000-30d5804000 rw-p 00003000 09:02 131321 /lib64/libdl-2.12.so
30d5a00000-30d5a17000 r-xp 00000000 09:02 131154 /lib64/libpthread-2.12.s
o
30d5a17000-30d5c17000 ---p 00017000 09:02 131154 /lib64/libpthread-2.12.s
o
30d5c17000-30d5c18000 r--p 00017000 09:02 131154 /lib64/libpthread-2.12.s
o
30d5c18000-30d5c19000 rw-p 00018000 09:02 131154 /lib64/libpthread-2.12.s
o
30d5c19000-30d5c1d000 rw-p 00000000 00:00 0
30d7200000-30d7216000 r-xp 00000000 09:02 131557 /lib64/libgcc_s-4.4.7-20
120601.so.1
30d7216000-30d7415000 ---p 00016000 09:02 131557 /lib64/libgcc_s-4.4.7-20
120601.so.1
30d7415000-30d7416000 rw-p 00015000 09:02 131557 /lib64/libgcc_s-4.4.7-20
120601.so.1
30d7600000-30d7602000 r-xp 00000000 09:02 131500 /lib64/libkeyutils.so.1.
3
30d7602000-30d7801000 ---p 00002000 09:02 131500 /lib64/libkeyutils.so.1.
3
30d7801000-30d7802000 r--p 00001000 09:02 131500 /lib64/libkeyutils.so.1.
3
30d7802000-30d7803000 rw-p 00002000 09:02 131500 /lib64/libkeyutils.so.1.
3
7f6e22da2000-7f6e22db3000 rw-s 00000000 00:04 1736704 /SYSV00000000 (deleted)
7f6e22db3000-7f6e22db7000 rw-p 00000000 00:00 0
7f6e22db7000-7f6e22dbb000 r-xp 00000000 09:02 131082 /lib64/libSegFault.so
7f6e22dbb000-7f6e22fba000 ---p 00004000 09:02 131082 /lib64/libSegFault.so
7f6e22fba000-7f6e22fbb000 r--p 00003000 09:02 131082 /lib64/libSegFault.so
7f6e22fbb000-7f6e22fbc000 rw-p 00004000 09:02 131082 /lib64/libSegFault.so
7f6e22fc9000-7f6e22fca000 rw-p 00000000 00:00 0
7fffaddf8000-7fffade0d000 rw-p 00000000 00:00 0 [stack]
7fffade25000-7fffade26000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

Comment by Jeff Johnson (Inactive) [ 28/Oct/15 ]

Running 'lctl --net o2ib0 peer_list' appears to function normally. Executing the lctl command with improper arguments results in either a segfault or no error at all.

'lctl network peer_list' results in a segfault

'lctl network o2ib0 peer_list' results in no segfault, no stderr, exit 0

'lctl --net o2ib0 peer_list' executes properly as expected.

Comment by Joseph Gmitter (Inactive) [ 28/Oct/15 ]

Hi Dmitry,
Can you have a look at this issue?
Thanks.
Joe

Comment by Gerrit Updater [ 28/Oct/15 ]

Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: http://review.whamcloud.com/16980
Subject: LU-7353 utils: fix lctl usage messages
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ae64bd87141af3187144f1ef1e233abd5eaf7a39

Comment by Gerrit Updater [ 11/Nov/15 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/16980/
Subject: LU-7353 utils: fix lctl usage messages
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fa54edb5af34c3bcbf878d7498ef337ef3a1df7a

Comment by Joseph Gmitter (Inactive) [ 11/Nov/15 ]

Landed for 2.8

Generated at Sat Feb 10 02:08:09 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.