[LU-3190] Interop 2.3.0<->2.4 Failed on lustre-rsync-test test 3b: ASSERTION( lio->lis_lsm != ((void *)0) ) failed

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.3.0, Lustre 2.4.0
• Labels: LB
• Environment: server: lustre-master tag-2.3.64 build #1411; client: 2.3.0
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Sarah Liu

Description

Hit following LBUG when running lustre-rsync-test test 3b
In tag-2.3.62, the same test passed

Lustre: DEBUG MARKER: == lustre-rsync-test test 3b: Replicate files created by writemany == 17:57:47 (1366246667)
LustreError: 6661:0:(lmv_obd.c:850:lmv_iocontrol()) error: iocontrol MDC lustre-MDT0000_UUID on MDTidx 0 cmd c0086696: err = -2
LustreError: 6661:0:(lmv_obd.c:850:lmv_iocontrol()) Skipped 1415 previous similar messages
Lustre: DEBUG MARKER: == lustre-rsync-test test 3c: Replicate files created by createmany/unlinkmany == 17:59:17 (1366246757)
Lustre: DEBUG MARKER: == lustre-rsync-test test 4: Replicate files created by iozone == 17:59:33 (1366246773)
LustreError: 7489:0:(lcommon_cl.c:1210:cl_file_inode_init()) Failure to initialize cl object [0x20001d0f0:0x340d:0x0]: -95
LustreError: 7489:0:(lcommon_cl.c:1210:cl_file_inode_init()) Failure to initialize cl object [0x20001d0f0:0x340d:0x0]: -95
LustreError: 7489:0:(lov_io.c:311:lov_io_slice_init()) ASSERTION( lio->lis_lsm != ((void *)0) ) failed: 
LustreError: 7489:0:(lov_io.c:311:lov_io_slice_init()) LBUG
Pid: 7489, comm: lustre_rsync

Message from
Call Trace:
 syslogd@client- [<ffffffffa0996905>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
5 at Apr 17 18:0 [<ffffffffa0996f17>] lbug_with_loc+0x47/0xb0 [libcfs]
0:04 ...
 kern [<ffffffffa06b5088>] lov_io_init_raid0+0x6d8/0x810 [lov]
el:LustreError:  [<ffffffffa06ac037>] lov_io_init+0x97/0x160 [lov]
7489:0:(lov_io.c [<ffffffffa0dd1578>] cl_io_init0+0x98/0x160 [obdclass]
 [<ffffffffa0dd4464>] cl_io_init+0x64/0x100 [obdclass]
 [<ffffffffa07e6fed>] cl_glimpse_size0+0x7d/0x190 [lustre]
:311:lov_io_slic [<ffffffffa07a3f32>] ll_inode_revalidate_it+0xf2/0x1c0 [lustre]
 [<ffffffffa07a4049>] ll_getattr_it+0x49/0x170 [lustre]
 [<ffffffffa07a41a7>] ll_getattr+0x37/0x40 [lustre]
 [<ffffffff81214343>] ? security_inode_getattr+0x23/0x30
e_init()) ASSERT [<ffffffff81180571>] vfs_getattr+0x51/0x80
 [<ffffffffa09a2088>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffff8118082f>] vfs_fstat+0x3f/0x60
 [<ffffffff81180874>] sys_newfstat+0x24/0x40
 [<ffffffff810d6b12>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b

ION( lio->lis_lsKernel panic - not syncing: LBUG
m != ((void *)0)Pid: 7489, comm: lustre_rsync Not tainted 2.6.32-279.5.1.el6.x86_64 #1
 ) failed: 
Call Trace:

Message from s [<ffffffff814fd24a>] ? panic+0xa0/0x168
yslogd@client-5  [<ffffffffa0996f6b>] ? lbug_with_loc+0x9b/0xb0 [libcfs]
at Apr 17 18:00: [<ffffffffa06b5088>] ? lov_io_init_raid0+0x6d8/0x810 [lov]
04 ...
 kernel [<ffffffffa06ac037>] ? lov_io_init+0x97/0x160 [lov]
:LustreError: 74 [<ffffffffa0dd1578>] ? cl_io_init0+0x98/0x160 [obdclass]
89:0:(lov_io.c:3 [<ffffffffa0dd4464>] ? cl_io_init+0x64/0x100 [obdclass]
11:lov_io_slice_ [<ffffffffa07e6fed>] ? cl_glimpse_size0+0x7d/0x190 [lustre]
init()) LBUG
 [<ffffffffa07a3f32>] ? ll_inode_revalidate_it+0xf2/0x1c0 [lustre]

Message from  [<ffffffffa07a4049>] ? ll_getattr_it+0x49/0x170 [lustre]
syslogd@client-5 [<ffffffffa07a41a7>] ? ll_getattr+0x37/0x40 [lustre]
 at Apr 17 18:00 [<ffffffff81214343>] ? security_inode_getattr+0x23/0x30
:04 ...
 kerne [<ffffffff81180571>] ? vfs_getattr+0x51/0x80
l:Kernel panic - [<ffffffffa09a2088>] ? libcfs_log_return+0x28/0x40 [libcfs]
 not syncing: LB [<ffffffff8118082f>] ? vfs_fstat+0x3f/0x60
UG
 [<ffffffff81180874>] ? sys_newfstat+0x24/0x40
 [<ffffffff810d6b12>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
Initializing cgroup subsys cpuset

Issue Links

• is related to: LU-3279 Interop 2.3.0 <-> 2.4 failure on test suite lustre-rsync-test test_7: Failure in replication; differences found