[LU-10294] LBUG lcommon_cl.c:144:cl_file_inode_init()) ASSERTION( (((inode->i_mode) & 00170000) == 0100000) ) failed:

• Type: Bug
• Resolution: Unresolved
• Priority: Major
• Affects Version/s: Lustre 2.11.0, Lustre 2.10.2
• Labels: performance
• Environment: Spirit performance cluster
• Severity: 3
• Assignee: WC Triage
• Reporter: Cliff White (Inactive)

Description

Attempting to run IOR performance test, one client immediately LBUGS:

[81191.988331] LustreError: 38708:0:(lcommon_cl.c:144:cl_file_inode_init()) ASSERTION( (((inode->i_mode) & 00170000) == 0100000) ) failed:
[81192.032178] LustreError: 38708:0:(lcommon_cl.c:144:cl_file_inode_init()) LBUG
[81192.059027] Pid: 38708, comm: mdtest
[81192.073424]
Call Trace:
[81192.091325]  [<ffffffffc092a7ae>] libcfs_call_trace+0x4e/0x60 [libcfs]
[81192.115180]  [<ffffffffc092a83c>] lbug_with_loc+0x4c/0xb0 [libcfs]
[81192.137740]  [<ffffffffc0e33008>] cl_file_inode_init+0x248/0x300 [lustre]
[81192.162137]  [<ffffffffc0e0acf5>] ll_update_inode+0x315/0x600 [lustre]
[81192.185533]  [<ffffffff8121dbb9>] ? iget5_locked+0x99/0x210
[81192.205669]  [<ffffffffc0d01b75>] ? mdc_get_lustre_md+0x825/0xd80 [mdc]
[81192.229177]  [<ffffffffc0e189f0>] ? ll_set_inode+0x0/0x190 [lustre]
[81192.251419]  [<ffffffffc0e196c3>] ll_iget+0x163/0x350 [lustre]
[81192.272145]  [<ffffffffc0e0d062>] ll_prep_inode+0x232/0xc80 [lustre]
[81192.294489]  [<ffffffffc0e1c3b3>] ll_atomic_open+0x783/0x12d0 [lustre]
[81192.317327]  [<ffffffff8121006d>] do_last+0xa4d/0x12c0
[81192.335495]  [<ffffffff812109a2>] path_openat+0xc2/0x490
[81192.354257]  [<ffffffffc0a386c1>] ? lprocfs_counter_sub+0xc1/0x130 [obdclass]
[81192.378986]  [<ffffffff81212f3b>] do_filp_open+0x4b/0xb0
[81192.397685]  [<ffffffff8122019a>] ? __alloc_fd+0x8a/0x130
[81192.416641]  [<ffffffff811ffb83>] do_sys_open+0xf3/0x1f0
[81192.435287]  [<ffffffff811ffc9e>] SyS_open+0x1e/0x20
[81192.452763]  [<ffffffff811ffcd6>] SyS_creat+0x16/0x20
[81192.470499]  [<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
[81192.491356]
[81192.497305] Kernel panic - not syncing: LBUG

Crash dump is available on spirit. This appear identical to some earlier bugs.

Issue Links

• is related to: LU-9352 Lustre clients asserting (lcommon_cl.c:148:cl_file_inode_init()) ASSERTION( (((inode->i_mode) & 00170000) == 0100000) ) failed

Comments

[Cliff White (Inactive)]

any cores left from this would be very long gone. This happened on the old soak test platform. No idea if that hardware even exists now.

[Yang Sheng]

Hi, Cliff and Oleg,

I would like to know do you still keep the vmcore? It should easy to understand what happened with it.

Thanks,
YangSheng

[Oleg Drokin]

Hit this in my testrig in sanity test 247a:

[337236.996600] Lustre: DEBUG MARKER: == sanity test 247a: mount subdir as fileset ========================================================= 00:07:05 (1511672825)
[337237.055464] LustreError: 20336:0:(lcommon_cl.c:146:cl_file_inode_init()) ASSERTION( (((inode->i_mode) & 00170000) == 0100000) ) failed: 
[337237.057047] LustreError: 20336:0:(lcommon_cl.c:146:cl_file_inode_init()) LBUG
[337237.058395] Pid: 20336, comm: mkdir
[337237.059094] 
Call Trace:
[337237.060319]  [<ffffffffa02137ce>] libcfs_call_trace+0x4e/0x60 [libcfs]
[337237.061020]  [<ffffffffa021385c>] lbug_with_loc+0x4c/0xb0 [libcfs]
[337237.061756]  [<ffffffffa14c2ff0>] cl_file_inode_init+0x260/0x320 [lustre]
[337237.096397]  [<ffffffffa149bab5>] ll_update_inode+0x325/0x630 [lustre]
[337237.097138]  [<ffffffffa03c748d>] ? lmv_get_lustre_md+0x7d/0x280 [lmv]
[337237.098661]  [<ffffffffa149dbd1>] ll_prep_inode+0x121/0xb70 [lustre]
[337237.099412]  [<ffffffffa146a6e1>] ll_revalidate_it_finish+0x71/0x170 [lustre]
[337237.100681]  [<ffffffffa1485ec8>] __ll_inode_revalidate+0x1c8/0xb30 [lustre]
[337237.101418]  [<ffffffffa1486c73>] ll_getattr+0x63/0x7c0 [lustre]
[337237.102096]  [<ffffffff810646c5>] ? kernel_map_pages+0xb5/0x120
[337237.102804]  [<ffffffff811f2676>] vfs_getattr+0x46/0x80
[337237.103480]  [<ffffffff811f27a5>] vfs_fstatat+0x75/0xc0
[337237.104129]  [<ffffffff811f2cfe>] SYSC_newstat+0x2e/0x60
[337237.105964]  [<ffffffff81385e06>] ? lockdep_sys_exit_thunk+0x35/0x67
[337237.106702]  [<ffffffff811f2fde>] SyS_newstat+0xe/0x10
[337237.107434]  [<ffffffff8170fc49>] system_call_fastpath+0x16/0x1b
[337237.108100] 
[337237.108691] Kernel panic - not syncing: LBUG
[337237.109336] CPU: 14 PID: 20336 Comm: mkdir Tainted: P        W  OE  ------------   3.10.0-debug #2
[337237.110649] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[337237.111757]  ffffffffa0233212 000000006f13d506 ffff880294607ab8 ffffffff816fd3e4
[337237.113041]  ffff880294607b38 ffffffff816f8c34 ffffffff00000008 ffff880294607b48
[337237.132392]  ffff880294607ae8 000000006f13d506 000000006f13d506 0000000000000001
[337237.133696] Call Trace:
[337237.134342]  [<ffffffff816fd3e4>] dump_stack+0x19/0x1b
[337237.134992]  [<ffffffff816f8c34>] panic+0xd8/0x1e7
[337237.135737]  [<ffffffffa0213874>] lbug_with_loc+0x64/0xb0 [libcfs]
[337237.138447]  [<ffffffffa14c2ff0>] cl_file_inode_init+0x260/0x320 [lustre]
[337237.139149]  [<ffffffffa149bab5>] ll_update_inode+0x325/0x630 [lustre]
[337237.140471]  [<ffffffffa03c748d>] ? lmv_get_lustre_md+0x7d/0x280 [lmv]
[337237.141162]  [<ffffffffa149dbd1>] ll_prep_inode+0x121/0xb70 [lustre]
[337237.141901]  [<ffffffffa146a6e1>] ll_revalidate_it_finish+0x71/0x170 [lustre]
[337237.143140]  [<ffffffffa1485ec8>] __ll_inode_revalidate+0x1c8/0xb30 [lustre]
[337237.143849]  [<ffffffffa1486c73>] ll_getattr+0x63/0x7c0 [lustre]
[337237.145846]  [<ffffffff810646c5>] ? kernel_map_pages+0xb5/0x120
[337237.146541]  [<ffffffff811f2676>] vfs_getattr+0x46/0x80
[337237.147183]  [<ffffffff811f27a5>] vfs_fstatat+0x75/0xc0
[337237.147844]  [<ffffffff811f2cfe>] SYSC_newstat+0x2e/0x60
[337237.148554]  [<ffffffff81385e06>] ? lockdep_sys_exit_thunk+0x35/0x67
[337237.149222]  [<ffffffff811f2fde>] SyS_newstat+0xe/0x10
[337237.149889]  [<ffffffff8170fc49>] system_call_fastpath+0x16/0x1b