[LU-1188] mkwrite issue : ASSERTION(!cl_env_info(env)->clt_counters[CNL_TOP].ctc_nr_locks_acquired)

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.3.0
• Affects Version/s: Lustre 2.1.0, Lustre 2.2.0
• Environment: RHEL6/64, 8CPU.
• Severity: 3
• Assignee: Jinshan Xiong (Inactive)
• Reporter: Alexey Lyashkov

Description

Mar 5 16:54:29 rhel6-64 kernel: Lustre: DEBUG MARKER: test message ID 29780 32002
Lustre: DEBUG MARKER: == sanity test 61: mmap() writes don't make sync hang ================== 16:54:30 (1330959270)
Mar 5 16:54:30 rhel6-64 kernel: Lustre: DEBUG MARKER: == sanity test 61: mmap() writes don't make sync hang ================== 16:54:30 (1330959270)
Lustre: DEBUG MARKER: cancel_lru_locks osc start
Mar 5 16:54:30 rhel6-64 kernel: Lustre: DEBUG MARKER: cancel_lru_locks osc start
Lustre: DEBUG MARKER: cancel_lru_locks osc stop
Mar 5 16:54:30 rhel6-64 kernel: Lustre: DEBUG MARKER: cancel_lru_locks osc stop
LustreError: 24138:0:(cl_io.c:523:cl_io_unlock()) ASSERTION(!cl_env_info(env)->clt_counters[CNL_TOP].ctc_nr_locks_acquired) failed
LustreError: 24138:0:(cl_io.c:523:cl_io_unlock()) LBUG

Mar 5 16:54:30 rhel6-64 kernel: LustreError: 24138:0:(cl_io.c:523:cl_io_unlock()) LBUG
Mar 5 16:54:30 rhel6-64 kernel: Pid: 24138, comm: multiop
Mar 5 16:54:30 rhel6-64 kernel:
Mar 5 16:54:30 rhel6-64 kernel: Call Trace:
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa04e1865>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa04e1ea5>] lbug_with_loc+0x75/0xe0 [libcfs]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa04ecf46>] libcfs_assertion_failed+0x66/0x70 [libcfs]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa0655a0d>] cl_io_unlock+0x23d/0x270 [obdclass]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa0651b80>] ? cl_io_end+0x60/0x120 [obdclass]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa0656769>] cl_io_loop+0x129/0x1c0 [obdclass]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa0b4c346>] ll_page_mkwrite+0x96/0x7d0 [lustre]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa0b4bfb9>] ? ll_fault+0x189/0x480 [lustre]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffffa04f1ce2>] ? cfs_hash_rw_lock+0x12/0x30 [libcfs]
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff81151e14>] __do_fault+0xd4/0x4f0
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff8151f2ab>] ? _spin_unlock+0x2b/0x40
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff811522c0>] handle_pte_fault+0x90/0xa90
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff81156398>] ? vma_link+0x58/0xf0
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff81152ea4>] handle_mm_fault+0x1e4/0x2b0
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff81043c23>] __do_page_fault+0x163/0x4e0
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff8115905a>] ? do_mmap_pgoff+0x33a/0x380
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff81522cae>] do_page_fault+0x3e/0xa0
Mar 5 16:54:30 rhel6-64 kernel: [<ffffffff8151fde5>] page_fault+0x25/0x30
Mar 5 16:54:30 rhel6-64 kernel:

Issue Links

• Trackbacks: Changelog 2.2 version 2.2.0 Support for networks: o2iblnd OFED 1.5.4 Server support for kernels: 2.6.32220.4.2.el6 (RHEL6) Client support for unpatched kernels: 2.6.18274.18.1.el5 (RHEL5) 2.6.32220.4.2.el6 (RHEL6) 2.6.32.360....

Comments

[Build Master (Inactive)]

Integrated in lustre-reviews » i686,server,el5,inkernel #4596
LU-1188 clio: acquire lockdep for cl_lock_peek() (Revision 08038971ff2253fa23305b94d74b0d683dc968fc)

Result = SUCCESS
Jinshan Xiong : 08038971ff2253fa23305b94d74b0d683dc968fc
Files :

• lustre/obdclass/cl_lock.c

[Build Master (Inactive)]

Integrated in lustre-reviews » i686,client,el6,inkernel #4596
LU-1188 clio: acquire lockdep for cl_lock_peek() (Revision 08038971ff2253fa23305b94d74b0d683dc968fc)

Result = SUCCESS
Jinshan Xiong : 08038971ff2253fa23305b94d74b0d683dc968fc
Files :

• lustre/obdclass/cl_lock.c

[Build Master (Inactive)]

Integrated in lustre-reviews » x86_64,client,el5,inkernel #4596
LU-1188 clio: acquire lockdep for cl_lock_peek() (Revision 08038971ff2253fa23305b94d74b0d683dc968fc)

Result = SUCCESS
Jinshan Xiong : 08038971ff2253fa23305b94d74b0d683dc968fc
Files :

• lustre/obdclass/cl_lock.c

[Build Master (Inactive)]

Integrated in lustre-reviews » x86_64,client,el6,inkernel #4596
LU-1188 clio: acquire lockdep for cl_lock_peek() (Revision 08038971ff2253fa23305b94d74b0d683dc968fc)

Result = SUCCESS
Jinshan Xiong : 08038971ff2253fa23305b94d74b0d683dc968fc
Files :

• lustre/obdclass/cl_lock.c

[Johann Lombardi (Inactive)]

This problem is related to CONFIG_LOCKDEP enabled in the kernel and Jinshan is working on a patch.

[Jinshan Xiong (Inactive)]

No, I don't need it, thanks for offering.

[Alexey Lyashkov]

Jay is it will help for you, if i provide a crash dump ?

[Alexey Lyashkov]

One note, my test env have 8CPU's.

[Alexey Lyashkov]

generic kernel config with debug options enabled, just to find a lock issues.

[Jinshan Xiong (Inactive)]

Shadow, can you please tell me the specific configuration of your kernel?

[Alexey Lyashkov]

Peter,

why it is isn't blocker? that real regression between 2.1 and 2.2 and hit at any sanity run.