[LU-15122] Lustre ASSERTION( iobuf->dr_rw == 0 ) crash on Arm server end

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.15.0
• Affects Version/s: Upstream
• Labels: arm arm-server

Environment

[centos@beegfs-test-01 ~]$ lsb_release -a
LSB Version: :core-4.1-aarch64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 8.3.2011
Release: 8.3.2011
Codename: n/a
[centos@beegfs-test-01 ~]$ uname -a
Linux beegfs-test-01.novalocal 4.18.0-305.7.1.el8_lustre.aarch64 #1 SMP Mon Jul 19 08:24:26 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
[centos@beegfs-test-01 ~]$

Epic/Theme

ldiskfs

• Severity: 3
• Assignee: Xinliang Liu
• Reporter: Xinliang Liu

Description

It is easy to enconter bellow crash when running some tests on Arm all-in-one(server and client) test evironment. Tests such as fsx tests: sanityn 16a, 16b etc.

[57580.041008] Lustre: DEBUG MARKER: -----============= acceptance-small: sanityn ============----- Tue Oct 12 02:22:12 UTC 2021
[57580.414512] Lustre: DEBUG MARKER: excepting tests: 28
[57580.528887] Lustre: Mounted lustre-client
[57580.868328] Lustre: DEBUG MARKER: beegfs-test-01.novalocal: executing check_config_client /mnt/lustre
[57581.696557] Lustre: DEBUG MARKER: Using TIMEOUT=20
[57581.854832] Lustre: Modifying parameter general.*.*.lbug_on_grant_miscount in log params
[57582.331371] Lustre: DEBUG MARKER: == sanityn test 16b: 12500 iterations of dual-mount fsx at small size ========================================================== 02:22:15 (1634005335)
[57585.485613] LustreError: 5594:0:(osd_io.c:538:osd_do_bio()) ASSERTION( iobuf->dr_rw == 0 ) failed: page_idx 1, block_idx 16, i 13,start_blocks: 0, count: 29, npages: 2
[57585.488764] LustreError: 5594:0:(osd_io.c:538:osd_do_bio()) LBUG
[57585.490011] Pid: 5594, comm: ll_ost_io03_000 4.18.0-305.7.1.el8_lustre.aarch64 #1 SMP Mon Jul 19 08:24:26 UTC 2021
[57585.492109] Call Trace:
[57585.492633] [<0>] libcfs_call_trace+0xb8/0x118 [libcfs]
[57585.493719] [<0>] lbug_with_loc+0x60/0xa0 [libcfs]
[57585.494721] [<0>] osd_do_bio.constprop.21+0x6c4/0xcf8 [osd_ldiskfs]
[57585.496012] [<0>] osd_ldiskfs_map_write.constprop.20+0xb4/0xf0 [osd_ldiskfs]
[57585.497464] [<0>] osd_ldiskfs_map_inode_pages+0x66c/0x958 [osd_ldiskfs]
[57585.498880] [<0>] osd_write_commit+0x4fc/0xbb8 [osd_ldiskfs]
[57585.500047] [<0>] ofd_commitrw_write+0x658/0x1e68 [ofd]
[57585.501133] [<0>] ofd_commitrw+0x360/0xa18 [ofd]
[57585.502180] [<0>] tgt_brw_write+0x1624/0x2d90 [ptlrpc]
[57585.503299] [<0>] tgt_handle_request0+0xd0/0x978 [ptlrpc]
[57585.504466] [<0>] tgt_request_handle+0x7c0/0x1a38 [ptlrpc]
[57585.505663] [<0>] ptlrpc_server_handle_request+0x3bc/0x11e8 [ptlrpc]
[57585.507026] [<0>] ptlrpc_main+0xd28/0x15f0 [ptlrpc]
[57585.508034] [<0>] kthread+0x130/0x138
[57585.508861] [<0>] ret_from_fork+0x10/0x18
[57585.509697] Kernel panic - not syncing: LBUG
[57585.510570] CPU: 6 PID: 5594 Comm: ll_ost_io03_000 Kdump: loaded Tainted: P           OE    --------- -  - 4.18.0-305.7.1.el8_lustre.aarch64 #1
[57585.513181] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[57585.514576] Call trace:
[57585.515080]  dump_backtrace+0x0/0x188
[57585.515826]  show_stack+0x24/0x30
[57585.516530]  dump_stack+0x9c/0xbc
[57585.517189]  panic+0x130/0x2f8
[57585.517802]  param_set_delay_minmax.isra.1+0x0/0xd0 [libcfs]
[57585.518972]  osd_do_bio.constprop.21+0x6c4/0xcf8 [osd_ldiskfs]
[57585.520204]  osd_ldiskfs_map_write.constprop.20+0xb4/0xf0 [osd_ldiskfs]
[57585.521607]  osd_ldiskfs_map_inode_pages+0x66c/0x958 [osd_ldiskfs]
[57585.522910]  osd_write_commit+0x4fc/0xbb8 [osd_ldiskfs]
[57585.524012]  ofd_commitrw_write+0x658/0x1e68 [ofd]
[57585.525023]  ofd_commitrw+0x360/0xa18 [ofd]
[57585.525982]  tgt_brw_write+0x1624/0x2d90 [ptlrpc]
[57585.527046]  tgt_handle_request0+0xd0/0x978 [ptlrpc]
[57585.528160]  tgt_request_handle+0x7c0/0x1a38 [ptlrpc]
[57585.529316]  ptlrpc_server_handle_request+0x3bc/0x11e8 [ptlrpc]
[57585.530622]  ptlrpc_main+0xd28/0x15f0 [ptlrpc]
[57585.531557]  kthread+0x130/0x138
[57585.532236]  ret_from_fork+0x10/0x18
[57585.533308] SMP: stopping secondary CPUs
[57585.539225] Starting crashdump kernel...
[57585.540042] Bye!

Issue Links

• is related to: LU-11787 sanityn test 71a fails with ‘data is not flushed from client’