[LU-12879] PPC client: sanity-flr test_0c: client crash

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.12.3, Lustre 2.12.4
• Type: Bug
• Priority: Minor
• Reporter: Maloo
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: ppc
• Severity: 3

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/f5ef765a-eb0e-11e9-b62b-52540065bddc

test_0c failed with the following error:

trevis-77vm1 crashed during sanity-flr test_0c

client crash

[ 4200.063488] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0b 2>/dev/null || echo foo
[ 4200.072615] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0b 2>/dev/null || echo foo
[ 4200.259777] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-flr test 0c: lfs mirror create composite layout mirrors ==================================== 01:22:56 \(1570584176\)
[ 4200.451977] Lustre: DEBUG MARKER: == sanity-flr test 0c: lfs mirror create composite layout mirrors ==================================== 01:22:56 (1570584176)
[ 4211.510363] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0c 2>/dev/null || echo foo
[ 4211.519372] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0c 2>/dev/null || echo foo
[ 4222.525313] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0c | sort -u | tr '\n' ' ' 
[ 4222.535472] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_0c | sort -u | tr '\n' ' ' 
[ 4222.563600] LustreError: 12033:0:(pack_generic.c:2364:lustre_swab_lov_comp_md_v1()) Invalid magic 0x1
[ 4222.564219] Unrecoverable VSX Unavailable Exception f40 at d000000002925e84
[ 4222.564283] Oops: Unrecoverable VSX Unavailable Exception, sig: 6 [#1]
[ 4222.564331] SMP NR_CPUS=2048 NUMA pSeries
[ 4222.564375] Modules linked in: lustre(OE) obdecho(OE) mgc(OE) lov(OE) mdc(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod crc_t10dif crct10dif_generic crct10dif_common ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_core virtio_balloon auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 virtio_net virtio_blk virtio_pci virtio_ring virtio
[ 4222.565015] CPU: 1 PID: 12033 Comm: lfs Kdump: loaded Tainted: G           OE  ------------   3.10.0-957.27.2.el7.ppc64 #1
[ 4222.565086] task: c000000002d06cc0 ti: c00000003ef5c000 task.ti: c00000003ef5c000
[ 4222.565139] NIP: d000000002925e84 LR: d000000003504438 CTR: d000000002925e80
[ 4222.565193] REGS: c00000003ef5f3e0 TRAP: 0f40   Tainted: G           OE  ------------    (3.10.0-957.27.2.el7.ppc64)
[ 4222.565263] MSR: 8000000000009032 <SF,EE,ME,IR,DR,RI>  CR: 28044444  XER: 20000000
[ 4222.565389] CFAR: d0000000035d5bd0 SOFTE: 1 
GPR00: d0000000034f7cbc c00000003ef5f660 d00000000295a974 c0000000bc82f180 
GPR04: c00000003ef5f930 c00000003ef5f6d0 000000000000000d 0000000000000000 
GPR08: 0000000000000000 0000000200000407 0000000000003000 d0000000035d5bb8 
GPR12: d000000002925e80 c000000007b80900 d00000000294da34 0000000000000000 
GPR16: 0000000000000000 0000000000000ca2 c00000003ef5fb60 c00000003ef5fb30 
GPR20: c00000003ef5f930 000000000000000d d00000000294da30 0000000000003740 
GPR24: d00000000294da34 c0000000b4535000 000000000000000d 0000000000000000 
GPR28: c0000000bc82f180 c00000003ef5f930 c0000000b4535000 c00000003ef5f660 
[ 4222.566095] NIP [d000000002925e84] .cfs_hash_bd_get+0x4/0xb0 [libcfs]
[ 4222.566190] LR [d000000003504438] .ldlm_resource_get+0x98/0xc30 [ptlrpc]
[ 4222.566235] Call Trace:
[ 4222.566255] [c00000003ef5f660] [c00000003ef5f6f0] 0xc00000003ef5f6f0 (unreliable)
[ 4222.566341] [c00000003ef5f740] [d0000000034f7cbc] .ldlm_lock_match_with_skip+0x10c/0xa70 [ptlrpc]
[ 4222.566425] [c00000003ef5f8b0] [d000000003d4b1bc] .mdc_lock_match+0x10c/0x270 [mdc]
[ 4222.566490] [c00000003ef5f9a0] [d000000003a330d8] .lmv_lock_match+0x398/0x770 [lmv]
[ 4222.566583] [c00000003ef5fac0] [d0000000042b13dc] .ll_file_release+0x6fc/0xc50 [lustre]
[ 4222.566655] [c00000003ef5fbc0] [d000000004293fec] .ll_dir_release+0x15c/0x200 [lustre]
[ 4222.566757] [c00000003ef5fc60] [c000000000377d9c] .____fput+0xdc/0x2f0
[ 4222.566819] [c00000003ef5fd10] [c000000000136048] .task_work_run+0xe8/0x160
[ 4222.566875] [c00000003ef5fdb0] [c000000000020078] .do_notify_resume+0xe8/0x100
[ 4222.566938] [c00000003ef5fe30] [c00000000000cd30] .ret_from_except_lite+0x5c/0x60
[ 4222.566999] Instruction dump:
[ 4222.567027] 0000423d 408daae8 c805e93b 0400403d 78fbe37f e0054991 0548ff4b 78fbe37f 
[ 4222.567122] 00000060 d9adfe4b 00000060 a602087c <f0ffc1fb> 100001f8 f8ffe1fb 71ff21f8 
[ 4222.567226] ---[ end trace 9a04feb90e321f93 ]---
[ 4222.570018] 
[ 4222.570054] Sending IPI to other CPUs
[ 4222.571089] IPI complete

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-flr test_0c - trevis-77vm1 crashed during sanity-flr test_0c

Comments

[Sarah Liu]

another one on PPC client
https://testing.whamcloud.com/test_sets/0e3ae6cc-eb0f-11e9-b62b-52540065bddc