[LU-6749] kernel panic during umount

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.8.0
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Di Wang

Description

I met this a few times on master branch on my local test. what I did is
1. MDSCOUNT=4 sh llmount.
2. sh llmountcleanup.sh
3. MDSCOUNT=4 sh llmountcleanup.sh

<4>Lustre: 71545:0:(client.c:2003:ptlrpc_expire_one_request()) Skipped 5 previous similar messages
<3>LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
<3>LustreError: Skipped 40 previous similar messages
<3>LustreError: 124973:0:(llog.c:155:llog_cancel_rec()) lustre-MDT0000-osp-MDT0001: fail to write header for llog #0x1:1025#00000000: rc = -5
<3>LustreError: 11-0: lustre-MDT0001-osp-MDT0003: operation obd_ping to node 0@lo failed: rc = -107
<4>Lustre: lustre-MDT0001-osp-MDT0002: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
<4>Lustre: Skipped 2 previous similar messages
<6>Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping)
<6>Lustre: Skipped 2 previous similar messages
<3>LustreError: Skipped 3 previous similar messages
<4>general protection fault: 0000 [#1] SMP
<4>last sysfs file: /sys/devices/system/cpu/possible
<4>CPU 6
<4>Modules linked in: zfs(P)(U) zcommon(P)(U) znvpair(P)(U) zavl(P)(U) zunicode(P)(U) spl(U) zlib_deflate ofd(U) osp(U) lod(U) mdt(U) mdd(U) osd_ldiskfs(U) ldiskfs(U) exportfs lquota(U) lfsck(U) jbd mgc(U) fid(U) fld(U) ptlrpc(U) obdclass(U) ksocklnd(U) lnet(U) sha512_generic crc32c_intel libcfs(U) rfcomm ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle sco bridge bnep l2cap autofs4 nfs lockd fscache auth_rpcgss nfs_acl sunrpc 8021q garp stp llc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vhost_net macvtap macvlan tun kvm_intel kvm uinput microcode vmware_balloon btusb bluetooth rfkill snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc e1000 sg i2c_piix4 i2c_core shpchp ext4 jbd2 mbcache sd_mod crc_t10dif sr_mod cdrom mptspi mptscsih mptbase scsi_transport_spi pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: lmv]
<4>
<4>Pid: 124973, comm: umount Tainted: P           ---------------    2.6.32-504.3.3.el6_lustre.gf8babaf.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
<4>RIP: 0010:[<ffffffff81293ec0>]  [<ffffffff81293ec0>] strchr+0x0/0x30
<4>RSP: 0018:ffff880238b8b7b0  EFLAGS: 00010206
<4>RAX: ffffffff81adee60 RBX: ffff880238b8b800 RCX: 0000000000000000
<4>RDX: ffff880238b8b810 RSI: 000000000000002f RDI: 5a5a5a5a5a5a5a5a
<4>RBP: ffff880238b8b7e8 R08: 0000000000000002 R09: 0000000000000000
<4>R10: ffff88023aeefaa0 R11: 0000000000000008 R12: 5a5a5a5a5a5a5a5a
<4>R13: 5a5a5a5a5a5a5a5a R14: 5a5a5a5a5a5a5a5a R15: ffff880238b8b810
<4>FS:  00007faa4b608740(0000) GS:ffff88002f6c0000(0000) knlGS:0000000000000000
<4>CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<4>CR2: 00007f9427e09000 CR3: 000000019a54a000 CR4: 00000000001407e0
<4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>Process umount (pid: 124973, threadinfo ffff880238b8a000, task ffff88023951cae0)
<4>Stack:
<4> ffffffff811ff7f5 ffff880238b8b828 5a5a5a5a5a5a5a5a ffff8801bc39c138
<4><d> ffffffffa10e30a0 ffff880238b8b8b8 0000000000000000 ffff880238b8b838
<4><d> ffffffff8120060b 00000000000000a0 5a5a5a5a5a5a5a5a ffff8801b9eb92c0
<4>Call Trace:
<4> [<ffffffff811ff7f5>] ? __xlate_proc_name+0x45/0xf0
<4> [<ffffffff8120060b>] remove_proc_subtree+0x3b/0x180
<4> [<ffffffffa06d1cab>] proc_remove+0x1b/0x20 [obdclass]
<4> [<ffffffffa06d1cc9>] lprocfs_remove+0x19/0x30 [obdclass]
<4> [<ffffffffa10b9b53>] lod_procfs_fini+0x33/0x70 [lod]
<4> [<ffffffffa10ac3f6>] lod_device_fini+0xd6/0x220 [lod]
<4> [<ffffffffa06e6ac2>] class_cleanup+0x552/0xd10 [obdclass]
<4> [<ffffffffa06c7136>] ? class_name2dev+0x56/0xe0 [obdclass]
<4> [<ffffffffa06e926a>] class_process_config+0x1fea/0x27c0 [obdclass]
<4> [<ffffffff81174f4c>] ? __kmalloc+0x20c/0x220
<4> [<ffffffffa06e2225>] ? lustre_cfg_new+0x435/0x630 [obdclass]
<4> [<ffffffffa06e9b61>] class_manual_cleanup+0x121/0x870 [obdclass]
<4> [<ffffffffa06c62b8>] ? class_disconnect+0xa8/0x4a0 [obdclass]
<4> [<ffffffffa10ac88a>] lod_obd_disconnect+0x12a/0x1f0 [lod]
<4> [<ffffffffa0f70501>] mdd_process_config+0x331/0x5d0 [mdd]
<4> [<ffffffffa0fe5138>] mdt_stack_fini+0x718/0x1240 [mdt]
<4> [<ffffffffa0fe6570>] mdt_device_fini+0x910/0x1370 [mdt]
<4> [<ffffffffa06ca366>] ? class_disconnect_exports+0x116/0x2f0 [obdclass]
<4> [<ffffffffa06e6ac2>] class_cleanup+0x552/0xd10 [obdclass]
<4> [<ffffffffa06c7136>] ? class_name2dev+0x56/0xe0 [obdclass]
<4> [<ffffffffa06e926a>] class_process_config+0x1fea/0x27c0 [obdclass]
<4> [<ffffffff81174f4c>] ? __kmalloc+0x20c/0x220
<4> [<ffffffffa06e2225>] ? lustre_cfg_new+0x435/0x630 [obdclass]
<4> [<ffffffffa06e9b61>] class_manual_cleanup+0x121/0x870 [obdclass]
<4> [<ffffffffa06c7136>] ? class_name2dev+0x56/0xe0 [obdclass]
<4> [<ffffffffa07225b7>] server_put_super+0xb17/0xea0 [obdclass]
<4> [<ffffffff8119082b>] generic_shutdown_super+0x5b/0xe0
<4> [<ffffffff81190916>] kill_anon_super+0x16/0x60
<4> [<ffffffffa06ebdc6>] lustre_kill_super+0x36/0x60 [obdclass]
<4> [<ffffffff811910b7>] deactivate_super+0x57/0x80
<4> [<ffffffff811b0cef>] mntput_no_expire+0xbf/0x110
<4> [<ffffffff811b183b>] sys_umount+0x7b/0x3a0
<4> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
<4>Code: 75 19 48 83 e9 01 84 c0 74 06 48 83 ea 01 75 db 31 c0 c9 c3 0f 1f 80 00 00 00 00 44 38 c0 c9 19 c0 83 c8 01 c3 66 0f 1f 44 00 00 <0f> b6 17 55 48 89 f8 48 89 e5 40 38 f2 75 15 eb 19 0f 1f 80 00
<1>RIP  [<ffffffff81293ec0>] strchr+0x0/0x30
<4> RSP <ffff880238b8b7b0>

Comments

[Joseph Gmitter (Inactive)]

Landed for 2.8.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/16011/
Subject: LU-6749 lod: properly remove proc entry
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: f31d05319bae3640ba9bb047f842d6f12723cf7b

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: http://review.whamcloud.com/16011
Subject: LU-6749 lod: properly remove proc entry
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c6a501a8f4669ffe4cf1b4b48ea7286a098b21bf

[Joseph Gmitter (Inactive)]

Hi Bobijam,
Can you look into this?
Thanks.
Joe