[LU-627] LBUG: (lproc_llite.c:852:lprocfs_register_mountpoint()) ASSERTION(obd != NULL) failed

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.1.0
• Fix Version/s: Lustre 2.1.0
• Type: Bug
• Priority: Critical
• Reporter: Jian Yu
• Assignee: Robert Read (Inactive)
• Resolution: Fixed

Environment

Lustre Tag: v2_1_0_0_RC0
Lustre Build: http://newbuild.whamcloud.com/job/lustre-master/267/
Distro/Arch: RHEL6/x86_64 (kernel version: 2.6.32-131.6.1.el6)

• Severity: 3

Description

While running conf-sanity test 24a, the following LBUG occurred on Lustre client (client-1-ib):

Lustre: DEBUG MARKER: == conf-sanity test 24a: Multiple MDTs on a single node == 20:12:46 (1314155566)
Lustre: 8366:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import MGC192.168.4.134@o2ib->MGC192.168.4.134@o2ib_0 netid 50000: select flavor null
LustreError: 152-6: Ignoring deprecated mount option 'acl'.
Lustre: MGC192.168.4.134@o2ib: Reactivating import
Lustre: 8366:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import lustre-MDT0000-mdc-ffff880312bde000->192.168.4.134@o2ib netid 50000: select flavor null
LustreError: 11-0: an error occurred while communicating with 192.168.4.134@o2ib. The mds_connect operation failed with -11
LustreError: 11-0: an error occurred while communicating with 192.168.4.134@o2ib. The mds_connect operation failed with -11
Lustre: 8366:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import lustre-OST0000-osc-ffff880312bde000->192.168.4.135@o2ib netid 50000: select flavor null
Lustre: Client lustre-client has started
Lustre: 8389:0:(debug.c:326:libcfs_debug_str2mask()) You are trying to use a numerical value for the mask - this will be deprecated in a future release.
LustreError: 8471:0:(lproc_llite.c:852:lprocfs_register_mountpoint()) ASSERTION(obd != NULL) failed
LustreError: 8471:0:(lproc_llite.c:852:lprocfs_register_mountpoint()) LBUG
Pid: 8471, comm: mount.lustre

Call Trace:
 [<ffffffffa0977855>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
 [<ffffffffa0977e95>] lbug_with_loc+0x75/0xe0 [libcfs]
 [<ffffffffa0982d76>] libcfs_assertion_failed+0x66/0x70 [libcfs]
 [<ffffffffa07460ea>] lprocfs_register_mountpoint+0xbaa/0xbb0 [lustre]
 [<ffffffffa0a32840>] ? lprocfs_rd_uuid+0x0/0x80 [obdclass]
 [<ffffffff81098a1a>] ? do_gettimeofday+0x1a/0x50
 [<ffffffffa073628e>] client_common_fill_super+0x1ae/0x5720 [lustre]
 [<ffffffff8126c9e8>] ? vsnprintf+0x198/0x5f0
 [<ffffffff8126ce80>] ? sprintf+0x40/0x50
 [<ffffffffa073c353>] ll_fill_super+0xb53/0x1580 [lustre]
 [<ffffffffa0a5a34a>] lustre_fill_super+0x10a/0x1280 [obdclass]
 [<ffffffff811753e9>] ? sget+0x3f9/0x490
 [<ffffffff81174420>] ? set_anon_super+0x0/0x100
 [<ffffffffa0a5a240>] ? lustre_fill_super+0x0/0x1280 [obdclass]
 [<ffffffff8117592f>] get_sb_nodev+0x5f/0xa0
 [<ffffffffa0a4b275>] lustre_get_sb+0x25/0x30 [obdclass]
 [<ffffffff8117558b>] vfs_kern_mount+0x7b/0x1b0
 [<ffffffff81175732>] do_kern_mount+0x52/0x130
 [<ffffffff81193207>] do_mount+0x2e7/0x870
 [<ffffffff81193820>] sys_mount+0x90/0xe0
 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b

Kernel panic - not syncing: LBUG
Pid: 8471, comm: mount.lustre Tainted: G           ---------------- T 2.6.32-131.6.1.el6.x86_64 #1
Call Trace:
 [<ffffffff814da518>] ? panic+0x78/0x143
 [<ffffffffa0977eeb>] ? lbug_with_loc+0xcb/0xe0 [libcfs]
 [<ffffffffa0982d76>] ? libcfs_assertion_failed+0x66/0x70 [libcfs]
 [<ffffffffa07460ea>] ? lprocfs_register_mountpoint+0xbaa/0xbb0 [lustre]
 [<ffffffffa0a32840>] ? lprocfs_rd_uuid+0x0/0x80 [obdclass]
 [<ffffffff81098a1a>] ? do_gettimeofday+0x1a/0x50
 [<ffffffffa073628e>] ? client_common_fill_super+0x1ae/0x5720 [lustre]
 [<ffffffff8126c9e8>] ? vsnprintf+0x198/0x5f0
 [<ffffffff8126ce80>] ? sprintf+0x40/0x50
 [<ffffffffa073c353>] ? ll_fill_super+0xb53/0x1580 [lustre]
 [<ffffffffa0a5a34a>] ? lustre_fill_super+0x10a/0x1280 [obdclass]
 [<ffffffff811753e9>] ? sget+0x3f9/0x490
 [<ffffffff81174420>] ? set_anon_super+0x0/0x100
 [<ffffffffa0a5a240>] ? lustre_fill_super+0x0/0x1280 [obdclass]
 [<ffffffff8117592f>] ? get_sb_nodev+0x5f/0xa0
 [<ffffffffa0a4b275>] ? lustre_get_sb+0x25/0x30 [obdclass]
 [<ffffffff8117558b>] ? vfs_kern_mount+0x7b/0x1b0
 [<ffffffff81175732>] ? do_kern_mount+0x52/0x130
 [<ffffffff81193207>] ? do_mount+0x2e7/0x870
 [<ffffffff81193820>] ? sys_mount+0x90/0xe0
 [<ffffffff8100b172>] ? system_call_fastpath+0x16/0x1b

Maloo report: https://maloo.whamcloud.com/test_sets/6d906f52-ce0d-11e0-8d02-52540025f9af

Comments

[Jian Yu]

conf-sanity test 33a: https://maloo.whamcloud.com/test_sets/8f184d64-ce14-11e0-8d02-52540025f9af
conf-sanity test 36: https://maloo.whamcloud.com/test_sets/1e5901f8-ce33-11e0-8d02-52540025f9af

[Jian Yu]

After the patch for LU-622 was merged on master branch, the issue in this ticket was also fixed:

Lustre Build: http://newbuild.whamcloud.com/job/lustre-master/273/
Distro/Arch: RHEL6/x86_64

conf-sanity test 24 and 33 passed: https://maloo.whamcloud.com/test_sets/d8a3493a-cfbc-11e0-8d02-52540025f9af
conf-sanity test 36 passed: https://maloo.whamcloud.com/test_sets/7ec91302-cfbe-11e0-8d02-52540025f9af