Console [ltest1] connected to at 01-18 08:01. 2012-01-18 08:14:35 Lustre: 4614:0:(gss_keyring.c:970:gss_sec_gc_ctx_kr()) running gc 2012-01-18 09:44:34 Lustre: 4614:0:(gss_keyring.c:970:gss_sec_gc_ctx_kr()) running gc 2012-01-18 10:44:34 Lustre: 4614:0:(gss_keyring.c:970:gss_sec_gc_ctx_kr()) running gc 2012-01-18 12:14:33 Lustre: 4614:0:(gss_keyring.c:970:gss_sec_gc_ctx_kr()) running gc 2012-01-18 13:16:09 Lustre: 31959:0:(sec_gss.c:345:cli_ctx_expire()) ctx ffff81040ac403c0(77602->extenci-MDT0000_UUID) get expired: 1326905442(-8732s) 2012-01-18 13:16:10 Lustre: 31959:0:(sec.c:477:sptlrpc_req_ctx_switch()) req ffff81016a119c00: switch ctx ffff81040ac403c0(77602->extenci-MDT0000_UUID) -> ffff81021f7b4b80(77602->extenci-MDT0000_UUID), switc h sec ffff81021e4fc0c0(gss.keyring) -> ffff81021e4fc0c0(gss.keyring) 2012-01-18 13:16:10 Lustre: 31959:0:(sec.c:477:sptlrpc_req_ctx_switch()) Skipped 5 previous similar messages 2012-01-18 13:16:11 LustreError: 31962:0:(gss_keyring.c:1338:gss_kt_update()) negotiation: rpc err 0, gss err d0000 2012-01-18 13:16:11 Lustre: 31962:0:(sec_gss.c:345:cli_ctx_expire()) ctx ffff81021f7b4b80(77602->extenci-MDT0000_UUID) get expired: 1326914375(+200s) 2012-01-18 13:16:11 LustreError: 31959:0:(mdc_locks.c:719:mdc_enqueue()) ldlm_cli_enqueue: -1 2012-01-18 13:16:11 Lustre: 31959:0:(sec_gss.c:1256:gss_cli_ctx_fini_common()) gss.keyring@ffff81021e4fc0c0: destroy ctx ffff81021f7b4b80(77602->extenci-MDT0000_UUID) 2012-01-18 13:16:11 LustreError: 31959:0:(dir.c:410:ll_get_dir_page()) lock enqueue: [0x200004a7c:0x3:0x0] at 0: rc -1 2012-01-18 13:16:11 LustreError: 31959:0:(dir.c:625:ll_readdir()) error reading dir [0x200004a7c:0x3:0x0] at 0: rc -1 2012-01-18 13:16:11 LustreError: 31965:0:(gss_keyring.c:1338:gss_kt_update()) negotiation: rpc err 0, gss err d0000 2012-01-18 13:16:11 Lustre: 31959:0:(sec_gss.c:1256:gss_cli_ctx_fini_common()) gss.keyring@ffff81021e4fc0c0: destroy ctx ffff81021fcfb9c0(77602->extenci-MDT0000_UUID) 2012-01-18 13:16:11 LustreError: 31959:0:(file.c:157:ll_close_inode_openhandle()) inode 144115507983810563 mdc close failed: rc = -1 2012-01-18 13:16:13 LustreError: 31971:0:(gss_keyring.c:1338:gss_kt_update()) negotiation: rpc err 0, gss err d0000 2012-01-18 13:16:13 LustreError: 31971:0:(gss_keyring.c:1338:gss_kt_update()) Skipped 1 previous similar message 2012-01-18 13:16:13 Lustre: 31971:0:(sec_gss.c:345:cli_ctx_expire()) ctx ffff810408503a80(77602->extenci-MDT0000_UUID) get expired: 1326914378(+200s) 2012-01-18 13:16:13 Lustre: 31971:0:(sec_gss.c:345:cli_ctx_expire()) Skipped 2 previous similar messages 2012-01-18 13:16:13 LustreError: 4871:0:(mdc_locks.c:719:mdc_enqueue()) ldlm_cli_enqueue: -1 2012-01-18 13:16:13 Lustre: 4871:0:(sec_gss.c:1256:gss_cli_ctx_fini_common()) gss.keyring@ffff81021e4fc0c0: destroy ctx ffff810408503a80(77602->extenci-MDT0000_UUID) 2012-01-18 13:16:14 Lustre: 4871:0:(sec_gss.c:1256:gss_cli_ctx_fini_common()) Skipped 1 previous similar message 2012-01-18 13:16:14 LustreError: 4871:0:(file.c:2228:ll_inode_revalidate_fini()) failure -1 inode 144115507983810563 2012-01-18 13:16:14 LustreError: 31972:0:(file.c:2228:ll_inode_revalidate_fini()) failure -1 inode 15728641 2012-01-18 13:16:17 Lustre: 31985:0:(sec_gss.c:405:gss_cli_ctx_uptodate()) client refreshed ctx ffff8104111db780 idx 0xbf75804b3ef9f34d (77602->extenci-MDT0000_UUID), expiry 1327004169(+89987s) 2012-01-18 13:16:17 Lustre: 31985:0:(sec_gss.c:405:gss_cli_ctx_uptodate()) Skipped 5 previous similar messages 2012-01-18 13:44:32 Lustre: 4614:0:(gss_keyring.c:970:gss_sec_gc_ctx_kr()) running gc 2012-01-18 13:59:43 Lustre: 32632:0:(sec_gss.c:345:cli_ctx_expire()) ctx ffff81021fcfb4c0(77602->extenci-MDT0000_UUID) get expired: 1326905442(-11346s) 2012-01-18 13:59:43 Lustre: 32632:0:(sec_gss.c:345:cli_ctx_expire()) Skipped 2 previous similar messages 2012-01-18 13:59:44 LustreError: 32632:0:(sec.c:468:sptlrpc_req_ctx_switch()) ASSERTION(req->rq_reqmsg) failed 2012-01-18 13:59:44 LustreError: 32632:0:(sec.c:468:sptlrpc_req_ctx_switch()) LBUG 2012-01-18 13:59:44 Pid: 32632, comm: ll_sa_32631 2012-01-18 13:59:44 2012-01-18 13:59:44 Call Trace: 2012-01-18 13:59:44 [] libcfs_debug_dumpstack+0x51/0x60 [libcfs] 2012-01-18 13:59:44 [] lbug_with_loc+0x7a/0xd0 [libcfs] 2012-01-18 13:59:44 [] cfs_tracefile_init+0x0/0x10a [libcfs] 2012-01-18 13:59:44 [] sptlrpc_req_replace_dead_ctx+0x24c/0xad0 [ptlrpc] 2012-01-18 13:59:44 [] __sched_text_start+0xf6/0xbd0 2012-01-18 13:59:44 [] sptlrpc_import_sec_ref+0x1c/0x30 [ptlrpc] 2012-01-18 13:59:44 [] import_sec_validate_get+0xd8/0x1f0 [ptlrpc] 2012-01-18 13:59:44 [] sptlrpc_req_refresh_ctx+0x5a9/0x9b0 [ptlrpc] 2012-01-18 13:59:44 [] sptlrpc_cli_ctx_wakeup+0x15/0xb0 [ptlrpc] 2012-01-18 13:59:44 [] sptlrpc_import_check_ctx+0x479/0x5e0 [ptlrpc] 2012-01-18 13:59:44 [] ldlm_lock_addref_internal_nolock+0x3a/0x90 [ptlrpc] 2012-01-18 13:59:44 [] ldlm_lock_match+0x72e/0x8b0 [ptlrpc] 2012-01-18 13:59:44 [] __rmqueue+0x44/0xc6 2012-01-18 13:59:44 [] sched_balance_self+0x276/0x2f0 2012-01-18 13:59:44 [] mdc_lock_match+0xf3/0x1a0 [mdc] 2012-01-18 13:59:44 [] lmv_lock_match+0x44f/0x5e0 [lmv] 2012-01-18 13:59:44 [] __next_cpu+0x19/0x28 2012-01-18 13:59:44 [] find_busiest_group+0x20d/0x621 2012-01-18 13:59:44 [] ll_get_dir_page+0x3e7/0x1a30 [lustre] 2012-01-18 13:59:44 [] thread_return+0x62/0xfe 2012-01-18 13:59:44 [] __wake_up+0x38/0x4f 2012-01-18 13:59:44 [] ll_statahead_thread+0x455/0x2810 [lustre] 2012-01-18 13:59:44 [] default_wake_function+0x0/0xe 2012-01-18 13:59:44 [] audit_syscall_exit+0x329/0x344 2012-01-18 13:59:44 [] ll_statahead_thread+0x0/0x2810 [lustre] 2012-01-18 13:59:44 [] child_rip+0xa/0x11 2012-01-18 13:59:44 [] ll_statahead_thread+0x0/0x2810 [lustre] 2012-01-18 13:59:44 [] ll_statahead_thread+0x0/0x2810 [lustre] 2012-01-18 13:59:44 [] child_rip+0x0/0x11 2012-01-18 13:59:44 2012-01-18 13:59:44 Kernel panic - not syncing: LBUG 2012-01-18 13:59:44 <1>LustreError: dumping log to /tmp/lustre-log.1326916789.32632