Details
-
Bug
-
Resolution: Fixed
-
Minor
-
Lustre 2.9.0
-
None
-
3
-
9223372036854775807
Description
Got a list_del corruption message after changing the shared key:
[ 987.235627] Lustre: 2444:0:(tgt_handler.c:894:tgt_init_sec_level()) client 192.168.122.32@tcp -> target test-MDT0000 uses old version, run under security level 0. [ 987.237530] Lustre: 2444:0:(sec_gss.c:2346:gss_svc_handle_destroy()) destroy svc ctx ffff8800112d2e40 idx 0xa62e511956836924 (0->192.168.122.32@tcp) [ 987.700506] Lustre: test-MDT0000: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 1010.991408] Lustre: 2500:0:(sec_gss.c:394:gss_cli_ctx_uptodate()) server installed reverse ctx ffff8800112cd3c0 idx 0xa62e511956836921, expiry 1466276335(+604750s) [ 1010.991421] Lustre: 2500:0:(sec_gss.c:394:gss_cli_ctx_uptodate()) Skipped 2 previous similar messages [ 1010.991438] Lustre: test-MDT0000: Connection restored to 155649eb-9bd7-edb3-7b65-66a22f0de548 (at 0@lo) [ 1010.991443] Lustre: Skipped 2 previous similar messages [ 1012.247047] Lustre: 2500:0:(sec_gss.c:2086:gss_svc_handle_init()) create svc ctx ffff880011a8c240: user from 192.168.122.32@tcp authenticated as oss [ 4560.355291] Lustre: 2416:0:(gss_keyring.c:1038:gss_sec_gc_ctx_kr()) running gc [ 8160.355220] Lustre: 2416:0:(gss_keyring.c:1038:gss_sec_gc_ctx_kr()) running gc [ 9968.634019] Lustre: 2444:0:(sec_gss.c:2346:gss_svc_handle_destroy()) destroy svc ctx ffff8800364b0040 idx 0xa62e51195683692a (0->192.168.122.35@tcp) [ 9968.634023] Lustre: 2444:0:(sec_gss.c:2346:gss_svc_handle_destroy()) Skipped 1 previous similar message [ 9968.802451] Lustre: 2416:0:(sec_gss.c:1245:gss_cli_ctx_fini_common()) reverse sec ffff88003bbc6000: destroy ctx ffff8800112cd3c0 [ 9968.802624] Lustre: Failing over test-MDT0000 [ 9968.803134] Lustre: 2416:0:(gss_cli_upcall.c:400:gss_do_ctx_fini_rpc()) client finishing forward ctx ffff88001129ff00 idx 0xa62e511956836922 (0->test-MDT0000_UUID) [ 9968.803200] Lustre: 2416:0:(sec_gss.c:1249:gss_cli_ctx_fini_common()) gss.keyring@ffff88003db67300: destroy ctx ffff88001129ff00(0->test-MDT0000_UUID) [ 9968.866112] ------------[ cut here ]------------ [ 9968.866121] WARNING: at lib/list_debug.c:59 __list_del_entry+0xa1/0xd0() [ 9968.866123] list_del corruption. prev->next should be ffff88003c2e7338, but was ffff88003db67338 [ 9968.866124] Modules linked in: ptlrpc_gss(OE) osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) sunrpc mbcache jbd2 sha512_generic crypto_null snd_hda_codec_generic crc32_pclmul crc32c_intel ppdev ghash_clmulni_intel snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_seq snd_seq_device serio_raw pcspkr virtio_balloon snd_pcm parport_pc parport snd_timer snd soundcore i2c_piix4 9pnet_virtio(OE) 9p(OE) 9pnet(OE) xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net virtio_console virtio_scsi qxl syscopyarea sysfillrect sysimgblt drm_kms_helper ttm ata_piix crct10dif_pclmul crct10dif_common [ 9968.866177] virtio_pci virtio_ring virtio drm i2c_core libata floppy dm_mirror dm_region_hash dm_log dm_mod loop [last unloaded: libcfs] [ 9968.866189] CPU: 0 PID: 2717 Comm: umount Tainted: G W OE ------------ 3.10.0-327.13.1.el7_lustre.x86_64 #1 [ 9968.866191] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 [ 9968.866193] ffff88003633f520 00000000e3385889 ffff88003633f4d8 ffffffff81635924 [ 9968.866197] ffff88003633f510 ffffffff8107b1e0 ffff88003c2e7300 ffff88003c2e7338 [ 9968.866200] ffff88003af7a338 ffff88003af79eb0 ffff88001dc56000 ffff88003633f578 [ 9968.866204] Call Trace: [ 9968.866211] [<ffffffff81635924>] dump_stack+0x19/0x1b [ 9968.866216] [<ffffffff8107b1e0>] warn_slowpath_common+0x70/0xb0 [ 9968.866219] [<ffffffff8107b27c>] warn_slowpath_fmt+0x5c/0x80 [ 9968.866276] [<ffffffffa081bf3c>] ? ptlrpc_invalidate_import+0x23c/0x8e0 [ptlrpc] [ 9968.866280] [<ffffffff8130c721>] __list_del_entry+0xa1/0xd0 [ 9968.866323] [<ffffffffa0837579>] sptlrpc_gc_del_sec+0x39/0xf0 [ptlrpc] [ 9968.866362] [<ffffffffa082c3e9>] sptlrpc_sec_put+0x29/0x70 [ptlrpc] [ 9968.866402] [<ffffffffa083020a>] sptlrpc_import_sec_put+0x2a/0x40 [ptlrpc] [ 9968.866435] [<ffffffffa07bdc52>] client_destroy_import+0x22/0x30 [ptlrpc] [ 9968.866468] [<ffffffffa07befcc>] client_obd_cleanup+0x8c/0x3c0 [ptlrpc] [ 9968.866479] [<ffffffffa0d902f4>] osp_device_fini+0x244/0x540 [osp] [ 9968.866515] [<ffffffffa05a638c>] class_cleanup+0x8dc/0xd70 [obdclass] [ 9968.866541] [<ffffffffa05a8d64>] class_process_config+0x1e24/0x2f60 [obdclass] [ 9968.866544] [<ffffffff811c1f53>] ? __kmalloc+0x1f3/0x230 [ 9968.866567] [<ffffffffa05a228b>] ? lustre_cfg_new+0x8b/0x400 [obdclass] [ 9968.866590] [<ffffffffa05a9f8f>] class_manual_cleanup+0xef/0x810 [obdclass] [ 9968.866599] [<ffffffffa0d8f268>] osp_obd_disconnect+0x178/0x210 [osp] [ 9968.866611] [<ffffffffa0cdaf48>] lod_putref+0x258/0x960 [lod] [ 9968.866620] [<ffffffffa0cdd0dd>] lod_fini_tgt+0xdd/0x3a0 [lod] [ 9968.866630] [<ffffffffa0cd1c70>] lod_device_fini+0x60/0x1f0 [lod] [ 9968.866652] [<ffffffffa05a638c>] class_cleanup+0x8dc/0xd70 [obdclass] [ 9968.866672] [<ffffffffa05a8d64>] class_process_config+0x1e24/0x2f60 [obdclass] [ 9968.866676] [<ffffffff811c1f53>] ? __kmalloc+0x1f3/0x230 [ 9968.866697] [<ffffffffa05a228b>] ? lustre_cfg_new+0x8b/0x400 [obdclass] [ 9968.866718] [<ffffffffa05a9f8f>] class_manual_cleanup+0xef/0x810 [obdclass] [ 9968.866730] [<ffffffffa0cd1bff>] lod_obd_disconnect+0x1bf/0x1d0 [lod] [ 9968.866740] [<ffffffffa0d3a0ba>] mdd_process_config+0x2ba/0x5c0 [mdd] [ 9968.866754] [<ffffffffa0bfa743>] mdt_stack_fini+0x1b3/0xe00 [mdt] [ 9968.866766] [<ffffffffa0bfbab6>] mdt_device_fini+0x726/0xfc0 [mdt] [ 9968.866788] [<ffffffffa05a638c>] class_cleanup+0x8dc/0xd70 [obdclass] [ 9968.866821] [<ffffffffa05a8d64>] class_process_config+0x1e24/0x2f60 [obdclass] [ 9968.866825] [<ffffffff811c1f53>] ? __kmalloc+0x1f3/0x230 [ 9968.866851] [<ffffffffa05a228b>] ? lustre_cfg_new+0x8b/0x400 [obdclass] [ 9968.866872] [<ffffffffa05a9f8f>] class_manual_cleanup+0xef/0x810 [obdclass] [ 9968.866900] [<ffffffffa05da60e>] server_put_super+0x8de/0xcd0 [obdclass] [ 9968.866904] [<ffffffff811e0ad6>] generic_shutdown_super+0x56/0xe0 [ 9968.866907] [<ffffffff811e0eb2>] kill_anon_super+0x12/0x20 [ 9968.866929] [<ffffffffa05ad6f2>] lustre_kill_super+0x32/0x50 [obdclass] [ 9968.866933] [<ffffffff811e1269>] deactivate_locked_super+0x49/0x60 [ 9968.866936] [<ffffffff811e1866>] deactivate_super+0x46/0x60 [ 9968.866939] [<ffffffff811fe7d5>] mntput_no_expire+0xc5/0x120 [ 9968.866943] [<ffffffff811ff90f>] SyS_umount+0x9f/0x3c0 [ 9968.866947] [<ffffffff816460c9>] system_call_fastpath+0x16/0x1b [ 9968.866950] ---[ end trace 8136914c4f134ff8 ]--- [ 9968.867903] ------------[ cut here ]------------