[LU-6974] RHEL 7.1 lustre-initialization-1: MDS crashed while lustre mount

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.8.0
• Environment: 3.10.0-229.11.1.el7_lustre.g853ea39.x86_64
• Severity: 3
• Assignee: Yang Sheng
• Reporter: Maloo

Description

This issue was created by maloo for ys <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/cebbc5a2-3e51-11e5-8006-5254006e85c2.

The sub-test lustre-initialization_1 failed with the following error:

Test system failed to start single suite, so abandoning all hope and giving up

Please provide additional information about the failure here.

Info required for matching: lustre-initialization-1 lustre-initialization_1

MDS crashed:

03:40:55:[ 2331.721214] ldiskfs: module verification failed: signature and/or required key missing - tainting kernel
03:40:55:[ 2331.740707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro
03:40:55:[ 2352.224410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro
03:40:55:[ 2352.319932] BUG: unable to handle kernel paging request at 0000000000001f50
03:40:55:[ 2352.320900] IP: [<ffffffff81155b02>] page_waitqueue+0x62/0x80
03:40:55:[ 2352.320900] PGD 36560067 PUD 7bb3a067 PMD 0 
03:40:55:[ 2352.320900] Oops: 0000 [#1] SMP 
03:40:55:[ 2352.320900] Modules linked in: ldiskfs(OF) dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd fscache xprtrdma sunrpc ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ppdev ib_cm iw_cm serio_raw pcspkr virtio_balloon i2c_piix4 parport_pc parport ib_sa ib_mad ib_core ib_addr ext4 mbcache jbd2 ata_generic pata_acpi cirrus syscopyarea sysfillrect virtio_blk sysimgblt drm_kms_helper 8139too floppy ttm ata_piix drm libata 8139cp mii virtio_pci virtio_ring i2c_core virtio
03:40:55:[ 2352.320900] CPU: 1 PID: 0 Comm: swapper/1 Tainted: GF          O--------------   3.10.0-229.11.1.el7_lustre.g853ea39.x86_64 #1
03:40:55:[ 2352.320900] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
03:40:55:[ 2352.320900] task: ffff88007c052220 ti: ffff88007c080000 task.ti: ffff88007c080000
03:40:55:[ 2352.320900] RIP: 0010:[<ffffffff81155b02>]  [<ffffffff81155b02>] page_waitqueue+0x62/0x80
03:40:55:[ 2352.320900] RSP: 0018:ffff88007fd03d50  EFLAGS: 00010086
03:40:55:[ 2352.320900] RAX: 0000000000001800 RBX: ffff88007c163100 RCX: 0000000000000040
03:40:55:[ 2352.320900] RDX: 97fd97a7b8163100 RSI: 0000000000000000 RDI: 0000000000000000
03:40:55:[ 2352.320900] RBP: ffff88007fd03d50 R08: 8800000000000000 R09: 2001f058c4000000
03:40:55:[ 2352.320900] R10: dffd97a7b8163100 R11: ffffffffa001a265 R12: ffff880035ea3500
03:40:55:[ 2352.320900] R13: 0000000000000001 R14: 0000000000000000 R15: ffff88003631e6c0
03:40:55:[ 2352.320900] FS:  0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
03:40:55:[ 2352.320900] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
03:40:55:[ 2352.320900] CR2: 0000000000001f50 CR3: 000000003652c000 CR4: 00000000000006e0
03:40:55:[ 2352.320900] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
03:40:55:[ 2352.320900] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
03:40:55:[ 2352.320900] Stack:
03:40:55:[ 2352.320900]  ffff88007fd03d68 ffffffff81156f8e ffff880035ea3568 ffff88007fd03d90
03:40:55:[ 2352.320900]  ffffffff8120652e ffff880035ea3500 0000000000000000 0000000000000000
03:40:55:[ 2352.320900]  ffff88007fd03da0 ffffffff811fd9dd ffff88007fd03de0 ffffffff812adf60
03:40:55:[ 2352.320900] Call Trace:
03:40:56:[ 2352.320900]  <IRQ> 
03:40:56:[ 2352.320900]  [<ffffffff81156f8e>] unlock_page+0x1e/0x30
03:40:56:[ 2352.320900]  [<ffffffff8120652e>] mpage_end_io+0x3e/0xb0
03:40:56:[ 2352.320900]  [<ffffffff811fd9dd>] bio_endio+0x1d/0x30
03:40:56:[ 2352.320900]  [<ffffffff812adf60>] blk_update_request+0x90/0x350
03:40:56:[ 2352.320900]  [<ffffffff812b708a>] blk_mq_end_request+0x1a/0x70
03:40:56:[ 2352.320900]  [<ffffffffa01202f2>] virtblk_request_done+0x32/0x80 [virtio_blk]
03:40:56:[ 2352.320900]  [<ffffffff812b783d>] __blk_mq_complete_request+0x7d/0x100
03:40:56:[ 2352.320900]  [<ffffffff812b78e1>] blk_mq_complete_request+0x21/0x30
03:40:56:[ 2352.320900]  [<ffffffffa0120076>] virtblk_done+0x76/0x100 [virtio_blk]
03:40:56:[ 2352.320900]  [<ffffffffa001a4a8>] vring_interrupt+0x38/0x90 [virtio_ring]
03:40:56:[ 2352.320900]  [<ffffffff8110b84e>] handle_irq_event_percpu+0x3e/0x1e0
03:40:56:[ 2352.320900]  [<ffffffff8110ba2d>] handle_irq_event+0x3d/0x60
03:40:56:[ 2352.320900]  [<ffffffff8110e6c7>] handle_edge_irq+0x77/0x130
03:40:56:[ 2352.320900]  [<ffffffff81015cff>] handle_irq+0xbf/0x150
03:40:57:[ 2352.320900]  [<ffffffff81610b4a>] ? atomic_notifier_call_chain+0x1a/0x20
03:40:57:[ 2352.320900]  [<ffffffff816175ef>] do_IRQ+0x4f/0xf0
03:40:57:[ 2352.320900]  [<ffffffff8160c82d>] common_interrupt+0x6d/0x6d
03:40:57:[ 2352.320900]  <EOI> 
03:40:57:[ 2352.320900]  [<ffffffff8109b938>] ? hrtimer_start+0x18/0x20
03:40:57:[ 2352.320900]  [<ffffffff81052dd6>] ? native_safe_halt+0x6/0x10
03:40:57:[ 2352.320900]  [<ffffffff8101c93f>] default_idle+0x1f/0xc0
03:40:57:[ 2352.320900]  [<ffffffff8101d236>] arch_cpu_idle+0x26/0x30
03:40:57:[ 2352.320900]  [<ffffffff810c6985>] cpu_startup_entry+0xf5/0x290
03:40:57:[ 2352.320900]  [<ffffffff810423ca>] start_secondary+0x1ba/0x230
03:40:57:[ 2352.320900] Code: 48 03 04 d5 80 31 a2 81 48 89 fa 48 c1 e1 39 48 c1 e2 36 48 c1 e7 3f 48 89 e5 4c 01 d2 4c 29 c2 48 01 f2 48 29 ca b9 40 00 00 00 <2b> 88 50 07 00 00 48 01 d7 48 8b 80 40 07 00 00 5d 48 d3 ef 48 
03:40:57:[ 2352.320900] RIP  [<ffffffff81155b02>] page_waitqueue+0x62/0x80
03:40:57:[ 2352.320900]  RSP <ffff88007fd03d50>
03:40:57:[ 2352.320900] CR2: 0000000000001f50
03:40:57:[ 2352.320900] ------------[ cut here ]------------
03:40:57:[ 2352.320900] kernel BUG at mm/vmalloc.c:1339!
03:40:57:[ 2352.320900] invalid opcode: 0000 [#2] SMP 
03:40:57:[ 2352.320900] Modules linked in: ldiskfs(OF) dm_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd fscache xprtrdma sunrpc ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ppdev ib_cm iw_cm serio_raw pcspkr virtio_balloon i2c_piix4 parport_pc parport ib_sa ib_mad ib_core ib_addr ext4 mbcache jbd2 ata_generic pata_acpi cirrus syscopyarea sysfillrect virtio_blk sysimgblt drm_kms_helper 8139too floppy ttm ata_piix drm libata 8139cp mii virtio_pci virtio_ring i2c_core virtio
03:40:57:[ 2352.320900] CPU: 1 PID: 0 Comm: swapper/1 Tainted: GF          O--------------   3.10.0-229.11.1.el7_lustre.g853ea39.x86_64 #1
03:40:57:[ 2352.320900] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
03:40:57:[ 2352.320900] task: ffff88007c052220 ti: ffff88007c080000 task.ti: ffff88007c080000
03:40:57:[ 2352.320900] RIP: 0010:[<ffffffff8119060e>]  [<ffffffff8119060e>] __get_vm_area_node+0x1ce/0x1d0
03:40:57:[ 2352.320900] RSP: 0018:ffff88007fd032b0  EFLAGS: 00010006
03:40:57:[ 2352.320900] RAX: ffff88007c083fd8 RBX: 00000000ffffffff RCX: ffffc90000000000
03:40:57:[ 2352.320900] RDX: 0000000000000022 RSI: 0000000000000001 RDI: 0000000000002000
03:40:57:[ 2352.320900] RBP: ffff88007fd03310 R08: ffffe8ffffffffff R09: 00000000ffffffff
03:40:57:[ 2352.320900] R10: ffff880077e6f480 R11: ffff8800362c48d0 R12: ffffffffa00d42c9
03:40:57:[ 2352.320900] R13: 0000000000001200 R14: 00000000000080d2 R15: ffffea0000d8e940
03:40:57:[ 2352.320900] FS:  0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
03:40:57:[ 2352.320900] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
03:40:57:[ 2352.320900] CR2: 0000000000001f50 CR3: 000000003652c000 CR4: 00000000000006e0
03:40:57:[ 2352.320900] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
03:40:57:[ 2352.320900] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
03:40:57:[ 2352.320900] Stack:
03:40:57:[ 2352.320900]  ffffffff81191d7d 00000000000080d2 ffffffffa00d42c9 8000000000000163
03:40:57:[ 2352.320900]  000080d200000000 0000000000000000 681e1a9a428ae35b ffff880077e6f480
03:40:57:[ 2352.320900]  ffff8800362c48b0 0000000000240000 0000000000000080 ffffea0000d8e940
03:40:57:[ 2352.320900] Call Trace:
03:40:57:[ 2352.320900]  <IRQ> 
03:40:57:[ 2352.320900]  [<ffffffff81191d7d>] ? __vmalloc_node_range+0x7d/0x270
03:40:57:[ 2352.320900]  [<ffffffffa00d42c9>] ? ttm_tt_init+0x69/0xb0 [ttm]
03:40:58:[ 2352.320900]  [<ffffffff81191fb1>] __vmalloc+0x41/0x50
03:40:58:[ 2352.320900]  [<ffffffffa00d42c9>] ? ttm_tt_init+0x69/0xb0 [ttm]
03:40:58:[ 2352.320900]  [<ffffffffa00d42c9>] ttm_tt_init+0x69/0xb0 [ttm]
03:40:58:[ 2352.320900]  [<ffffffffa013b4e8>] cirrus_ttm_tt_create+0x58/0x90 [cirrus]
03:40:58:[ 2352.320900]  [<ffffffffa00d4a7d>] ttm_bo_add_ttm+0x8d/0xc0 [ttm]
03:40:58:[ 2352.320900]  [<ffffffffa00d60f1>] ttm_bo_handle_move_mem+0x571/0x5b0 [ttm]
03:40:58:[ 2352.320900]  [<ffffffff81601ae4>] ? __slab_free+0x10e/0x277
03:40:58:[ 2352.320900]  [<ffffffffa00d674a>] ? ttm_bo_mem_space+0x10a/0x310 [ttm]
03:40:58:[ 2352.320900]  [<ffffffffa00d6e17>] ttm_bo_validate+0x247/0x260 [ttm]
03:40:58:[ 2352.320900]  [<ffffffff81059e69>] ? iounmap+0x79/0xa0
03:40:58:[ 2352.320900]  [<ffffffff81050069>] ? kgdb_arch_late+0xe9/0x180
03:40:58:[ 2352.320900]  [<ffffffffa013bac2>] cirrus_bo_push_sysram+0x82/0xe0 [cirrus]
03:40:58:[ 2352.320900]  [<ffffffffa0139c84>] cirrus_crtc_do_set_base.isra.8.constprop.10+0x84/0x430 [cirrus]
03:40:58:[ 2352.320900]  [<ffffffffa013a479>] cirrus_crtc_mode_set+0x449/0x4d0 [cirrus]
03:40:58:[ 2352.320900]  [<ffffffffa0107939>] drm_crtc_helper_set_mode+0x2e9/0x520 [drm_kms_helper]
03:40:58:[ 2352.320900]  [<ffffffffa01086bf>] drm_crtc_helper_set_config+0x87f/0xaa0 [drm_kms_helper]
03:40:58:[ 2352.320900]  [<ffffffff816092eb>] ? __ww_mutex_lock+0x1b/0xa0
03:40:58:[ 2352.320900]  [<ffffffffa0094711>] drm_mode_set_config_internal+0x61/0xe0 [drm]
03:40:58:[ 2352.320900]  [<ffffffffa0110a94>] drm_fb_helper_pan_display+0x94/0xf0 [drm_kms_helper]
03:40:58:[ 2352.320900]  [<ffffffff81326309>] fb_pan_display+0xc9/0x190
03:40:58:[ 2352.320900]  [<ffffffff81335390>] bit_update_start+0x20/0x50
03:40:58:[ 2352.320900]  [<ffffffff81334dbd>] fbcon_switch+0x39d/0x5a0
03:40:58:[ 2352.320900]  [<ffffffff813a35f9>] redraw_screen+0x1a9/0x270
03:40:58:[ 2352.320900]  [<ffffffff8132650e>] ? fb_blank+0xae/0xc0
03:40:58:[ 2352.320900]  [<ffffffff813322da>] fbcon_blank+0x22a/0x2f0
03:40:58:[ 2352.320900]  [<ffffffff81070384>] ? wake_up_klogd+0x34/0x50
04:40:23:********** Timeout by autotest system **********

Issue Links

• is related to: LU-5888 mount.lustre: set max_sectors_kb to 2147483647