[LU-6821] conf-sanity test_62: tune2fs failed

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.8.0
• Labels: interop
• Environment: server: lustre-master build # 3092 RHEL7; client: 2.7.0
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for sarah_lw <[e-mail]>

This issue relates to the following test suite run: https://testing-archive.whamcloud.com/test_sets/45673fc0-250b-11e5-bf7b-5254006e85c2.

The sub-test test_62 failed with the following error:

tune2fs failed

== conf-sanity test 62: start with disabled journal == 21:38:30 (1436218710)
CMD: shadow-18vm4 /usr/sbin/lctl get_param -n version
disable journal for mds
CMD: shadow-18vm4 tune2fs -O ^has_journal /dev/lvm-Role_MDS/P1
shadow-18vm4: device /dev/dm-0 mounted by lustre
shadow-18vm4: The has_journal feature may only be cleared when the filesystem is
shadow-18vm4: unmounted or mounted read-only.
tune2fs 1.42.12.wc1 (15-Sep-2014)
 conf-sanity test_62: @@@@@@ FAIL: tune2fs failed 

MDS dmesg

/BUILD/BUILD/lustre-2.7.55/ldiskfs/ldiskfs.h:2631 ldiskfs_xattr_set_entry+0x7c4/0x7f0 [ldiskfs]()
[27480.366065] Modules linked in: osp(OF) mdd(OF) lod(OF) mdt(OF) lfsck(OF) mgs(OF) mgc(OF) osd_ldiskfs(OF) lquota(OF) fid(OF) fld(OF) ksocklnd(OF) ptlrpc(OF) obdclass(OF) lnet(OF) libcfs(OF) ldiskfs(OF) loop sha512_generic dm_mod nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd fscache xprtrdma sunrpc ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ppdev serio_raw pcspkr virtio_balloon i2c_piix4 ib_mad parport_pc parport ib_core ib_addr ext4 mbcache jbd2 ata_generic pata_acpi virtio_blk cirrus ata_piix syscopyarea sysfillrect sysimgblt drm_kms_helper 8139too ttm virtio_pci drm virtio_ring virtio 8139cp mii i2c_core libata floppy [last unloaded: libcfs]

[27480.373248] CPU: 0 PID: 25023 Comm: mdt00_002 Tainted: GF       W  O--------------   3.10.0-229.4.2.el7_lustre.x86_64 #1
[27480.374159] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
[27480.374640]  0000000000000000 000000004b6949fd ffff8800792c3708 ffffffff816050da
[27480.375315]  ffff8800792c3740 ffffffff8106e34b ffff88000083eb48 ffff88007b57bfc0
[27480.375976]  ffff8800792c3890 0000000000010000 ffff88006219ad68 ffff8800792c3750
[27480.376656] Call Trace:
[27480.376879]  [<ffffffff816050da>] dump_stack+0x19/0x1b
[27480.377343]  [<ffffffff8106e34b>] warn_slowpath_common+0x6b/0xb0
[27480.377846]  [<ffffffff8106e49a>] warn_slowpath_null+0x1a/0x20
[27480.378363]  [<ffffffffa05dd3b4>] ldiskfs_xattr_set_entry+0x7c4/0x7f0 [ldiskfs]
[27480.379011]  [<ffffffffa05dde55>] ldiskfs_xattr_ibody_set+0x35/0x90 [ldiskfs]
[27480.379626]  [<ffffffffa05de244>] ldiskfs_xattr_set_handle+0x1a4/0x500 [ldiskfs]
[27480.380282]  [<ffffffffa05de6e6>] ldiskfs_xattr_set+0x146/0x1c0 [ldiskfs]
[27480.381053]  [<ffffffffa060a01d>] ldiskfs_xattr_trusted_set+0x2d/0x30 [ldiskfs]
[27480.381762]  [<ffffffff811eb4b2>] generic_setxattr+0x62/0x80
[27480.382283]  [<ffffffffa0bd7b94>] osd_xattr_set+0x3e4/0x700 [osd_ldiskfs]
[27480.382925]  [<ffffffffa0e8437d>] lod_sub_object_xattr_set+0x1ed/0x480 [lod]
[27480.383548]  [<ffffffffa0e75a79>] lod_xattr_set_internal+0xb9/0x2d0 [lod]
[27480.384216]  [<ffffffffa0e7ba96>] lod_xattr_set+0x136/0x3f0 [lod]
[27480.384793]  [<ffffffffa0ee23e3>] mdd_xattr_set+0x483/0x720 [mdd]
[27480.385354]  [<ffffffffa0d976c7>] ? mdt_version_save+0x67/0x120 [mdt]
[27480.385963]  [<ffffffffa0da4057>] mdt_reint_setxattr+0x967/0xf90 [mdt]
[27480.386545]  [<ffffffffa0da16e0>] mdt_reint_rec+0x80/0x210 [mdt]
[27480.387160]  [<ffffffffa0d854e9>] mdt_reint_internal+0x5d9/0xb20 [mdt]
[27480.387768]  [<ffffffffa0d8e6d7>] mdt_reint+0x67/0x140 [mdt]
[27480.388339]  [<ffffffffa0a2029b>] tgt_request_handle+0x88b/0x1100 [ptlrpc]
[27480.389023]  [<ffffffffa09c7fbb>] ptlrpc_server_handle_request+0x21b/0xa90 [ptlrpc]
[27480.389713]  [<ffffffffa09c5078>] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc]
[27480.390391]  [<ffffffffa05648e7>] ? libcfs_debug_msg+0x57/0x80 [libcfs]
[27480.391036]  [<ffffffffa09cb900>] ptlrpc_main+0xc00/0x1f60 [ptlrpc]
[27480.391587]  [<ffffffff810ad8b6>] ? __dequeue_entity+0x26/0x40
[27480.392195]  [<ffffffffa09cad00>] ? ptlrpc_register_service+0x1080/0x1080 [ptlrpc]
[27480.392887]  [<ffffffff8109739f>] kthread+0xcf/0xe0
[27480.393329]  [<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140
[27480.393944]  [<ffffffff81614f7c>] ret_from_fork+0x7c/0xb0
[27480.394428]  [<ffffffff810972d0>] ? kthread_create_on_node+0x140/0x140
[27480.395065] ---[ end trace cba16f9e09ca39e5 ]---
[27480.400410] LustreError: 25023:0:(mdt_handler.c:895:mdt_getattr_internal()) lustre-MDT0000: getattr error for [0x200000401:0x1:0x0]: rc = -5
[27480.589180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  conf-sanity test_61: @@@@@@ FAIL: trusted.big different after saving 
[27480.758141] Lustre: DEBUG MARKER: conf-sanity test_61: @@@@@@ FAIL: trusted.big different after saving
[27481.024667] Lustre: DEBUG MARKER: /usr/sbin/lctl dk > /logdir/test_logs/2015-07-06/lustre-master-el7-x86_64-vs-lustre-b2_7-el6_6-x86_64--full--2_5_1__3092__-70321105568440-131041/conf-sanity.test_61.debug_log.$(hostname -s).1436218708.log;
         dmesg > /logdir/test_logs/2015-07-06/lus
[27483.314375] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == conf-sanity test 62: start with disabled journal == 21:38:30 \(1436218710\)
[27483.467369] Lustre: DEBUG MARKER: == conf-sanity test 62: start with disabled journal == 21:38:30 (1436218710)
[27483.614848] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version
[27483.834160] Lustre: DEBUG MARKER: tune2fs -O ^has_journal /dev/lvm-Role_MDS/P1
[27484.112686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  conf-sanity test_62: @@@@@@ FAIL: tune2fs failed 
[27484.266789] Lustre: DEBUG MARKER: conf-sanity test_62: @@@@@@ FAIL: tune2fs failed

Comments

[Andreas Dilger]

All of the cases of this test failing are because the MDS was not unmounted after test_61() failed, so it isn't something to be fixed on its own, except possibly ensuring that the MDS is unmounted before trying to disable the journal to avoid propagating the previous failure into more tests.

[Sarah Liu]

another instance:
https://testing-archive.whamcloud.com/test_sets/482e1f82-282e-11e5-b595-5254006e85c2

server: lustre-master build #3093 RHEL7.1
client: RHEL6.6