[LU-5649] conf-sanity test_32a: FAIL: Mounting the MDT

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.5.4
• Affects Version/s: Lustre 2.5.4
• Labels: zfs
• Environment: Lustre Branch: b2_5; Lustre Build: https://build.hpdd.intel.com/job/lustre-b2_5/87/; FSTYPE=zfs
• Severity: 3
• Assignee: Jian Yu
• Reporter: Jian Yu

Description

conf-sanity test 32a failed as follows:

CMD: shadow-4vm4 mount -t lustre -o exclude=t32fs-OST0000 t32fs-mdt1/mdt1 /tmp/t32/mnt/mdt
shadow-4vm4: mount.lustre: mount t32fs-mdt1/mdt1 at /tmp/t32/mnt/mdt failed: No such file or directory
shadow-4vm4: Is the MGS specification correct?
shadow-4vm4: Is the filesystem name correct?
shadow-4vm4: If upgrading, is the copied client log valid? (see upgrade docs)
CMD: shadow-4vm4 losetup -a
 conf-sanity test_32a: @@@@@@ FAIL: Mounting the MDT

Dmesg on MDS showed that:

Lustre: DEBUG MARKER: mount -t lustre -o exclude=t32fs-OST0000 t32fs-mdt1/mdt1 /tmp/t32/mnt/mdt
LustreError: 20713:0:(obd_config.c:572:class_setup()) setup t32fs-MDT0000 failed (-2)
LustreError: 20713:0:(obd_config.c:1591:class_config_llog_handler()) MGC10.1.4.32@tcp: cfg command failed: rc = -2
Lustre:    cmd=cf003 0:t32fs-MDT0000  1:t32fs-MDT0000_UUID  2:0  3:t32fs-MDT0000-mdtlov  4:f  
LustreError: 15c-8: MGC10.1.4.32@tcp: The configuration from log 't32fs-MDT0000' failed (-2). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
LustreError: 20657:0:(obd_mount_server.c:1252:server_start_targets()) failed to start server t32fs-MDT0000: -2
LustreError: 20657:0:(obd_mount_server.c:1723:server_fill_super()) Unable to start targets: -2
LustreError: 20657:0:(obd_mount_server.c:845:lustre_disconnect_lwp()) t32fs-MDT0000-lwp-MDT0000: Can't end config log t32fs-client.
LustreError: 20657:0:(obd_mount_server.c:1420:server_put_super()) t32fs-MDT0000: failed to disconnect lwp. (rc=-2)
LustreError: 20657:0:(obd_config.c:619:class_cleanup()) Device 5 not setup
Lustre: 20657:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1411052033/real 1411052033]  req@ffff880070e6ac00 x1479580630318032/t0(0) o251->MGC10.1.4.32@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1411052039 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
Lustre: 20657:0:(client.c:1918:ptlrpc_expire_one_request()) Skipped 6 previous similar messages
LustreError: 20657:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount  (-2)
Lustre: DEBUG MARKER: losetup -a
Lustre: DEBUG MARKER: /usr/sbin/lctl mark  conf-sanity test_32a: @@@@@@ FAIL: Mounting the MDT

Maloo report: https://testing.hpdd.intel.com/test_sets/837308d4-40e2-11e4-b0c7-5254006e85c2

This is a regression failure introduced by Lustre b2_5 build #87: http://review.whamcloud.com/11951

LU-3489 osd-zfs: varsize index support
LU-4334 utils: Only set a single property for nodes
LU-3660 acl: support mount option "noacl" for zfs (detail / gitweb)
LU-5150 acl: 'cp -rp' set empty default ACL (detail / gitweb)
LU-5434 mdd: disregard empty POSIX ACLs (detail / gitweb)

Comments

[Sarah Liu]

hit on master branch build #3071 ldiskfs

https://testing.hpdd.intel.com/test_sets/5bfb8c8c-12eb-11e5-b47b-5254006e85c2

[Jian Yu]

"LU-3489 osd-zfs: varsize index support" (commit d9e627c09ccc5c22aa10704020fb9f45bcf26ed0) was reverted from Lustre b2_5 branch.

[Jian Yu]

It seems that the failure was not present with the patch 12043

Hi Oleg, the failure also occurred with the patch 12043: https://testing.hpdd.intel.com/test_sets/731cdab0-44a1-11e4-8e4d-5254006e85c2

[Oleg Drokin]

It seems that the failure was not present with the patch 12043, Yu Jian, can you please verify it's really gone somehow? Then I'll just land this patch and we are done here.

[Oleg Drokin]

So, LU-3489 was a two patch series, but only one patch landed because the other one was conflicting.
I wonder if the other patch fixes issues in the first one? (commit cf753f6b4bdc267b5169e97293f69b5362b826de )
I ported it to b2_5 here: http://review.whamcloud.com/12043

[Jian Yu]

Hi Oleg,

The test results showed that "LU-3489 osd-zfs: varsize index support" (commit d9e627c09ccc5c22aa10704020fb9f45bcf26ed0) was the cause of the regression failure. Should we revert this change from Lustre b2_5 branch so as to make other patches pass review testing on Lustre b2_5 branch?

[Jian Yu]

I suspected "LU-3489 osd-zfs: varsize index support" (commit d9e627c) was the culprit and pushed http://review.whamcloud.com/12012 to verify.

[Jian Yu]

This is blocking patch review testing on Lustre b2_5 branch.