[LU-5986] conf-sanity test_83, test_84: failed to start OST

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.7.0
• Affects Version/s: Lustre 2.7.0
• Labels: zfs
• Environment: FSTYPE=zfs
• Severity: 3
• Assignee: Nathaniel Clark
• Reporter: Jian Yu

Description

Patch http://review.whamcloud.com/9078 for LU-4119 introduced a regression failure in conf-sanity test 83 under ZFS configuration:

start ost1 service on onyx-36vm8
CMD: onyx-36vm8 mkdir -p /mnt/ost1
CMD: onyx-36vm8 zpool list -H lustre-ost1 >/dev/null 2>&1 ||
			zpool import -f -o cachefile=none -d /dev/lvm-Role_OSS lustre-ost1
Starting ost1:   lustre-ost1/ost1 /mnt/ost1
CMD: onyx-36vm8 mkdir -p /mnt/ost1; mount -t lustre   		                   lustre-ost1/ost1 /mnt/ost1
onyx-36vm8: mount.lustre: mount lustre-ost1/ost1 at /mnt/ost1 failed: No such file or directory
onyx-36vm8: Is the MGS specification correct?
onyx-36vm8: Is the filesystem name correct?
onyx-36vm8: If upgrading, is the copied client log valid? (see upgrade docs)
Start of lustre-ost1/ost1 on ost1 failed 2

On OSS node:

07:55:10:Lustre: DEBUG MARKER: mkdir -p /mnt/ost1; mount -t lustre   		                   lustre-ost1/ost1 /mnt/ost1
07:55:10:LustreError: 2367:0:(obd_mount_server.c:1168:server_register_target()) lustre-OST0000: error registering with the MGS: rc = -2 (not fatal)
07:55:10:LustreError: 13a-8: Failed to get MGS log lustre-OST0000 and no local copy.

On MDS node:

08:56:17:Lustre: DEBUG MARKER: zfs get -H -o value lustre:svname 		                           lustre-mdt1/mdt1 2>/dev/null
08:56:17:Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1
08:56:17:LustreError: 13b-9: lustre-OST0000 claims to have registered, but this MGS does not know about it, preventing registration.
08:56:17:LustreError: 13b-9: lustre-OST0001 claims to have registered, but this MGS does not know about it, preventing registration.

Maloo reports:
https://testing.hpdd.intel.com/test_sets/77dccc26-6ccf-11e4-960c-5254006e85c2
https://testing.hpdd.intel.com/test_sets/f6eaf896-717d-11e4-b5de-5254006e85c2
https://testing.hpdd.intel.com/test_sets/04731068-7c08-11e4-bdab-5254006e85c2
https://testing.hpdd.intel.com/test_sets/bb09e096-7c0c-11e4-bdab-5254006e85c2

Info required for matching: conf-sanity 83

Issue Links

• is related to: LU-6029 conf-sanity test_84: recovery_duration > recovery_time_hard
• is related to: LU-4119 recovery time hard doesn't limit recovery duration
• is related to: LU-5729 VFS: Busy inodes after unmount of loop0. Self-destruct in 5 seconds. Have a nice day...

Comments

[Andreas Dilger]

Patch landed to master for 2.7.0.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/13016/
Subject: LU-5986 test: Ensure correct start for conf-sanity/84
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: f2e76a69233a2fd348ae1f8af3c937048695a4a1

[Gerrit Updater]

Nathaniel Clark ([e-mail]) uploaded a new patch: http://review.whamcloud.com/13016
Subject: LU-5986 test: Ensure correct start for conf-sanity/84
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a63e8110da2689482fbd2a112d34a7f69f555678

[Andreas Dilger]

Nathaniel, it looks like this may be the last major blocker for enforcing review-zfs. Any chance to get a patch for this quickly?

[Nathaniel Clark]

It still seems to fail on ZFS, though test 84 passed in local testing, but I didn't run full conf-sanity, I just ran test_84. I think it's expecting a setup it's not getting for ZFS because of skipped test 83.

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch http://review.whamcloud.com/12984/
Subject: LU-5986 test: fix conflicting conf-sanity 83 test.
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: feaeafe8186fafa4cb0c8180bd9746c4a7af809f

[Andreas Dilger]

Nathaniel, does the simple patch in 12984 to create separate test numbers also fix the ZFS problem, or does that still need a second patch?

[Nathaniel Clark]

Testing an identical patch locally, and it appears that it does.

[James A Simmons]

Lets see if a simple fix corrects this.

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: http://review.whamcloud.com/12984
Subject: LU-5986 test: fix conflicting conf-sanity 83 test.
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 829e13cad6f84706d3cd5e203fd7bdf6fd18ff19