[LU-10907] sanity-sec test_23a: 'mgs and default trusted_nodemap mismatch, 10 attempts' Created: 11/Apr/18  Updated: 17/May/18  Resolved: 17/May/18

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.10.3
Fix Version/s: Lustre 2.12.0

Type: Bug Priority: Minor
Reporter: Maloo Assignee: Emoly Liu
Resolution: Fixed Votes: 0
Labels: None
Environment:

SLES12 SP3 Server/Client ldiskfs DNE
b2_10
build 96

Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

This issue was created by maloo for Saurabh Tandan <saurabh.tandan@intel.com>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/86a4bb42-3cf6-11e8-8f8a-52540065bddc

test_23a failed with the following error:

mgs and default trusted_nodemap mismatch, 10 attempts

client1 test_logs:

== sanity-sec test 23a: test mapped regular ACLs ===================================================== 12:00:26 (1523386826)
CMD: onyx-48vm8 /usr/sbin/lctl get_param -n version 2>/dev/null ||
				/usr/sbin/lctl lustre_build_version 2>/dev/null ||
				/usr/sbin/lctl --version 2>/dev/null | cut -d' ' -f2
CMD: onyx-48vm8,onyx-48vm9 /usr/sbin/lctl set_param mdt.*.identity_upcall=NONE
mdt.lustre-MDT0001.identity_upcall=NONE
mdt.lustre-MDT0003.identity_upcall=NONE
mdt.lustre-MDT0000.identity_upcall=NONE
mdt.lustre-MDT0002.identity_upcall=NONE
CMD: onyx-48vm5 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add c0
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_range --name c0 --range 10.2.8.227@tcp
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype uid --idmap 60003:60000
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype gid --idmap 60003:60000
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype uid --idmap 60004:60002
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c0 --idtype gid --idmap 60004:60002
CMD: onyx-48vm8 /usr/sbin/lctl get_param -n nodemap.active
CMD: onyx-48vm6 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add c1
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_range --name c1 --range 10.2.8.228@tcp
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype uid --idmap 60005:60001
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype gid --idmap 60005:60001
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype uid --idmap 60006:60002
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_add_idmap --name c1 --idtype gid --idmap 60006:60002
CMD: onyx-48vm8 /usr/sbin/lctl get_param -n nodemap.active
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_activate 1
CMD: onyx-48vm8 /usr/sbin/lctl get_param -n nodemap.active
CMD: onyx-48vm8 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
CMD: onyx-48vm8 /usr/sbin/lctl get_param nodemap.active
On MGS 10.2.8.230, active = nodemap.active=1
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, active = nodemap.active=1
CMD: onyx-48vm8 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
CMD: onyx-48vm9 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm9 10.2.8.231, active = nodemap.active=1
waited 7 seconds for sync
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_modify --name default --property admin --value 1
CMD: onyx-48vm8 /usr/sbin/lctl nodemap_modify --name default --property trusted --value 1
CMD: onyx-48vm8 /usr/sbin/lctl get_param -n nodemap.active
CMD: onyx-48vm8 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
CMD: onyx-48vm8 /usr/sbin/lctl get_param nodemap.default.trusted_nodemap
On MGS 10.2.8.230, default.trusted_nodemap = nodemap.default.trusted_nodemap=1
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
CMD: onyx-48vm7 /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d @
On onyx-48vm7 10.2.8.229, default.trusted_nodemap = nodemap.default.trusted_nodemap=0
MGS
nodemap.default.trusted_nodemap=1
OTHER - IP: 10.2.8.229
nodemap.default.trusted_nodemap=0
 sanity-sec test_23a: @@@@@@ FAIL: mgs and default trusted_nodemap mismatch, 10 attempts

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-sec test_23a - mgs and default trusted_nodemap mismatch, 10 attempts

 Comments   
Comment by Peter Jones [ 12/Apr/18 ]

Emoly

Could you please advise on this one?

Thanks

Peter

Comment by Gerrit Updater [ 17/Apr/18 ]

Emoly Liu (emoly.liu@intel.com) uploaded a new patch: https://review.whamcloud.com/32018
Subject: LU-10907 tests: improve nodemap_test_setup()
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4a5c023a70b50d6af5ea36556d157c6c9da242ae

Comment by Emoly Liu [ 17/Apr/18 ]

standan,

I can't reproduce this issue, but after checking the code, I think my above patch should be helpful. If you can reproduce this issue, could you please apply the patch to b2_10 and see if it works?

Thanks,
Emoly

Comment by Saurabh Tandan (Inactive) [ 18/Apr/18 ]

Sure, I will do that. Thanks.

Comment by Saurabh Tandan (Inactive) [ 01/May/18 ]

I tried to reproduce this issue but I was not able to reproduce it. It also never happened after that. It has been seen couple of times on master branch before but this issue is very rarely posing up in autotest environment.

Comment by Gerrit Updater [ 17/May/18 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/32018/
Subject: LU-10907 tests: improve nodemap_test_setup()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: bb22454d2203553f657f26e13d6bbeea66270a48

Comment by Peter Jones [ 17/May/18 ]

Landed for 2.12

Generated at Sat Feb 10 02:39:14 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.