Details
-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
This issue was created by maloo for Andreas Dilger <adilger@whamcloud.com>
This issue relates to the following test suite run:
https://testing.whamcloud.com/test_sets/72dfd1b1-7efa-4dcc-a366-043549333756
https://testing.whamcloud.com/test_sets/1dde1cd4-24fd-4ae3-ab20-d7261788833f
test_1 failed with the following error:
CMD: onyx-106vm11 /usr/sbin/lctl set_param -P lod.*.mdt_hash=crush comparing 520 previously copied files Files /etc/yum.repos.d/redhat.repo and /mnt/lustre/d1.runtests//etc/yum.repos.d/redhat.repo differ Files /etc/pki/entitlement/2519028287967039457.pem and /mnt/lustre/d1.runtests//etc/pki/entitlement/2519028287967039457.pem differ runtests test_1: @@@@@@ FAIL: old and new files are different: rc=22
Test session details:
clients: https://build.whamcloud.com/job/lustre-master-next/784 - 5.14.0-362.18.1.el9_3.x86_64
servers: https://build.whamcloud.com/job/lustre-master-next/784 - 5.14.0-362.18.1_lustre.el9.x86_64
This failed for the first time with this error on 2024-04-24 for on two separate test runs, one an unlanded patch, and one a "full" test run on master. Strangely, both failures were reported on the same two files. There don't appear to be any Lustre console errors immediately before this failure (a few back when the filesystem is remounted in the test).
VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
runtests test_1 - old and new files are different: rc=22
Attachments
Issue Links
Activity
ys it should be enough to exclude the /etc/yum_repos.d and /etc/pki subdirectories during the test. While copying the whole /etc tree to another path would work, this slows down the test and is probably not needed.
Can we copy the /etc to somewhere in local FS, then copy it to lustre to compare?
Something is updating /etc/yum.repos.d/redhat.repo and /etc/pki/entitlement/xxxx.pem during the test run. Not very often, but enough to cause some failures. If we don't know what that is, I guess we need a patch to make the runtests script more robust.
mdiep or mkvardakov would be better suited to answer this question
+1 on b2_15:
https://testing.whamcloud.com/test_sets/6ffef452-6051-4199-90f0-cec559c8aaf6
colmstea, is there some RHEL license manager that is running which is randomly updating these files in /etc while the test is running? We can make the test more robust by having it re-check and exclude files in /etc that have been modified since the test was started, but that is not retroactive for all of the branches and interop tests, so it would be nice if whatever is triggering this was turned off, or done during provisioning instead while the test was running...
"Arshad Hussain <arshad.hussain@aeoncomputing.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58129
Subject: LU-17777 tests: Exclude /etc/yum*/* and /etc/pki/*
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d230e830e6eb168d59ce0fb4fe3fb626ee4f843b