[LU-10073] lnet-selftest test_smoke: lst Error found

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.16.0
• Affects Version/s: Lustre 2.11.0, Lustre 2.13.0, Lustre 2.10.7, Lustre 2.12.1, Lustre 2.12.3, Lustre 2.12.4, Lustre 2.12.5, Lustre 2.12.6
• Labels: LTS12 arm ppc rhel8 ubuntu
• Environment: trevis, full, x86_64 servers, ppc clients; servers: el7.4, ldiskfs, branch master, v2.10.53.1, b3642; clients: el7.4, branch master, v2.10.53.1, b3642
• Severity: 3
• Assignee: James A Simmons
• Reporter: James Casper (Inactive)

Description

https://testing.whamcloud.com/test_sets/87032fec-9d50-11e7-b778-5254006e85c2

Seen previously in 2.9 testing (LU-6622).

From test_log:

Batch is stopped
12345-10.9.0.84@tcp: [Session 0 brw errors, 30 ping errors] [RPC: 0 errors, 0 dropped, 30 expired]
12345-10.9.0.85@tcp: [Session 0 brw errors, 30 ping errors] [RPC: 0 errors, 0 dropped, 30 expired]
c:
Total 2 error nodes in c
12345-10.9.5.24@tcp: [Session 0 brw errors, 30 ping errors] [RPC: 0 errors, 0 dropped, 30 expired]
12345-10.9.5.25@tcp: [Session 0 brw errors, 30 ping errors] [RPC: 0 errors, 0 dropped, 30 expired]
s:
Total 2 error nodes in s
session is ended
Total 2 error nodes in c
Total 2 error nodes in s

and

Started clients trevis-77vm3.trevis.hpdd.intel.com,trevis-77vm4: 
CMD: trevis-77vm3.trevis.hpdd.intel.com,trevis-77vm4 mount | grep /mnt/lustre' '
10.9.5.25@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs)
10.9.5.25@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs)
CMD: trevis-77vm4 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/qt-3.3/bin:/usr/lib64/openmpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/sbin:/sbin:/bin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh set_default_debug \"vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck\" \"all\" 4 
trevis-77vm4: h2tcp: deprecated, use h2nettype instead
trevis-77vm4: trevis-77vm4.trevis.hpdd.intel.com: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 4
 lnet-selftest test_smoke: @@@@@@ FAIL: lst Error found 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:5289:error()
  = /usr/lib64/lustre/tests/lnet-selftest.sh:153:check_lst_err()
  = /usr/lib64/lustre/tests/lnet-selftest.sh:179:test_smoke()
  = /usr/lib64/lustre/tests/test-framework.sh:5565:run_one()
  = /usr/lib64/lustre/tests/test-framework.sh:5604:run_one_logged()
  = /usr/lib64/lustre/tests/test-framework.sh:5451:run_test()
  = /usr/lib64/lustre/tests/lnet-selftest.sh:182:main()

Issue Links

• is related to: LU-13063 sanity test 411 times out for RHEL8.1
• is related to: LU-12269 Support RHEL 8.0
• is related to: LU-15781 Ubuntu 22.04 LTS release support
• is related to: LU-10157 LNET_MAX_IOV hard coded to 256
• is related to: LU-11389 lnet-setltest test smoke fails with ‘lst Error found’
• is related to: LU-11878 sanity test 103b: OOM because of too many bash processes: page allocation stalls for 18420ms
• is related to: LU-10300 Can the Lustre 2.10.x clients support 64K kernel page?

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/38857/
Subject: LU-10073 tests: re-enable lnet selftest smoke test for PPC + ARM
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: c25a4e1fd92523247a0fa5a5c2809321765df263

[Gerrit Updater]

"James Simmons <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/46169
Subject: LU-10073 tests: re-enable lnet selftest smoke test for PPC + ARM
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 187eef8fd9cc5dce0014afe7db45c60caf7ee605

[James A Simmons]

We still have Power8 to test on to verify it works.

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/46037/
Subject: LU-10073 tests: re-enable lnet selftest smoke test 4.4+ kernels
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 24424791ac7233e393a32be189fe77102857653b

[James A Simmons]

Once PowerPC comes back I will test on that platform. I suspect this issue is resolved but we will have to wait and see.