[LU-9322] parallel-scale test_connectathon: connectathon failed: 1

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Affects Version/s: Lustre 2.10.0
• Environment: onyx-44, Full, ; RHEL7.3, DNE, ZFS, master branch, v2.10.55, b3550
• Severity: 3
• Assignee: WC Triage
• Reporter: James Casper (Inactive)

Description

https://testing.hpdd.intel.com/test_sessions/4b44547b-cba5-4172-ab2c-b902e4049a92

From test_log:

test telldir cookies
expected file 94 at cookie 0, found .
special tests failed
 parallel-scale test_connectathon: @@@@@@ FAIL: connectathon failed: 1 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:4893:error()
  = /usr/lib64/lustre/tests/functions.sh:542:run_connectathon()
  = /usr/lib64/lustre/tests/parallel-scale.sh:100:test_connectathon()
  = /usr/lib64/lustre/tests/test-framework.sh:5169:run_one()
  = /usr/lib64/lustre/tests/test-framework.sh:5208:run_one_logged()
  = /usr/lib64/lustre/tests/test-framework.sh:5055:run_test()
  = /usr/lib64/lustre/tests/parallel-scale.sh:102:main()

From OST console (onyx-44vm8):

LustreError: 784:0:(ofd_dev.c:1688:ofd_create_hdl()) lustre-OST0005: invalid precreate request for 0x0:302369, last_id 302881. Likely MDS last_id corruption
LustreError: 784:0:(ofd_dev.c:1688:ofd_create_hdl()) Skipped 120 previous similar messages

From MDS console (onyx-44vm7):

LustreError: 28135:0:(osp_precreate.c:917:osp_precreate_cleanup_orphans()) lustre-OST0005-osc-MDT0000: cannot cleanup orphans: rc = -22
LustreError: 28135:0:(osp_precreate.c:917:osp_precreate_cleanup_orphans()) Skipped 120 previous similar messages

May be related to LU-8806. It also has a "cannot cleanup orphans" message, but with a different return code.

Issue Links

• is related to: LU-11728 parallel-scale test connectathon fails with ''connectathon failed: 1''
• is related to: LU-9285 revert LU-8367 and LU-8972

Comments

[James Nunez (Inactive)]

Resolving issue because the patches were reverted and we haven't seen this issue for at least the past six months.

[Alex Zhuravlev]

both, actually. the fix depends on the reverts.

[John Hammond]

Alex, would you suggest one of your changes https://review.whamcloud.com/#/c/25925/ or https://review.whamcloud.com/#/c/25926 here?