[LU-16658] performance sanity test_6 mdsrate-lookup-10dirs - UCX ERROR

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.16.0
• Labels: failing_tests
• Severity: 3
• Assignee: Andreas Dilger
• Reporter: Andreas Dilger

Description

This issue was created by maloo for Cliff White <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/d2a273de-f0d8-426b-adb8-ccd3fed94347

Client appears to hand/drop connection while doing initial file/dir creation

===== mdsrate-lookup-10dirs.sh Test preparation: creating 10 dirs with 12650 files.
+ /usr/lib64/openmpi/bin/mdsrate --mknod --ndirs 10 --dirfmt '/mnt/lustre/mdsrate/lookup-%d' --nfiles 12650 --filefmt 'f%%d'
+ chmod 0777 /mnt/lustre
drwxrwxrwx 4 root root 4096 Jul 19 19:11 /mnt/lustre
+ su mpiuser sh -c "/usr/lib64/openmpi/bin/mpirun --mca btl tcp,self --mca btl_tcp_if_include eth0 -mca boot ssh --oversubscribe --oversubscribe -machinefile /tmp/auster.machines -np 10 /usr/lib64/openmpi/bin/mdsrate --mknod --ndirs 10 --dirfmt '/mnt/lustre/mdsrate/lookup-%d' --nfiles 12650 --filefmt 'f%%d' "
0: onyx-91vm11.onyx.whamcloud.com starting at Tue Jul 19 19:11:39 2022
[1658257899.971284] [onyx-91vm12:60266:0]          select.c:514  UCX  ERROR   no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory0 - Destination is unreachable, tcp/eth0 - Destination is unreachable, tcp/lo - Destination is unreachable

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
performance-sanity test_6 - Timeout occurred after 610 minutes, last suite running was performance-sanity

Issue Links

• is related to: LU-3786 performance-sanity test_6: mkdir hung (sys_mkdir)
• is related to: LU-16729 replace mdsrate with mdtest in Lustre test suite
• is related to: LU-16270 performance-sanity test_6: mkdir: cannot create directory '/mnt/lustre/mdsrate': File exists

Comments

[Andreas Dilger]

This has been fixed by the change to use mdtest.

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50425/
Subject: LU-16658 tests: disable performance-sanity test_6
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 254ee03c017c3457f3f064152ade8695100780ee

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50386/
Subject: LU-16658 tests: disable performance-sanity test_6
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5e4897eb6f1c97d4f0120803780904db49c5abe7

[Gerrit Updater]

"Jian Yu <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50425
Subject: LU-16658 tests: disable performance-sanity test_6
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 5a12022bf71e8c9d7e88eea4843ae73dc004840d

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50386
Subject: LU-16658 tests: disable performance-sanity test_6
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c4b23f1975e750e8c8cbe64741b8427d708cb7a6