Details
Description
This issue was created by maloo for jianyu <yujian@whamcloud.com>
This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/819093e4-599f-4cdd-a07b-beb8ba8b3c62
test_134 failed with the following error:
rm: cannot remove '/mnt/lustre/d134.recovery-small/1/f134.recovery-small': Input/output error pdsh@onyx-81vm4: onyx-81vm4: ssh exited with exit code 5 onyx-81vm6: mv: failed to access '/mnt/lustre/d134.recovery-small/2/f134.recovery-small_2': Cannot send after transport endpoint shutdown pdsh@onyx-81vm4: onyx-81vm6: ssh exited with exit code 1 pdsh@onyx-81vm4: onyx-81vm6: ssh exited with exit code 5 CMD: onyx-81vm4.onyx.whamcloud.com,onyx-81vm6,onyx-81vm7 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/opt/iozone/bin:/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config TESTLOG_PREFIX=/autotest/autotest-2/2024-09-30/lustre-master_failover-part-1_4581_150_55f5e009-b5b6-4b5e-89f5-a0d648cecea4//recovery-small TESTNAME=test_134 CONFIG=/usr/lib64/lustre/tests/cfg/autotest_config.sh bash rpc.sh wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-*.mds_server_uuid onyx-81vm7: onyx-81vm7.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid onyx-81vm4: onyx-81vm4.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid onyx-81vm6: onyx-81vm6.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid onyx-81vm4: CMD: onyx-81vm4.onyx.whamcloud.com lctl get_param -n at_max onyx-81vm7: CMD: onyx-81vm7.onyx.whamcloud.com lctl get_param -n at_max onyx-81vm6: CMD: onyx-81vm6.onyx.whamcloud.com lctl get_param -n at_max onyx-81vm4: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec onyx-81vm7: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec onyx-81vm6: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec recovery-small test_134: @@@@@@ FAIL: rm failed
Test session details:
clients: https://build.whamcloud.com/job/lustre-master/4581 - 4.18.0-513.24.1.el8_9.x86_64
servers: https://build.whamcloud.com/job/lustre-master/4581 - 4.18.0-513.24.1.el8_lustre.x86_64
<<Please provide additional information about the failure here>>
VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
recovery-small test_134 - rm failed