Details
-
Bug
-
Resolution: Cannot Reproduce
-
Minor
-
None
-
Lustre 2.8.0
-
None
-
Client: lustre-master# 3166, SLES11 SP3
Server: lustre-master# 3166, RHEL 7
-
3
-
9223372036854775807
Description
This issue was created by maloo for Saurabh Tandan <saurabh.tandan@intel.com>
This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/46803cda-52de-11e5-b61c-5254006e85c2.
The sub-test test_0b failed with the following error:
import is not in FULL state
Test log:
Started lustre:OST0006 CMD: shadow-16vm5,shadow-16vm6,shadow-16vm9 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh wait_import_state_mount FULL osc.lustre:OST0000-osc-*.ost_server_uuid,osc.lustre:OST0001-osc-*.ost_server_uuid,osc.lustre:OST0002-osc-*.ost_server_uuid,osc.lustre:OST0003-osc-*.ost_server_uuid,osc.lustre:OST0004-osc-*.ost_server_uuid,osc.lustre:OST0005-osc-*.ost_server_uuid,osc.lustre:OST0006-osc-*.ost_server_uuid shadow-16vm9: CMD: shadow-16vm9 lctl get_param -n at_max shadow-16vm5: CMD: shadow-16vm5 lctl get_param -n at_max shadow-16vm6: CMD: shadow-16vm6 lctl get_param -n at_max shadow-16vm9: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm6: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm5: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm9: Trace dump: shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm9: = rpc.sh:20:main() shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm9: CMD: shadow-16vm3 uname -n shadow-16vm9: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282291.log shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282291.log; shadow-16vm9: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282291.log shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282291.log shadow-16vm9:/tmp/test_logs/1441280803 shadow-16vm5: Trace dump: shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm5: = rpc.sh:20:main() shadow-16vm6: Trace dump: shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm6: = rpc.sh:20:main() shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm5: CMD: shadow-16vm5 uname -n shadow-16vm6: CMD: shadow-16vm5 uname -n shadow-16vm5: CMD: shadow-16vm6 uname -n shadow-16vm5: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282293.log; shadow-16vm5: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282293.log shadow-16vm6: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282293.log; shadow-16vm6: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282293.log shadow-16vm5: shadow-16vm7: open /proc/sys/lnet/dump_kernel failed: No such file or directory shadow-16vm5: shadow-16vm7: open(dump_kernel) failed: No such file or directory shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm5:/tmp/test_logs/1441280803 shadow-16vm6: shadow-16vm7: open /proc/sys/lnet/dump_kernel failed: No such file or directory shadow-16vm6: shadow-16vm7: open(dump_kernel) failed: No such file or directory shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm6:/tmp/test_logs/1441280803 shadow-16vm5: shadow-16vm6: Warning: Permanently added 'shadow-16vm5,10.1.4.189' (ECDSA) to the list of known hosts. shadow-16vm5: shadow-16vm5: Warning: Permanently added 'shadow-16vm5,10.1.4.189' (ECDSA) to the list of known hosts. shadow-16vm6: shadow-16vm6: Warning: Permanently added 'shadow-16vm6,10.1.4.190' (ECDSA) to the list of known hosts. shadow-16vm6: shadow-16vm5: Warning: Permanently added 'shadow-16vm6,10.1.4.190' (ECDSA) to the list of known hosts. replay-ost-single test_0b: @@@@@@ FAIL: import is not in FULL state
LustreError inside dmesg for replay-ost-single test 0b in mds :
[ 3574.906399] LustreError: 16331:0:(events.c:306:request_in_callback()) event type 2, status -5, service mdt