[LU-7165] Failover: replay-ost-single test_0b: import is not in FULL state Created: 15/Sep/15 Updated: 14/Dec/21 Resolved: 14/Dec/21 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.8.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor |
| Reporter: | Maloo | Assignee: | WC Triage |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Client: lustre-master# 3166, SLES11 SP3 |
||
| Severity: | 3 |
| Rank (Obsolete): | 9223372036854775807 |
| Description |
|
This issue was created by maloo for Saurabh Tandan <saurabh.tandan@intel.com> This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/46803cda-52de-11e5-b61c-5254006e85c2. The sub-test test_0b failed with the following error: import is not in FULL state Test log: Started lustre:OST0006 CMD: shadow-16vm5,shadow-16vm6,shadow-16vm9 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh wait_import_state_mount FULL osc.lustre:OST0000-osc-*.ost_server_uuid,osc.lustre:OST0001-osc-*.ost_server_uuid,osc.lustre:OST0002-osc-*.ost_server_uuid,osc.lustre:OST0003-osc-*.ost_server_uuid,osc.lustre:OST0004-osc-*.ost_server_uuid,osc.lustre:OST0005-osc-*.ost_server_uuid,osc.lustre:OST0006-osc-*.ost_server_uuid shadow-16vm9: CMD: shadow-16vm9 lctl get_param -n at_max shadow-16vm5: CMD: shadow-16vm5 lctl get_param -n at_max shadow-16vm6: CMD: shadow-16vm6 lctl get_param -n at_max shadow-16vm9: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm6: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm5: rpc : @@@@@@ FAIL: can't put import for osc.lustre:OST0000-osc-*.ost_server_uuid into FULL state after 1475 sec, have shadow-16vm9: Trace dump: shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm9: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm9: = rpc.sh:20:main() shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm9: CMD: shadow-16vm3 uname -n shadow-16vm9: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282291.log shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282291.log; shadow-16vm9: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282291.log shadow-16vm9: CMD: shadow-16vm3,shadow-16vm5,shadow-16vm6,shadow-16vm8,shadow-16vm9 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282291.log shadow-16vm9:/tmp/test_logs/1441280803 shadow-16vm5: Trace dump: shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm5: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm5: = rpc.sh:20:main() shadow-16vm6: Trace dump: shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:4748:error_noexit() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:4779:error() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5837:_wait_import_state() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5856:wait_import_state() shadow-16vm6: = /usr/lib64/lustre/tests/test-framework.sh:5865:wait_import_state_mount() shadow-16vm6: = rpc.sh:20:main() shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:./../utils:/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh check_logdir /tmp/test_logs/1441280803 shadow-16vm5: CMD: shadow-16vm5 uname -n shadow-16vm6: CMD: shadow-16vm5 uname -n shadow-16vm5: CMD: shadow-16vm6 uname -n shadow-16vm5: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282293.log; shadow-16vm5: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282293.log shadow-16vm6: Dumping lctl log to /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 /usr/sbin/lctl dk > /tmp/test_logs/1441280803/rpc..debug_log.\$(hostname -s).1441282293.log; shadow-16vm6: dmesg > /tmp/test_logs/1441280803/rpc..dmesg.\$(hostname -s).1441282293.log shadow-16vm5: shadow-16vm7: open /proc/sys/lnet/dump_kernel failed: No such file or directory shadow-16vm5: shadow-16vm7: open(dump_kernel) failed: No such file or directory shadow-16vm5: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm5:/tmp/test_logs/1441280803 shadow-16vm6: shadow-16vm7: open /proc/sys/lnet/dump_kernel failed: No such file or directory shadow-16vm6: shadow-16vm7: open(dump_kernel) failed: No such file or directory shadow-16vm6: CMD: shadow-16vm5,shadow-16vm6,shadow-16vm7,shadow-16vm8 rsync -az /tmp/test_logs/1441280803/rpc..*.1441282293.log shadow-16vm6:/tmp/test_logs/1441280803 shadow-16vm5: shadow-16vm6: Warning: Permanently added 'shadow-16vm5,10.1.4.189' (ECDSA) to the list of known hosts. shadow-16vm5: shadow-16vm5: Warning: Permanently added 'shadow-16vm5,10.1.4.189' (ECDSA) to the list of known hosts. shadow-16vm6: shadow-16vm6: Warning: Permanently added 'shadow-16vm6,10.1.4.190' (ECDSA) to the list of known hosts. shadow-16vm6: shadow-16vm5: Warning: Permanently added 'shadow-16vm6,10.1.4.190' (ECDSA) to the list of known hosts. replay-ost-single test_0b: @@@@@@ FAIL: import is not in FULL state LustreError inside dmesg for replay-ost-single test 0b in mds : [ 3574.906399] LustreError: 16331:0:(events.c:306:request_in_callback()) event type 2, status -5, service mdt |
| Comments |
| Comment by Sarah Liu [ 16/Sep/15 ] |
|
I think this is a dup of |