[LU-13626] sanity-lfsck test 2b fails with ‘(4) unexpected status’

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.12.5
• Type: Bug
• Priority: Minor
• Reporter: James Nunez (Inactive)
• Assignee: WC Triage
• Resolution: Unresolved
• Severity: 3

Description

sanity-lfsck test_2b fails with ‘(4) unexpected status’.

Looking at the failure for 2.12.4.66 at https://testing.whamcloud.com/test_sets/865fbc93-bf34-4241-b926-4ee2e0fa2f3e , in the suite_log, we see that the status of LFSCK is 'scanning-phase2', but we wanted to see ‘completed’

CMD: onyx-40vm5 /usr/sbin/lctl get_param -n 		mdd.lustre-MDT0000.lfsck_namespace |
		awk '/^status/ { print \$2 }'
Update not seen after 32s: wanted 'completed' got 'scanning-phase2'
CMD: onyx-40vm5 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace
name: lfsck_namespace
magic: 0xa06249ff
version: 2
status: scanning-phase2
flags: scanned-once,inconsistent
param:
last_completed_time: 1590228456
time_since_last_completed: 87 seconds
latest_start_time: 1590228499
time_since_latest_start: 44 seconds
last_checkpoint_time: 1590228499
time_since_last_checkpoint: 44 seconds
latest_start_position: 12, N/A, N/A
last_checkpoint_position: 20138, [0x200000402:0x1:0x0], 0x590db2753fd97cf2
first_failure_position: N/A, N/A, N/A
checked_phase1: 89
checked_phase2: 16
updated_phase1: 0
updated_phase2: 1
failed_phase1: 0
failed_phase2: 0
directories: 5
dirent_repaired: 0
linkea_repaired: 0
nlinks_repaired: 0
multiple_linked_checked: 1
multiple_linked_repaired: 0
unknown_inconsistency: 0
unmatched_pairs_repaired: 0
dangling_repaired: 0
multiple_referenced_repaired: 0
bad_file_type_repaired: 0
lost_dirent_repaired: 0
local_lost_found_scanned: 0
local_lost_found_moved: 0
local_lost_found_skipped: 0
local_lost_found_failed: 0
striped_dirs_scanned: 0
striped_dirs_repaired: 0
striped_dirs_failed: 0
striped_dirs_disabled: 0
striped_dirs_skipped: 0
striped_shards_scanned: 0
striped_shards_repaired: 0
striped_shards_failed: 0
striped_shards_skipped: 0
name_hash_repaired: 0
linkea_overflow_cleared: 0
agent_entries_repaired: 0
success_count: 6
run_time_phase1: 0 seconds
run_time_phase2: 9 seconds
average_speed_phase1: 89 items/sec
average_speed_phase2: 1 objs/sec
average_speed_total: 10 items/sec
real_time_speed_phase1: N/A
real_time_speed_phase2: 1 objs/sec
current_position: [0x0:0x0:0x0]
 sanity-lfsck test_2b: @@@@@@ FAIL: (4) unexpected status 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:5907:error()
  = /usr/lib64/lustre/tests/sanity-lfsck.sh:398:test_2b()

In the past 8 months, we've only seen this test fail with this error twice on the b2_12 and derived branches on May 23 and 30, 2020.