[LU-12869] interop: lustre-rsync-test test 2a fails with 'Failure in replication; differences found.'

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.12.3
• Type: Bug
• Priority: Minor
• Reporter: James Nunez (Inactive)
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: interop
• Severity: 3

Description

We have seen lustre-rsync-test test_2a fail with 'Failure in replication; differences found.' once for 2.10.8 servers and 2.12.3 RC1 testing; see https://testing.whamcloud.com/test_sets/93e5d5ae-eac2-11e9-9874-52540065bddc . Note that this test fails 100% of the time for PPC client testing. This ticket does not cover the PPC failures.

In the suite_log, we don’t see any obvious errors

Throughput 2.75769 MB/sec  2 clients  2 procs  max_latency=2835.173 ms
Lustre filesystem: lustre
MDT device: lustre-MDT0000
Source: /mnt/lustre
Target: /tmp/target
Target: /tmp/target2
Statuslog: /tmp/lustre_rsync.log
Changelog registration: cl3
Starting changelog record: 0
Clear changelog after use: no
Errors: 0
lustre_rsync took 168 seconds
Changelog records consumed: 7580
Only in /mnt/lustre/d2a.lustre-rsync-test/clients/client0: ~dmtmp
 lustre-rsync-test test_2a: @@@@@@ FAIL: Failure in replication; differences found. 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:5864:error()
  = /usr/lib64/lustre/tests/lustre-rsync-test.sh:122:check_diff()
  = /usr/lib64/lustre/tests/lustre-rsync-test.sh:293:test_2a()

Looking at the console logs, we don’t see any obvious errors.

Issue Links

• duplicates: LU-11426 2/2 Olafs agree: changelog entries ar...

Comments

[Andreas Dilger]

It seems likely that this is caused by LU-11426 problems with ChangeLog records being skipped on the 2.10.8 server. There is only a single file difference in the target copy, which seems similar to that problem. If there was a serious problem, it would have had dozens of files different as is typically seen with lustre_rsync problems.

[Alexander Zarochentsev]

recently seen not in interopt testing : https://testing.whamcloud.com/sub_tests/2a65106b-6525-459b-9933-f0548ebaf4e1