This issue was created by maloo for Andreas Dilger <adilger@dilger.ca>

This issue relates to the following test suite run:
https://testing.whamcloud.com/test_sets/e7e79f88-b555-4a79-9fc6-0f9a9d1fd879

test_230w failed with the following error when running with a 2.15 server with multiple MDTs:

lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d230w.sanity: Invalid argument (22)
d230w.sanity stripe count mismatch

Test session details:

Interop

Client
Lustre: 2.17.57.103
Branch: master
Build: https://build.whamcloud.com/job/lustre-reviews/130753
Kernel: 5.14.0-611.5.1.el9_7.x86_64

Server
Lustre: 2.15.8
Branch: b2_15
Build: https://build.whamcloud.com/job/lustre-b2_15/117
Kernel: 4.18.0-553.82.1.el8_lustre.x86_64

The test is skipped when run with MDSCOUNT=1 (which is unfortunately the default for branch landing interop sessions), so it only fails when explicitly run with "Test-Parameters: testlist=sanity serverversion=2.15".

Both MDS nodes report messages like:

[ 9601.587759] Lustre: DEBUG MARKER: == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 05:11:09 (1788325869)
[ 9602.473089] LustreError: 616183:0:(mdt_xattr.c:435:mdt_dir_layout_update()) lustre-MDT0000: [0x2000061c1:0x364a:0x0] migrate mdt hash mismatch 2181038083 != 33554432
[ 9603.023701] LustreError: 616183:0:(mdt_xattr.c:435:mdt_dir_layout_update()) lustre-MDT0002: [0x280002346:0x14c5:0x0] migrate mdt hash mismatch 2181038083 != 33554432
[ 9603.026088] LustreError: 616183:0:(mdt_xattr.c:435:mdt_dir_layout_update()) Skipped 3 previous similar messages
[ 9604.284707] LustreError: 616183:0:(mdt_xattr.c:435:mdt_dir_layout_update()) lustre-MDT0000: [0x2000061c1:0x3656:0x0] migrate mdt hash mismatch 2181038083 != 33554432
[ 9604.287072] LustreError: 616183:0:(mdt_xattr.c:435:mdt_dir_layout_update()) Skipped 12 previous similar messages
[ 9606.369832] LustreError: 607762:0:(mdt_xattr.c:435:mdt_dir_layout_update()) lustre-MDT0000: [0x2000061c1:0x3664:0x0] migrate mdt hash mismatch 2181038083 != 33554432

These hash values are unfortunately printed as decimal (which should also be fixed), but in hex they are 0x82000003 = LMV_HASH_FLAG_MIGRATION|LMV_HASH_FLAG_FIXED|LMV_HASH_TYPE_CRUSH, and 0x2000000 = LMV_HASH_FLAG_FIXED, so it looks like this is a real protocol interop issue with 2.15 related to "lfs migrate -m".

It might relate to patch https://review.whamcloud.com/58932 ("LU-18904 llite: set hash fnv_1a_64 for old server") which was landed in commit v2_16_58-42-g19e6393c24 and claims an interop fix for 2.12 by checking for OBD_CONNECT2_CRUSH, or patch https://review.whamcloud.com/57784 ("LU-18639 dne: a correct check for dir split") which changed test_230p, but there seems to be something else that is is needed (different connect flag or hash version check).

Especially since 2.18 is an LTS release, it needs to interoperate properly with the previous LTS release 2.15.

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_230w - d230w.sanity stripe count mismatch