-
Bug
-
Resolution: Fixed
-
Medium
-
None
-
None
-
3
-
9223372036854775807
sanity-sec test_79 TIMEOUTs in the review-dne-selinux-ssk-part-2 test group (lustre-reviews, el8.10 and el9.8 x86_64 servers/clients):
Timeout occurred after 313 minutes, last suite running was sanity-sec
The test hangs right after the first client remount with the c0 key (gssonly_identification), at "[[ -f $DIR/this_is_c0 ]]". It is failing across unrelated patches (several of which only touch lustre/tests).
Frequency: 4 TIMEOUT out of 33 non-skipped test_79 runs in lustre-reviews on 2026-10-01..02 (all in review-dne-selinux-ssk-part-2, on both el8.10 and el9.8; test_79 is skipped without shared keys). lustre-master has no sanity-sec runs in the scanned sessions. The code involved (lsvcgssd nodemap lookup, test_79, the full OST remounts in test_70/test_75b) all landed between 2024 and June 2026, so there is no obvious recent landing that started it; a possible but unconfirmed contributor is 14cf8275e232 ("LU-20087 tests: fix sanity-sec/27ad hang with shared key", landed 2026-08-22), which adds another per-nodemap SSK mount before test_70.
Example sessions:
- https://testing.whamcloud.com/test_sessions/8082e408-13ae-49b3-8f3b-fe8a5f00f225 (review 69363, el8.10)
- https://testing.whamcloud.com/test_sessions/f8abc11a-12d3-4990-af7c-9564eaaa76ab (review 63189, el8.10)
- https://testing.whamcloud.com/test_sessions/83e643f3-fb77-4fdc-9b99-6eb02f43770c (review 63188, el8.10)
- https://testing.whamcloud.com/test_sessions/44aa4198-e816-4e66-ac5e-9ac4c8e93742 (el9.8)
- https://testing.whamcloud.com/test_sessions/384b320f-f919-4696-8bb0-9af18f980b0c (el9.8)
Client stat is stuck in a glimpse because GSS context negotiation to the OSTs never succeeds:
LustreError: gss_keyring.c:1858:gss_kt_update()) lustre-OST0000_UUID at 10.240.38.135@tcp: negotiation: rpc err 0, gss err d0000 LustreError: gss_keyring.c:1858:gss_kt_update()) Skipped 3813 previous similar messages task:bash state:I pid:850930 cl_sync_io_wait+0xda/0x350 [obdclass] cl_lock_request+0x1b5/0x1e0 [obdclass] cl_glimpse_lock+0x1ef/0x2a0 [lustre] __cl_glimpse_size+0x168/0x2a0 [lustre] ll_getattr_dentry+0xbec/0x1140 [lustre] vfs_statx+0x8a/0xe0
On the OSS, lsvcgssd sends the gss identification nodemap lookup ioctl to the wrong obd device, so the rsi upcall fails:
LustreError: 9815:0:(ofd_obd.c:1327:ofd_iocontrol()) lustre-OST0000: iocontrol from 'lsvcgssd' cmd=c00866c5 _IOWR('f', 197, 8) unrecognized: rc = -25
LustreError: 597333:0:(gss_svc_upcall.c:904:gss_svc_upcall_handle_init()) lustre-OST0000: failed to get entry from rsi cache (nid 10.240.38.133@tcp): rc = -43
In session 83e643f3 the same ioctl lands on lustre-OST0006-osd on the OSS and on lustre-MDT0003-osd on MDS2:
LustreError: 13487:0:(obd_class.h:1208:obd_iocontrol()) lustre-OST0006-osd: no obd_iocontrol operation LustreError: 5864:0:(obd_class.h:1208:obd_iocontrol()) lustre-MDT0003-osd: no obd_iocontrol operation LustreError: 349718:0:(gss_svc_upcall.c:904:gss_svc_upcall_handle_init()) lustre-MDT0001: failed to get entry from rsi cache (nid 10.240.46.101@tcp): rc = -43
VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-sec test_79 - Timeout occurred after 312 minutes, last suite running was sanity-sec