[LU-17911] Faked flexible array usage causes crash when Fortify feature is enabled

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.16.0
• Severity: 3
• Assignee: Bruno Faccini
• Reporter: Bruno Faccini

Description

This seems to start to happen with v6 Kernel versions.
"lmv fid2path" crashes solid for any file with the following stack trace :

2024-06-03T02:33:16.895852-07:00 foo0164 kernel: detected buffer overflow in __fortify_strlen
2024-06-03T02:33:16.895862-07:00 foo0164 kernel: ------------[ cut here ]------------
2024-06-03T02:33:16.895865-07:00 foo0164 kernel: kernel BUG at lib/string_helpers.c:1048!
2024-06-03T02:33:16.895865-07:00 foo0164 kernel: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
2024-06-03T02:33:17.445719-07:00 foo0164 kernel: CPU: 217 PID: 2690017 Comm: lfs Tainted: G        W  OE      6.8.0-31-generic #31-Ubuntu
...................
2024-06-03T02:33:17.445728-07:00 foo0164 kernel: RIP: 0010:fortify_panic+0x13/0x20
2024-06-03T02:33:17.445729-07:00 foo0164 kernel: Code: cc 66 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 fe 48 c7 c7 70 97 2a 98 48 89 e5 e8 9d b6 94 ff <0f> 0b 90 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90
2024-06-03T02:33:17.445730-07:00 foo0164 kernel: RSP: 0018:ff32067b5e45f990 EFLAGS: 00010246
2024-06-03T02:33:17.445732-07:00 foo0164 kernel: RAX: 000000000000002c RBX: ff1d4eab96f30800 RCX: 0000000000000000
2024-06-03T02:33:17.445733-07:00 foo0164 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
2024-06-03T02:33:17.445734-07:00 foo0164 kernel: RBP: ff32067b5e45f990 R08: 0000000000000000 R09: 0000000000000000
2024-06-03T02:33:17.445735-07:00 foo0164 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ff1d50225095c000
2024-06-03T02:33:17.445736-07:00 foo0164 kernel: R13: 0000000000001020 R14: 0000000000001020 R15: ff1d50225095c000
2024-06-03T02:33:17.445737-07:00 foo0164 kernel: FS:  00007aa293b91740(0000) GS:ff1d50a5bfa80000(0000) knlGS:0000000000000000
2024-06-03T02:33:17.445738-07:00 foo0164 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2024-06-03T02:33:17.445739-07:00 foo0164 kernel: CR2: 0000591faf6b88d8 CR3: 00000101c8600003 CR4: 0000000000f71ef0
2024-06-03T02:33:17.445739-07:00 foo0164 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
2024-06-03T02:33:17.445740-07:00 foo0164 kernel: DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
2024-06-03T02:33:17.445749-07:00 foo0164 kernel: PKRU: 55555554
2024-06-03T02:33:17.445749-07:00 foo0164 kernel: Call Trace:
2024-06-03T02:33:17.445749-07:00 foo0164 kernel:  <TASK>
2024-06-03T02:33:17.445750-07:00 foo0164 kernel:  ? show_regs+0x6d/0x80
2024-06-03T02:33:17.445750-07:00 foo0164 kernel:  ? die+0x37/0xa0
2024-06-03T02:33:17.445750-07:00 foo0164 kernel:  ? do_trap+0xd4/0xf0
2024-06-03T02:33:17.445751-07:00 foo0164 kernel:  ? do_error_trap+0x71/0xb0
2024-06-03T02:33:17.445753-07:00 foo0164 kernel:  ? fortify_panic+0x13/0x20
2024-06-03T02:33:17.445757-07:00 foo0164 kernel:  ? exc_invalid_op+0x52/0x80
2024-06-03T02:33:17.445757-07:00 foo0164 kernel:  ? fortify_panic+0x13/0x20
2024-06-03T02:33:17.445758-07:00 foo0164 kernel:  ? asm_exc_invalid_op+0x1b/0x20
2024-06-03T02:33:17.445758-07:00 foo0164 kernel:  ? fortify_panic+0x13/0x20
2024-06-03T02:33:17.445758-07:00 foo0164 kernel:  lmv_fid2path+0x7a2/0x830 [lmv]
2024-06-03T02:33:17.445759-07:00 foo0164 kernel:  lmv_iocontrol+0x505/0x2c20 [lmv]
2024-06-03T02:33:17.445759-07:00 foo0164 kernel:  ? ldlm_lock_decref_internal+0x41f/0x5c0 [ptlrpc]
2024-06-03T02:33:17.445761-07:00 foo0164 kernel:  ? rmqueue+0x825/0xee0
2024-06-03T02:33:17.445761-07:00 foo0164 kernel:  obd_iocontrol+0x72/0x330 [lustre]
2024-06-03T02:33:17.445761-07:00 foo0164 kernel:  __ll_fid2path+0x64/0x5b0 [lustre]
2024-06-03T02:33:17.445761-07:00 foo0164 kernel:  ? get_page_from_freelist+0x452/0x6d0
2024-06-03T02:33:17.445762-07:00 foo0164 kernel:  ? __kmalloc+0x1c0/0x4e0
2024-06-03T02:33:17.445762-07:00 foo0164 kernel:  ll_fid2path+0x33a/0x580 [lustre]
2024-06-03T02:33:17.445764-07:00 foo0164 kernel:  ll_dir_ioctl+0x203f/0x5e90 [lustre]

2024-06-03T02:33:17.445764-07:00 foo0164 kernel:  ? __mod_memcg_lruvec_state+0xd6/0x1a0 2024-06-03T02:33:17.445765-07:00 foo0164 kernel:  ? __mod_lruvec_state+0x36/0x50 2024-06-03T02:33:17.445765-07:00 foo0164 kernel:  ? __lruvec_stat_mod_folio+0x70/0xc0 2024-06-03T02:33:17.445765-07:00 foo0164 kernel:  ? set_ptes.isra.0+0x2b/0xb0 2024-06-03T02:33:17.445765-07:00 foo0164 kernel:  ? do_anonymous_page+0x1a3/0x430 2024-06-03T02:33:17.445767-07:00 foo0164 kernel:  ? handle_pte_fault+0x1cb/0x1d0 2024-06-03T02:33:17.445768-07:00 foo0164 kernel:  ? __handle_mm_fault+0x653/0x790 2024-06-03T02:33:17.445768-07:00 foo0164 kernel:  ? __count_memcg_events+0x6b/0x120 2024-06-03T02:33:17.445768-07:00 foo0164 kernel:  __x64_sys_ioctl+0xa0/0xf0 2024-06-03T02:33:17.445768-07:00 foo0164 kernel:  ? __x64_sys_ioctl+0xa0/0xf0 2024-06-03T02:33:17.445769-07:00 foo0164 kernel:  x64_sys_call+0x143b/0x25c0 2024-06-03T02:33:17.445769-07:00 foo0164 kernel:  do_syscall_64+0x7f/0x180 2024-06-03T02:33:17.445770-07:00 foo0164 kernel:  ? exc_page_fault+0x94/0x1b0 2024-06-03T02:33:17.445771-07:00 foo0164 kernel:  entry_SYSCALL_64_after_hwframe+0x73/0x7b 2024-06-03T02:33:17.445771-07:00 foo0164 kernel: RIP: 0033:0x7aa293e63ded 2024-06-03T02:33:17.445772-07:00 foo0164 kernel: Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 2024-06-03T02:33:17.445772-07:00 foo0164 kernel: RSP: 002b:00007ffdcae82320 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 2024-06-03T02:33:17.445772-07:00 foo0164 kernel: RAX: ffffffffffffffda RBX: 00007ffdcae8242c RCX: 00007aa293e63ded 2024-06-03T02:33:17.445773-07:00 foo0164 kernel: RDX: 0000591faf6b78b0 RSI: 00000000c0086696 RDI: 0000000000000003 2024-06-03T02:33:17.445774-07:00 foo0164 kernel: RBP: 00007ffdcae82370 R08: 00007aa293f42b20 R09: 0000000000000000 2024-06-03T02:33:17.445774-07:00 foo0164 kernel: R10: 00007aa293d57808 R11: 0000000000000246 R12: 00007ffdcae82440 2024-06-03T02:33:17.445775-07:00 foo0164 kernel: R13: 00007ffdcae82438 R14: 0000000000001000 R15: 0000000000000003 2024-06-03T02:33:17.445775-07:00 foo0164 kernel:  </TASK> ....................... 2024-06-03T02:33:17.445778-07:00 foo0164 kernel: ---[ end trace 0000000000000000 ]--- 2024-06-03T02:33:17.525908-07:00 foo0164 kernel: RIP: 0010:fortify_panic+0x13/0x20 2024-06-03T02:33:17.525918-07:00 foo0164 kernel: Code: cc 66 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 fe 48 c7 c7 70 97 2a 98 48 89 e5 e8 9d b6 94 ff <0f> 0b 90 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90 2024-06-03T02:33:17.532153-07:00 foo0164 kernel: RSP: 0018:ff32067b5e45f990 EFLAGS: 00010246 2024-06-03T02:33:17.540659-07:00 foo0164 kernel: RAX: 000000000000002c RBX: ff1d4eab96f30800 RCX: 0000000000000000 2024-06-03T02:33:17.549145-07:00 foo0164 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 2024-06-03T02:33:17.557643-07:00 foo0164 kernel: RBP: ff32067b5e45f990 R08: 0000000000000000 R09: 0000000000000000 2024-06-03T02:33:17.566118-07:00 foo0164 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ff1d50225095c000 2024-06-03T02:33:17.574599-07:00 foo0164 kernel: R13: 0000000000001020 R14: 0000000000001020 R15: ff1d50225095c000 2024-06-03T02:33:17.584153-07:00 foo0164 kernel: FS:  00007aa293b91740(0000) GS:ff1d50a5bfa80000(0000) knlGS:0000000000000000 2024-06-03T02:33:17.591189-07:00 foo0164 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2024-06-03T02:33:17.599551-07:00 foo0164 kernel: CR2: 0000591faf6b88d8 CR3: 00000101c8600003 CR4: 0000000000f71ef0 2024-06-03T02:33:17.608125-07:00 foo0164 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 2024-06-03T02:33:17.616483-07:00 foo0164 kernel: DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400 2024-06-03T02:33:17.619977-07:00 foo0164 kernel: PKRU: 55555554

having a look to the disassembly of lmv_fid2path() routine, it looks like the code, for the first strlen() call, generates this panic with no alternative.

After reading the document at https://people.kernel.org/kees/bounded-flexible-arrays-in-c , it seems like faked flexible arrays need some rework to comply with new coding rules.

And this seems to be confirmed by the fact that moving from :

/** fid2path request/reply structure */
struct getinfo_fid2path {
        struct lu_fid   gf_fid; 
        __u64           gf_recno;
        __u32           gf_linkno;
        __u32           gf_pathlen;
        union {
                char            gf_path[0];
                struct lu_fid   gf_root_fid[0];
        } gf_u;
} __attribute__((packed)); 

to

/** fid2path request/reply structure */
struct getinfo_fid2path {
        struct lu_fid   gf_fid;
        __u64           gf_recno;
        __u32           gf_linkno;
        __u32           gf_pathlen;
        union { 
                struct {
                        struct { } __unused_member1;
                        char gf_path[];
                };
                struct {
                        struct { } __unused_member2;
                        struct lu_fid gf_root_fid[];
                };
        } gf_u;
} __attribute__((packed)); 

fixes the problem.

I will push a patch to Gerrit soon, but looks like more work in this direction may be needed for full Lustre code to be compliant...

 

Issue Links

• is related to: LU-16363 build fix for fiemap flexible array wiretest
• is related to: LU-17504 build errors with gcc-13
• is related to: LU-17545 memcpy: detected field-spanning write (size 64) of single field "&lp->lp_data->pb_info" at .../lnet/lnet/peer.c:2456 (size 16)
• is related to: LU-17784 improve wiretest for flexible arrays