[LU-6158] always shrink_capsule in mdt_getxattr_all

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.8.0
• Type: Bug
• Priority: Critical
• Reporter: Sergey Cheremencev
• Assignee: Lai Siyao
• Resolution: Fixed
• Labels: patch
• Severity: 3

Description

When mdt_getxattr_one returns error, capsule should be
shrunk to zero before exit in mdt_getxattr_all. Otherwise reply may include rubbish and could be bigger 1 MB.

In seagate we faced a kernel panic caused this issue. Panic occurred in o2iblnd because reply size was > 1 MB.

[260641.975629] BUG: unable to handle kernel NULL pointer dereference at (null)
[260641.983967] IP: [<ffffffff81291d32>] sg_next+0x2/0x30
[260641.990153] PGD 0 
[260641.992912] Oops: 0000 [#1] SMP 
[260641.997062] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/irq
[260642.006436] CPU 14 
[260642.008694] Modules linked in: ost(U) mgs(U) osc(U) lmv(U) ksocklnd(U) osp(U) mdd(U) lfsck(U) lod(U) mdt(U) mgc(U) osd_ldiskfs(U) lquota(U) ldiskfs(U) linear raid10 raid456 asyn
c_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx lustre(U) lov(U) mdc(U) fid(U) fld(U) ko2iblnd(U) ptlrpc(U) obdclass(U) lnet(U) sha512_generic sha256_generic crc
32c_intel libcfs(U) ext4 jbd2 mbcache ib_ipoib(U) rdma_ucm(U) ib_ucm(U) ib_uverbs(U) ib_umad(U) rdma_cm(U) ib_cm(U) iw_cm(U) mlx4_ib(U) ib_sa(U) ib_mad(U) ib_core(U) ib_addr(U) nf_c
onntrack_ipv4 nf_defrag_ipv4 xt_state xt_multiport iptable_filter xt_NOTRACK nf_conntrack iptable_raw ip_tables ipmi_devintf cpufreq_ondemand acpi_cpufreq freq_table mperf dm_mod sg
 ses enclosure sd_mod crc_t10dif wmi iTCO_wdt iTCO_vendor_support isci libsas mpt2sas scsi_transport_sas raid_class i2c_i801 lpc_ich mfd_core ahci shpchp nfs lockd fscache auth_rpcg
ss nfs_acl sunrpc igb dca i2c_algo_bit i2c_core mlx4_en(U) ptp pps_core mlx4_core(U) compat(U) bonding ipv6 8021q garp stp llc [last unloaded: ib_core]
[260642.130255] 
[260642.132423] Pid: 139276, comm: mdt02_000 Not tainted 2.6.32-431.17.1.x2.0.43.x86_64 #1 Intel Corporation S2600JF/S2600JF
[260642.145551] RIP: 0010:[<ffffffff81291d32>]  [<ffffffff81291d32>] sg_next+0x2/0x30
[260642.154849] RSP: 0018:ffff8807af5dd908  EFLAGS: 00010246
[260642.161295] RAX: 0000000000000000 RBX: ffff88101247d000 RCX: 0000000000000000
[260642.170181] RDX: 0000000000000101 RSI: ffffc900191805d8 RDI: 0000000000000000
[260642.179064] RBP: ffff8807af5dd980 R08: ffffea002b4d8108 R09: 0000000000000301
[260642.187950] R10: 0000000000001000 R11: 0000000000000000 R12: ffff88083001dec0
[260642.196835] R13: ffff881012476000 R14: ffffc900191805d8 R15: ffff8810336bb090
[260642.205721] FS:  0000000000000000(0000) GS:ffff88085c480000(0000) knlGS:0000000000000000
[260642.215676] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[260642.222610] CR2: 0000000000000000 CR3: 0000000001a85000 CR4: 00000000001407e0
[260642.231490] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[260642.240379] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[260642.249266] Process mdt02_000 (pid: 139276, threadinfo ffff8807af5dc000, task ffff880833576ae0)
[260642.259909] Stack:
[260642.262653]  ffffffffa0a6a2fe 0000000000000010 0000000000000000 ffff880700000101
[260642.271283] <d> ffffffff81a98ec0 ffff880832e45dc0 0000030100000001 ffffffffffffffff
[260642.280841] <d> 0000000affffffff ffffffffffffffff ffff8808216852c0 ffffc900191805d8
[260642.291006] Call Trace:
[260642.294269]  [<ffffffffa0a6a2fe>] ? kiblnd_map_tx+0x19e/0x540 [ko2iblnd]
[260642.302285]  [<ffffffffa0a6ab6a>] kiblnd_setup_rd_iov+0x13a/0x2b0 [ko2iblnd]
[260642.310691]  [<ffffffffa0a7029a>] kiblnd_send+0x5da/0x9b0 [ko2iblnd]
[260642.318374]  [<ffffffffa03faedb>] lnet_ni_send+0x4b/0xf0 [lnet]
[260642.325522]  [<ffffffffa03ff046>] lnet_send+0x656/0xb60 [lnet]
[260642.332566]  [<ffffffffa040006a>] LNetPut+0x30a/0x850 [lnet]
[260642.339507]  [<ffffffffa086f160>] ptl_send_buf+0x1e0/0x550 [ptlrpc]
[260642.347103]  [<ffffffffa088f3e8>] ? at_measured+0x108/0x380 [ptlrpc]
[260642.354800]  [<ffffffffa08b13d5>] ? null_authorize+0x75/0x100 [ptlrpc]
[260642.362674]  [<ffffffffa086f74b>] ptlrpc_send_reply+0x27b/0x7f0 [ptlrpc]
[260642.370736]  [<ffffffffa0838784>] target_send_reply_msg+0x54/0x190 [ptlrpc]
[260642.379086]  [<ffffffffa0838ca6>] target_send_reply+0x3e6/0x720 [ptlrpc]
[260642.387158]  [<ffffffffa087604c>] ? lustre_msg_set_last_committed+0x6c/0xc0 [ptlrpc]
[260642.396803]  [<ffffffffa08d8370>] tgt_request_handle+0x2c0/0xac0 [ptlrpc]
[260642.404972]  [<ffffffffa0887e6a>] ptlrpc_main+0xd1a/0x1960 [ptlrpc]
[260642.412551]  [<ffffffffa0887150>] ? ptlrpc_main+0x0/0x1960 [ptlrpc]
[260642.420074]  [<ffffffff8109ac66>] kthread+0x96/0xa0
[260642.426040]  [<ffffffff8100c20a>] child_rip+0xa/0x20
[260642.432099]  [<ffffffff8109abd0>] ? kthread+0x0/0xa0
[260642.438158]  [<ffffffff8100c200>] ? child_rip+0x0/0x20
[260642.444411] Code: 5c 41 5d 41 5e 41 5f c9 c3 55 48 c7 c2 c0 22 29 81 be 80 00 00 00 48 89 e5 e8 6b ff ff ff c9 c3 66 0f 1f 84 00 00 00 00 00 31 c0 <f6> 07 02 55 48 89 e5 75 0d 48 8b 57 20 48 8d 47 20 f6 c2 01 75 
[260642.468112] RIP  [<ffffffff81291d32>] sg_next+0x2/0x30
[260642.474390]  RSP <ffff8807af5dd908>
[260642.478781] CR2: 0000000000000000

Issue Links

• is duplicated by: LU-6799 getxattr failed: -2 triggers a Kernel...

Comments

[Gerrit Updater]

Sergey Cheremencev ([e-mail]) uploaded a new patch: http://review.whamcloud.com/13524
Subject: LU-6158 mdt: always shrink_capsule in getxattr_all
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 61c93a95e7706d59c8d34e7e713ac185c883caf9

[Sergey Cheremencev]

Patch that helped in seagate http://review.whamcloud.com/13524

[Peter Jones]

Lai

Could you please review this patch?

Thanks

Peter

[Bruno Travouillon (Inactive)]

Should'nt we shrink the capsule when mdt_getxattr_one is called by mdt_getxattr and returns an error?

220 int mdt_getxattr(struct mdt_thread_info *info)
221 {
[...]
282     if (valid == OBD_MD_FLXATTR) {
283         char *xattr_name = req_capsule_client_get(info->mti_pill,
284                               &RMF_NAME);
285         rc = mdt_getxattr_one(info, xattr_name, next, buf, med, uc);
286     } else if (valid == OBD_MD_FLXATTRLS) {
[...]

[Sergey Cheremencev]

Should'nt we shrink the capsule when mdt_getxattr_one is called by mdt_getxattr and returns an error?

I am not sure it is reasonable here.
We get the easize in mdt_getxattr_pack_reply(calling mo_xattr_get and mo_xattr_list):

       if (valid == OBD_MD_FLXATTR) {
                xattr_name = req_capsule_client_get(pill, &RMF_NAME);
                if (!xattr_name)
                        RETURN(-EFAULT);

                if (!(exp_connect_flags(req->rq_export) & OBD_CONNECT_XATTR) &&
                    !strncmp(xattr_name, user_string, sizeof(user_string) - 1)) 
                        RETURN(-EOPNOTSUPP);

                size = mo_xattr_get(info->mti_env,
                                    mdt_object_child(info->mti_object),
                                    &LU_BUF_NULL, xattr_name);
        } else if (valid == OBD_MD_FLXATTRLS) {
                size = mo_xattr_list(info->mti_env,
                                     mdt_object_child(info->mti_object),
                                     &LU_BUF_NULL);

Exception is only XATTRALL:

        } else if (valid == OBD_MD_FLXATTRALL) {
                /* N.B. eadatasize = 0 is not valid for FLXATTRALL */
                /* We could calculate accurate sizes, but this would
                 * introduce a lot of overhead, let's do it later... */
                size = info->mti_body->mbo_eadatasize;

Thus we need to shrink the capsule inside of mdt_getxattr_all.

[Lai Siyao]

hmm, easize is always correctly set before calling mdt_getxattr_one(), so it's not necessary to shrink capsule upon failure.

[Bruno Travouillon (Inactive)]

Indeed, you're right.

Is the patch ready to merge?

[Lai Siyao]

yes, it's on the landing list.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/13524/
Subject: LU-6158 mdt: always shrink_capsule in getxattr_all
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b046468f58a1f40e85cb59ed9abf75fd2fd5ea5a

[Joseph Gmitter (Inactive)]

Landed for 2.8.