[LU-12602] Lustre mdt_getxattr_pack_reply() bug Created: 29/Jul/19 Updated: 23/Sep/19 Resolved: 16/Sep/19 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.13.0 |
| Fix Version/s: | Lustre 2.13.0, Lustre 2.12.3 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Alibaba Cloud | Assignee: | Emoly Liu |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Attachments: |
image-2019-07-29-17-28-29-872.png
image-2019-07-29-17-28-55-842.png
|
||||||||
| Issue Links: |
|
||||||||
| Severity: | 3 | ||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||
| Description |
|
In the latest version of lustre file system, mdt module has a LBUG bug due to the lack of validation for specific fields of packets sent by client. The kenrel panic: [ 3513.346370] Kernel panic - not syncing: LBUG [ 3513.348092] CPU: 2 PID: 3714 Comm: mdt00_004 Kdump: loaded Tainted: G OE ------------ 3.10.0-957.10.1.el7_lustre.x86_64 #1 [ 3513.351756] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 3288b3c 04/01/2014 [ 3513.353768] Call Trace: [ 3513.355288] [<ffffffff98162e41>] dump_stack+0x19/0x1b [ 3513.357062] [<ffffffff9815c550>] panic+0xe8/0x21f [ 3513.358896] [<ffffffffc056e8cb>] lbug_with_loc+0x9b/0xa0 [libcfs] [ 3513.360755] [<ffffffffc0961c8f>] req_capsule_set_size+0x15f/0x1a0 [ptlrpc] [ 3513.362672] [<ffffffffc101f825>] mdt_getxattr+0x7a5/0x1260 [mdt] [ 3513.364493] [<ffffffffc0ffec50>] ? mdt_object_lock_internal+0x70/0x360 [mdt] [ 3513.366397] [<ffffffffc09392dc>] ? lustre_msg_get_flags+0x2c/0xa0 [ptlrpc] [ 3513.368279] [<ffffffffc1007f43>] mdt_intent_getxattr+0xc3/0x2c0 [mdt] [ 3513.370101] [<ffffffffc10049e4>] mdt_intent_policy+0x2d4/0xdd0 [mdt] [ 3513.371910] [<ffffffffc1007e80>] ? mdt_intent_getattr+0x480/0x480 [mdt] [ 3513.373741] [<ffffffffc08ecc66>] ldlm_lock_enqueue+0x356/0xa20 [ptlrpc] [ 3513.375561] [<ffffffffc05783d3>] ? cfs_hash_bd_add_locked+0x63/0x80 [libcfs] [ 3513.377410] [<ffffffffc057b96e>] ? cfs_hash_add+0xbe/0x1a0 [libcfs] [ 3513.379211] [<ffffffffc0915587>] ldlm_handle_enqueue0+0xa47/0x15a0 [ptlrpc] [ 3513.381061] [<ffffffffc093d520>] ? lustre_swab_ldlm_lock_desc+0x30/0x30 [ptlrpc] [ 3513.382952] [<ffffffffc099e082>] tgt_enqueue+0x62/0x210 [ptlrpc] [ 3513.384719] [<ffffffffc09a42ca>] tgt_request_handle+0x91a/0x15c0 [ptlrpc] [ 3513.386524] [<ffffffffc0574fa7>] ? libcfs_debug_msg+0x57/0x80 [libcfs] [ 3513.388283] [<ffffffffc094788e>] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [ 3513.390126] [<ffffffff97acbadb>] ? __wake_up_common+0x5b/0x90 [ 3513.391810] [<ffffffffc094b384>] ptlrpc_main+0xbb4/0x20f0 [ptlrpc] [ 3513.393498] [<ffffffff97ad08c0>] ? finish_task_switch+0x50/0x1c0 [ 3513.395167] [<ffffffffc094a7d0>] ? ptlrpc_register_service+0xfa0/0xfa0 [ptlrpc] [ 3513.396915] [<ffffffff97ac1c71>] kthread+0xd1/0xe0 [ 3513.398390] [<ffffffff97ac1ba0>] ? insert_kthread_work+0x40/0x40 [ 3513.399973] [<ffffffff98175c1d>] ret_from_fork_nospec_begin+0x7/0x21 [ 3513.401565] [<ffffffff97ac1ba0>] ? insert_kthread_work+0x40/0x40 In fucntion mdt_getxattr_pack_reply, it don't check the vaule of mbo_eadatasize and pass it to the req_capsule_set_size function. In function req_capsule_set_size, it checks if the condition of 'size%4==0' is satisfied. If it is not, we will trigger LBUG() and cause kernel panic. The `mbo_eadatasize` parameter is derived from the packet whose lustre request is `LDLM_ENQUEUE`. The attacker can modify the `eadatasize` parameter in the `MDT Body` section of the packet to a larger multiple of 4 (eg 0x44444444).
The backtrace: ptlrpc_main -> ptlrpc_sever_handle_request -> tgt_request_handle -> tgt_enqueue -> ldlm_handle_enqueue0 -> ldlm_lock_enqueue -> mdt_intent_policy -> mdt_intent_getxattr -> mdt_getxattr -> mdt_getxattr_pack_reply -> req_capsule_set_size |
| Comments |
| Comment by Peter Jones [ 30/Jul/19 ] |
|
Emoly Could you please look into this one Peter |
| Comment by Andreas Dilger [ 01/Aug/19 ] |
|
Please add "Reported-by: Alibaba Cloud <yunye.ry@alibaba-inc.com>" to the patch commit message. |
| Comment by Gerrit Updater [ 12/Aug/19 ] |
|
Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35768 |
| Comment by Gerrit Updater [ 27/Aug/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35768/ |
| Comment by Peter Jones [ 27/Aug/19 ] |
|
Landed for 2.13 |
| Comment by Gerrit Updater [ 27/Aug/19 ] |
|
Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/35936 |
| Comment by Peter Jones [ 31/Aug/19 ] |
|
Reopening until Gerrit comments in the master patch have been addressed. |
| Comment by Gerrit Updater [ 09/Sep/19 ] |
|
Emoly Liu (emoly@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36103 |
| Comment by Gerrit Updater [ 16/Sep/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36103/ |
| Comment by Peter Jones [ 16/Sep/19 ] |
|
Second patch landed for 2.13 |
| Comment by Gerrit Updater [ 17/Sep/19 ] |
|
Minh Diep (mdiep@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36208 |
| Comment by Gerrit Updater [ 18/Sep/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/35936/ |
| Comment by Gerrit Updater [ 23/Sep/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36208/ |
