[LU-16640] parallellize generic I/O

• Status: Open
• Project: Lustre
• Type: Improvement
• Priority: Minor
• Reporter: Patrick Farrell
• Assignee: Patrick Farrell
• Resolution: Unresolved

Description

LU-8964 tried to improve I/O performance by splitting I/Os to multiple threads at the ll_file/cl_io level (ie, the top level).  This is a decent idea, but had some problems.

Specifically, the original LU-8964 parallel I/O work didn't improve performance much and in some cases harmed it (mostly because of limitations in the buffered I/O path, and the synchronous nature of the DIO path at the time), the new parallel DIO path should make it possible to get large performance improvements with roughly the same approach as in LU-8964.  It's mostly a matter of using this or not intelligently - We will only do parallel IO on the DIO path, since it's mostly a net loss on the buffered path.  We will also add some sized based tuning.

But the patch will be modeled on the approach taken in LU-8964.

Issue Links

• is related to: LU-16540 move ll_get_user_pages to ptlrpcd thr...

Comments

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50280
Subject: LU-16640 llite: add parallel io work queue
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 356756289459826cc716ab6af0e3cea2c32d0801

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50282
Subject: LU-16640 llite: add __ll_file_io_generic
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: eb6e3817917f06a8875488af1d8d7c9597dba131

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50295
Subject: LU-16640 llite: add pio_list to ll_file_io_generic
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e6c6834bafc6ca66970c462b16b7f827a25c8cf0

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50296
Subject: LU-16640 llite: Add ci_pio and conditions
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 92e4f350590dec2bd5a303a2656b2888f7498d61

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50297
Subject: LU-16640 llite: convert to cl_rw_io
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: fe6cf640a1c8ea8b0d20f68388213ae051f414b6

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50306
Subject: LU-16640 llite: Add other pio flag checks
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0a2b272554fe1c88653f47bce944402db446968c

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50307
Subject: LU-16640 tests: add old test
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: bc08740fc1015110c2fe145517032db732b8fa81

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50308
Subject: LU-16640 clio: Add most supporting code for pio
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: bcda7c831c852b063a508ae4435b85eca1f20226

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50380
Subject: LU-16640 llite: remove vui_iter and vui_iocb
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b208981d20312c4fb0d27c7dcfe12b2d95fbc11c