[LU-12043] improve Lustre single thread read performances

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Labels: LTS12
• Assignee: Wang Shilong (Inactive)
• Reporter: Wang Shilong (Inactive)

Description

There are several efforts here and there which try to improve the performances of single
thread read performances.

this ticket is opened to track a simple enough patch to improve the performances as much
as possible.

Here is whole history:

Currently, for sequential read IO, We grow up window
size very quickly, and once we cached @max_readahead_per_file
pages. For following command:

dd if=/mnt/lustre/file of=/dev/null bs=1M

We will do something like following:
...
64M bytes cached.
fast io for 16M bytes
readahead extra 16M to fill up window.
fast io for 16M bytes
readahead extra 16M to fill up window.
....

In this way, we could only use fast IO for 16M bytes and
then fall through non-fast IO mode. this is also reason
that why increasing @max_readahead_per_file don't give us
performances up, since this value only changes how much
memory we cached in memory, during my testing whatever
I changed the value, i could only get 2GB/s for single thread
read.

Actually, we could do this better, if we have used
more than 16M bytes readahead pages, submit another readahead
requests in the background. and ideally, we could always
use fast IO..I did a quick test with fake-IO on my limited
PC server:

Without patch VS Patched:
~2.0GB/S vs ~3.0 GB/s

So we could gain at least 50% performance up, i supposed
We could get more maybe more than 4GB/S with patch.

Issue Links

• is duplicated by: LU-10557 Should form read RPCs according to read syscal input when readahead is disabled
• is related to: LU-13386 allow over reservation to cover current read
• is related to: LU-11526 Support 64MB I/O RPC
• is related to: LU-13258 Bind linux workqueues to specific core
• is related to: LU-8964 use parallel I/O to improve performance on machines with slow single thread performance
• is related to: LU-8709 parallel asynchronous readahead
• is related to: LU-12518 improve Lustre unaligned IO read performances

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35217/
Subject: LU-12043 llite: fix to submit complete read block with ra disabled
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 7e8efb339b0958146eb294fc9d961688f5d16079

[Wang Shilong (Inactive)]

All patches has been merged.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34849/
Subject: LU-12043 llite: move tunable params to sysfs_memparse()
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 59a3d7237df82ac2ce14b6779577584163f9122b

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37620
Subject: LU-12043 llite: move tunable params to sysfs_memparse()
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 9fa91e61e5daccc9bba90b9c65c9fb054715b778

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35559/
Subject: LU-12043 llite,readahead: don't always use max RPC size
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 57d2dd820ed430009a6146180d11c5e2b04b5426

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35438/
Subject: LU-12043 llite: extend readahead locks for striped file
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cfbeae97d736152e5bdf8f3e759a260f6ee903d8

[Wang Shilong (Inactive)]

Yup, still two patches not landed yet.

https://review.whamcloud.com/35217
LU-12043 llite: fix bad performance with readahead disabled

https://review.whamcloud.com/35438
LU-12043 llite: extend readahead locks for striped file

I think second one will be landed soon, while the first one need be some refreshed.

[Patrick Farrell (Inactive)]

Are there any patches still not landed under this ticket, wshilong ?

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35216/
Subject: LU-12043 llite: don't miss every first stride page
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 29d8eb5ee7dff422294c9ee3d0f34f1370cec800

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/35559
Subject: LU-12043 llite,readahead: don't always use max RPC size
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: eccd7b3873e5395fc4087dec6b09acb91cadc1ef

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35215/
Subject: LU-12043 llite: make sure readahead cover current read
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8fbef5ee761920faa3c84f142f56934d6266a409