https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13669 Lustre <p>We have observed slow performance on mmap read on some applications.</p> <p>The problem is if access pattern is neither sequential nor stride reading, but<br/> still adjacent in a small range and then seek a random position.</p> <p>So the pattern could be something like this:</p> <p><span class="error">&#91;1M data&#93;</span> <span class="error">&#91;hole..&#93;</span> <span class="error">&#91;0.5M data&#93;</span> <span class="error">&#91;hole&#93;</span>......<span class="error">&#91;0.7M data&#93;</span>......<span class="error">&#91;1M data&#93;</span></p> <p>So every time application want to access some data, the data is not only 4K but<br/> cosed to let's say 1M range, so if we could predict this kind of behavior and next time we hit page miss, we use last time range for prefetching.</p> LU-13669

improve mmap performances

Improvement Minor Resolved Fixed Wang Shilong Wang Shilong Fri, 12 Jun 2020 07:19:26 +0000 Sat, 4 Dec 2021 13:08:15 +0000 Mon, 28 Jun 2021 23:50:32 +0000 Lustre 2.14.0 0 7 <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/38916" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38916</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13669" title="improve mmap performances" class="issue-link" data-issue-key="LU-13669"><del>LU-13669</del></a> llite: try to improve mmap performance<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 84116d4451917e861671079ddf7ef57c722565c2</p> <p>Here is a benchmarking from Ihara:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>master [root@amd01 ~]# echo 3 &gt; /proc/sys/vm/drop_caches; fio --name=randread --directory=/ai400/fio --rw=randread --ioengine=mmap --bs=128K --numjobs=32 --filesize=200G --filename=randread --time_based --status-interval=10s --runtime=30s --allow_file_create=1 --group_reporting --disable_lat=1 --disable_clat=1 --disable_slat=1 --disk_util=0 --aux-path=/tmp --randrepeat=0 --unique_filename=0 --fallocate=0 [root@amd01 ~]# lctl get_param llite.*.stats llite.ai400-ffff8d855cf6a800.stats= snapshot_time 1594299686.416386775 secs.nsecs open 49 samples [usec] 0 1401 22183 close 49 samples [usec] 6 246 1117 mmap 32 samples [usec] 7 17 381 page_fault 1867904 samples [usec] 0 37698 956414944 getattr 34 samples [usec] 1 25 107 inode_permission 214 samples [usec] 0 22 407 master + patch [root@amd01 ~]# echo 3 &gt; /proc/sys/vm/drop_caches; fio --name=randread --directory=/ai400/fio --rw=randread --ioengine=mmap --bs=128K --numjobs=32 --filesize=200G --filename=randread --time_based --status-interval=10s --runtime=30s --allow_file_create=1 --group_reporting --disable_lat=1 --disable_clat=1 --disable_slat=1 --disk_util=0 --aux-path=/tmp --randrepeat=0 --unique_filename=0 --fallocate=0 [root@amd01 ~]# lctl get_param llite.*.stats llite.ai400-ffff8d862cbc6000.stats= snapshot_time 1594299943.816443708 secs.nsecs open 63 samples [usec] 1 4007 118809 close 63 samples [usec] 4 250 1410 mmap 32 samples [usec] 8 51 659 page_fault 17773312 samples [usec] 0 6543 933220595 getattr 34 samples [usec] 2 224 440 inode_permission 256 samples [usec] 0 276 1292 </pre> </div></div> <p>tested first 30sec, mostly no cached mmap<br/> 512usec vs 52usec<br/> 10x latency reduction.</p> <p>max latency 37698usec vs 6543usec<br/> max latency is also very small</p> <p>Wang Shilong (wshilong@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/39926" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39926</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13669" title="improve mmap performances" class="issue-link" data-issue-key="LU-13669"><del>LU-13669</del></a> readahead: increase ra window progressively<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: fc72ad754bc8fcc86ac96661329d9b074e3703e8</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38916/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38916/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13669" title="improve mmap performances" class="issue-link" data-issue-key="LU-13669"><del>LU-13669</del></a> llite: try to improve mmap performance<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0c5ad4b6df5bf35b291842fc6d42c2720246a026</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/41228/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41228/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13669" title="improve mmap performances" class="issue-link" data-issue-key="LU-13669"><del>LU-13669</del></a> llite: make readahead aware of hints<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 7542820698696ed5853ded30c9bf7fd5a78f0937</p> Related LUDOC-485 Development Rank 1|i012lr: Rank (Obsolete) 9223372036854775807