[LU-13805] i/o path: Unaligned direct i/o

• Type: Improvement
• Resolution: Unresolved
• Priority: Major
• Assignee: Patrick Farrell (Inactive)
• Reporter: Patrick Farrell

Description

With the DIO performance improvements in LU-13798 and LU-13799, it becomes interesting to do larger buffered i/o (BIO) using the DIO path, as in LU-13802.

LU-13802 covers the code for switching between the BIO and DIO paths, allowing BIO which meets the requirements for DIO to use the BIO path when appropriate.

The problem is, the requirements for DIO are sometimes hard to meet.  i/o must be both page & size aligned.  This ticket is about how to do unaligned DIO, in order to let us do any BIO through the DIO path.

 

This cannot be done with the existing Lustre i/o path.  There are a few minor issues, but the central problem is that if an i/o is unaligned, we no longer have a 1-to-1 mapping between a page on the client and a page in the file/on the server.  (Buffered i/o creates this 1-to-1 mapping by copying in to an aligned buffer.)  This 1-to-1 mapping could possibly be removed, but it would require a significant rework of the Lustre i/o path to make this possible.

So, one option is creating a new DIO path which permits unaligned i/o from userspace all the way to disk.

The other option comes from the following observation:
When doing buffered i/o, about 20% of the time is spent in allocating the buffer and doing memcopy() in to that buffer.  Of the remaining 80%, something like 70% is page tracking of various kinds.
Because each page in the page cache can be accessed from multiple threads, including being flushed at any time from various threads (memory pressure va kswapd, lock cancellation, writeout...), it has to be on various lists & have references on (effectively) the file it is part of, etc.

This work, not allocation and memcopy, is where most of the time goes.

So if we implement a simple buffering scheme - allocate an aligned buffer, then copy data to (or from) that buffer - and then do a normal DIO write(/read) from(/to) that buffer, this can be hugely faster than buffered i/o.

If we use the normal DIO path (ie, sync write, and do not keep pages after read), we keep this as a buffer, and not a cache, so we can keep the DIO path lockless.

Also, if we implement this correctly, we have a number of excellent options for speeding this up:

1. Move allocation (if we're not pre-allocated) and memcopy from the user thread to the ptlrpcd threads handling RPC submission - This allows us to do these operations in parallel, which should dramatically improve speed.
2. Use pre-allocated buffers.
3. Potentially, since we control the entire copying path, we could enable the FPU to use vectorized memcopies.  (Various aspects of the buffered i/o path in the kernel mean the FPU has to be turned on and off for each page.  The cost of this outweighs the benefit of vectorized memcopy.)

Issue Links

• is blocked by: LU-17597 interop: master/2.15/2.14/2.12 sanity test_56x: migrate failed rc = 22
• is related to: LU-13802 New i/o path: Buffered i/o as DIO
• is related to: LU-18006 sanity test_119f: crash in ll_dio_user_copy
• is related to: LU-17450 sanity: interop test failures with master+2.15
• is related to: LU-17525 Unaligned DIO interop with different page sizes fails
• is related to: LU-18284 interop sanity test_119e test_119f: UDIO files differ, bsize 1048575, 2.12 servers crash
• is related to: LU-13799 DIO/AIO efficiency improvements
• is related to: LU-17156 sanityn test_16j: timeout
• is related to: LU-17215 sanity/398q should use $tfile
• is related to: LU-12550 automatic lockahead
• is related to: LU-17433 async hybrid writes
• is related to: LU-18880 Hybrid IO: Add ladvise to disable on single file
• is related to: LU-13798 Improve direct i/o performance with multiple stripes: Submit all stripes of a DIO and then wait
• is related to: LU-17422 unaligned DIO: use page pools
• is related to: LU-16964 I/O Path: Auto switch from BIO to DIO
• is related to: LU-17194 parallelize DIO submit
• is related to: LU-247 Lustre client slow performance on BG/P IONs: unaligned DIRECT_IO
• is related to: LU-13814 DIO performance: cl_page struct removal for DIO path

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51126/
Subject: LU-13805 llite: Implement unaligned DIO connect flag
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0e6e60b1233b08952c338b2c4f121ef749a99f8b

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51125/
Subject: LU-13805 llite: add flag to disable unaligned DIO
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 33ed79ba61a3275519c897557407619d576a9dc2

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52668
Subject: LU-13805 llite: udio and encryption
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5ea5a9c13a81d0168137230b74b4450ce66de54a

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/45616/
Subject: LU-13805 clio: bounce buffer for unaligned DIO
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 7194eb6431d2ef7245ef3b13394b60e220145187

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49993/
Subject: LU-13805 llite: unaligned direct_rw_pages
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 40bb067da1f40714af4f77d24e840fdb6285b08e

[Gerrit Updater]

"Patrick Farrell <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52391
Subject: LU-13805 llite: fail unaligned DIO for RDMA pages
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4f72f0b4b5277eca792bad33b0e6df4df9d32ce1

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49947/
Subject: LU-13805 llite: add mm to dio struct
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 7df8bd69fbe59afba0a43fe19e7a5b1d2c3fd115

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49913/
Subject: LU-13805 clio: Add csi_complete
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a2b722214a99c8c65fb764a67770b47a1195691a

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/51075/
Subject: LU-13805 obd: Reserve unaligned DIO connect flag
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4c96cbf89dba5e4bf8ddf98a18b72142c22a4289

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49940/
Subject: LU-13805 llite: Add copy of iovec to sub-dio
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d011e65a8936bc105e19b3fad64bfdece455de61

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49991/
Subject: LU-13805 clio: Add write to sdio
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 89f489ef9a18b5fea0571e2606dfdf064777626b