[LU-15829] DIO small reads fails - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Major
Fix Version/s: Lustre 2.16.0
Affects Version/s: Lustre 2.15.0
Labels:
None

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

On Lustre there is one strange quirk:

DIO writing now works, assuming you are writing from an aligned buffer, you are not restricted to the 512 byte limit.

However DIO reading of a single byte is broken, reading for more than 1 byte is okay.

Attached test source: dio_test.c

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -w -d -b 1 /mnt/lustre/tmp/dio_w3.dat
/mnt/lustre/tmp/dio_w3.dat - DIRECT WRITE 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)

If the block you are reading 1 byte from is in the PAGE_CACHE then the 1 byte read will succeed.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33  '
/mnt/lustre/tmp/dio_w3.dat - BUFFER READ 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33  '
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/1 bytes
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)

So what happens when doing a DIO read of 2 or more bytes and the block is NOT in the PAGE_CACHE?

[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 2 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33  '
read: 0 (EOF)
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/2 bytes
[shaun@alma8 dio]$

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

dio_test.c
3 kB
06/May/22 7:57 AM

Activity

[LU-15829] DIO small reads fails

Peter Jones made changes - 12/Sep/22 2:58 PM

Fix Version/s		New: Lustre 2.16.0 [ 15190 ]
Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Resolved [ 5 ]

Peter Jones added a comment - 12/Sep/22 2:58 PM

Landed for 2.16

Peter Jones added a comment - 12/Sep/22 2:58 PM Landed for 2.16

Gerrit Updater added a comment - 19/May/22 5:37 PM

"Alexey Lyashkov <alexey.lyashkov@hpe.com>" uploaded a new patch: https://review.whamcloud.com/47395
Subject: ~~LU-15829~~ llite: don't use a kms if it invalid.
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b96430385b93e30f0bea1e147e0b17f28577bdcd

Gerrit Updater added a comment - 19/May/22 5:37 PM "Alexey Lyashkov <alexey.lyashkov@hpe.com>" uploaded a new patch: https://review.whamcloud.com/47395 Subject: LU-15829 llite: don't use a kms if it invalid. Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: b96430385b93e30f0bea1e147e0b17f28577bdcd

Andreas Dilger made changes - 06/May/22 5:30 PM

Affects Version/s

New: Lustre 2.15.0 [ 14791 ]

Shaun Tancheff made changes - 06/May/22 8:00 AM

Description

Original: On Lustre there is one strange quirk: [^dio_test.c]

DIO writing now works, assuming you are writing from an aligned buffer, you are not restricted to the 512 byte limit.

However DIO reading of a single byte is broken, reading for more than 1 byte is okay.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -w -d -b 1 /mnt/lustre/tmp/dio_w3.dat
/mnt/lustre/tmp/dio_w3.dat - DIRECT WRITE 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
If the block you are reading 1 byte from is in the PAGE_CACHE then the 1 byte read will succeed.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - BUFFER READ 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/1 bytes
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
So what happens when doing a DIO read of 2 or more bytes and the block is NOT in the PAGE_CACHE?

[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 2 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
read: 0 (EOF)
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/2 bytes
[shaun@alma8 dio]$

New: On Lustre there is one strange quirk:

DIO writing now works, assuming you are writing from an aligned buffer, you are not restricted to the 512 byte limit.

However DIO reading of a single byte is broken, reading for more than 1 byte is okay.

Attached test source: [^dio_test.c]

{noformat}
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -w -d -b 1 /mnt/lustre/tmp/dio_w3.dat
/mnt/lustre/tmp/dio_w3.dat - DIRECT WRITE 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
{noformat}

If the block you are reading 1 byte from is in the PAGE_CACHE then the 1 byte read will succeed.

{noformat}
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - BUFFER READ 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/1 bytes
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
{noformat}

So what happens when doing a DIO read of 2 or more bytes and the block is NOT in the PAGE_CACHE?

{noformat}
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 2 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
read: 0 (EOF)
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/2 bytes
[shaun@alma8 dio]$
{noformat}

Shaun Tancheff made changes - 06/May/22 7:57 AM

Description

Original: On Lustre there is one strange quirk:

DIO writing now works, assuming you are writing from an aligned buffer, you are not restricted to the 512 byte limit.

However DIO reading of a single byte is broken, reading for more than 1 byte is okay.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -w -d -b 1 /mnt/lustre/tmp/dio_w3.dat
/mnt/lustre/tmp/dio_w3.dat - DIRECT WRITE 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
If the block you are reading 1 byte from is in the PAGE_CACHE then the 1 byte read will succeed.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - BUFFER READ 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/1 bytes
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
So what happens when doing a DIO read of 2 or more bytes and the block is NOT in the PAGE_CACHE?

[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 2 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
read: 0 (EOF)
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/2 bytes
[shaun@alma8 dio]$

New: On Lustre there is one strange quirk: [^dio_test.c]

DIO writing now works, assuming you are writing from an aligned buffer, you are not restricted to the 512 byte limit.

However DIO reading of a single byte is broken, reading for more than 1 byte is okay.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -w -d -b 1 /mnt/lustre/tmp/dio_w3.dat
/mnt/lustre/tmp/dio_w3.dat - DIRECT WRITE 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
If the block you are reading 1 byte from is in the PAGE_CACHE then the 1 byte read will succeed.

[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - BUFFER READ 1/1 bytes
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/1 bytes
[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
So what happens when doing a DIO read of 2 or more bytes and the block is NOT in the PAGE_CACHE?

[shaun@alma8 dio]$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 1 /mnt/lustre/tmp/dio_w3.dat
read: 0 (EOF)
[shaun@alma8 dio]$ /home/shaun/tools/dio/dio_test -d -b 2 /mnt/lustre/tmp/dio_w3.dat
read: 1, x'33 '
read: 0 (EOF)
/mnt/lustre/tmp/dio_w3.dat - DIRECT READ 1/2 bytes
[shaun@alma8 dio]$

Shaun Tancheff made changes - 06/May/22 7:57 AM

Attachment

New: dio_test.c [ 43553 ]

Gerrit Updater added a comment - 06/May/22 5:13 AM

"Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/47225
Subject: ~~LU-15829~~ clio: Force buffered I/O on small reads
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 57f500ecaada808af3a2fb5f5dd5d895d5a6aa3b

Gerrit Updater added a comment - 06/May/22 5:13 AM "Shaun Tancheff <shaun.tancheff@hpe.com>" uploaded a new patch: https://review.whamcloud.com/47225 Subject: LU-15829 clio: Force buffered I/O on small reads Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 57f500ecaada808af3a2fb5f5dd5d895d5a6aa3b

Shaun Tancheff created issue - 06/May/22 5:12 AM

People

Assignee:: Shaun Tancheff

Reporter:: Shaun Tancheff

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 06/May/22 5:12 AM

Updated:: 31/Oct/22 4:09 AM

Resolved:: 12/Sep/22 2:58 PM