[LU-12687] Fast ENOSPC on direct I/O

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0, Lustre 2.12.6
• Severity: 3
• Assignee: Vladimir Saveliev
• Reporter: Vladimir Saveliev

Description

On direct I/O a client gets substantial amount of grants without consuming them so that direct I/O writes face with ENOSPC long before disk space is over.
The below example shows how dd oflag=direct fails to write 400kb to ~70mb ost.

[root@sl75master tests]# OSTSIZE=100000 sh llmount.sh
...
Updated after 6s: wanted 'procname_uid' got 'procname_uid'
disable quota as required
[root@sl75master tests]# lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID       122.4M        1.9M      109.5M   2% /mnt/lustre[MDT:0]
lustre-OST0000_UUID        69.4M        1.2M       61.4M   2% /mnt/lustre[OST:0]
lustre-OST0001_UUID        69.4M        1.2M       61.4M   2% /mnt/lustre[OST:1]

filesystem_summary:       138.9M        2.5M      122.7M   2% /mnt/lustre

[root@sl75master tests]# dd if=/dev/zero of=/mnt/lustre/file bs=4k count=100 oflag=direct
dd: error writing ‘/mnt/lustre/file’: No space left on device
54+0 records in
53+0 records out
217088 bytes (217 kB) copied, 0.138233 s, 1.6 MB/s
[root@sl75master tests]# 

Issue Links

• is related to: LU-12757 sanity-lfsck test 36a fails with '(N) Fail to resync /mnt/lustre/d36a.sanity-lfsck/f2'
• is related to: LU-13766 tgt_grant_check() lsrza-OST000a: cli dfdf1aff-07d9-53b3-5632-c18a78027eb2 claims 1703936 GRANT, real grant 0
• is related to: LU-13847 sanity test_64f: grants mismatch
• is related to: LU-4198 Improve IO performance when using DIRECT IO using libaio
• is related to: LU-4664 sync write should consume grant on client