[LU-10810] SEEK_HOLE and SEEK_DATA support for lseek

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Labels: medium
• Assignee: Mikhail Pershin
• Reporter: Lokesh N J (Inactive)

Description

lseek with SEEK_HOLE and SEEK_DATA are really helpful  and easy to use tools for any usersapce applications like copy and backup. Currently lustre has min support implementation as per the man page which is not that useful however lustre does support fiemap ioctl which can be used for mapping data in the file.
As we already have support for fiemap I guess with some implementation lustre can support SEEK_HOLE and SEEK_DATA flags. I guess having this support will be helpful to deal with sparse files. Any feedback about the implementation will be really helpful.

Issue Links

• duplicates: LU-13397 lfs migrate/mirror extend/resync does not preserve sparse file
• is related to: LU-14143 SEEK_HOLE returns -ENXIO if file ends at components boundary
• is related to: LU-11621 Add copy_file_range() API and use it for lfs migrate and mirror resync
• is related to: LU-14217 better SEEK_HOLE/DATA support in OSD-ZFS
• is related to: LU-1941 ZFS FIEMAP support
• is related to: LU-3606 Implement fallocate() support for ldiskfs Lustre
• is related to: LU-13155 "lfs migrate" and "lfs mirror" should fallocate() the target file

Comments

[Gerrit Updater]

"Etienne AUJAMES <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50325
Subject: LU-10810 osd: implement lseek method in OSD
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: b21bc7e262aff42aa3fd90966a791de4ad515dd0

[Mikhail Pershin]

Andreas, I've check these reports and in fact only couple of them are ldiskfs, in these cases there were old servers used, with just no lseek support. All ZFS tests are expected to be skipped. 

In fact, I was testing recently ZFS 8.5 with exported dmu_offset_mext() to check how it behave and have found that there are problems.
ZFS SEEK_HOLE/DATA support is unreliable because it doesn't work if there are dirty data, that cause all test to fail because we don't do full data sync prior each lseek. I was trying to sync data in tests but with no big success. Moreover I don't think test failures are problems, after all we could sync everything just to make test pass but that would not mean ZFS lseek is reliable until ZFS itself will handle dirty data.

For now it can report HOLE at offset 1000 (correct) and immediately SEEK_DATA(1000) returns also 1000 (incorrectly) just because there are dirty data and it switches to generic approach (all file is data with hole at the end). I am afraid we cannot rely on ZFS until its option zfs_dmu_offset_next_sync is set, which in turn would cause zfs slowdown when SEEK_DATA is performed on it

So basically ZFS server should report LSEEK support only if zfs_dmu_offset_next_sync is set AND dmu_offset_next is exported. And that should be done now I think, before we will start using ZFS 8.5 and find out that lseek is reported as supported but doesn't work as expected

[Andreas Dilger]

Mike, I noticed some test runs are reporting no SEEK_HOLE support for test_430a,430b,430c, even though client and server are running on the current tip of master (v2_13_56-149-ge5c8f6670f):
https://testing.whamcloud.com/search?horizon=518400&status%5B%5D=SKIP&test_set_script_id=f9516376-32bc-11e0-aaee-52540025f9ae&sub_test_script_id=4154d1b3-ab47-47e4-9ecd-ea85afa35353&source=sub_tests#redirect

It doesn't look like all test runs are being skipped, but I didn't think this feature is specific to ZFS or ldiskfs, so I'm not sure why the subtests are being skipped.

[Peter Jones]

Landed for 2.14

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/40502/
Subject: LU-10810 test: test lseek support in tools
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d633172a38519aba2585c2a1fdcdd821cb19010c

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39708/
Subject: LU-10810 clio: SEEK_HOLE/SEEK_DATA on client side
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cda353e6efae5013a26aedbe49d8aa6fb8fe456e

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39707/
Subject: LU-10810 ptlrpc: introduce OST_SEEK RPC
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6d5fe29066af5f8e40055fd89b285853c363e947

[Gerrit Updater]

Oleg Drokin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/40570
Subject: LU-10810 tests: Fix calls to lseek_test_425
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 14cc826f465679531e352fd3852b115d369f6773

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/40502
Subject: LU-10810 test: check seek support in tools
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 9dc5e733c58bcdd12a987a4b06eb2f4845597dfb

[John Hammond]

coreutils cp added SEEK_HOLE/DATA support earlier this year. See https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=a6eaee501f6ec0c152abe88640203a64c390993e

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/39894
Subject: LU-10810 test: extended debug for lseek test
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f397a37df3b2f8f636686edc37b1720f58aa088a