Details
-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
3
-
Orion
-
2992
Description
While running some single shared file IOR tests I noticed, with 'zpool iostat', that there were a lots write going on during the read phase. This was absolutely destroying performance and it wasn't at all clear to me why there should be any reads at all. IOR is just reading during this phase, no writes.
On a hunch I set 'sync_on_lock_cancel=never' and the writes stopped entirely and performance improved at least 6x. I haven't looked at the code yet but I think there might be a few issues here.
1) Why were so many lock revocations occurring during the read phase. I would have thought that each client would end up with a single concurrent read lock rather quickly. However, this behavior persistent for the 30 minutes it took to read all the data back.
2) When revoking a read lock, or even a write lock which isn't actually covering any dirty data, there's no reason to issue the sync. I suspect this was never noticed because under ldiskfs if there's no dirty data to sync performing the sync is basically free. Under ZFS however if you call sync you will force the current txg to be written out. That will result in at a minimum the vdev labels at the beginning and end of each disk to be written. Lots of syncs will result each disk head basically seeking from the being to end of the disk repeatedly. Very very bad.
Attachments
Issue Links
- is related to
-
-
- Closed
-