[LU-4388] fsync on client does not cause OST_SYNCs to be issued

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.6.0
• Affects Version/s: Lustre 2.5.0, Lustre 2.6.0
• Labels: MB
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Nathaniel Clark

Description

Issuing an fsync() only causes MDS_SYNC's to be issued.

Testing:
dd with conv=fsync vs. dd with oflag=sync

Comments

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14840
Subject: LU-4388 llite: issue OST_SYNC for fsync()
Project: fs/lustre-release
Branch: b2_5
Current Patch Set: 1
Commit: 3951a7c7aed155c206043838e4e800c94ac1c692

[Hans Henrik Happe]

Given the severity (data loss, inconsistent databases, etc..), shouldn't it be included in the maintenance release?

[Zhenyu Xu]

2.6.0 has the patches and 2.5.0 does not.

[Hans Henrik Happe]

Should this have been fixed after 2.5.0 or is it only 2.6.0? I see the same problem with b2_5 on zfs (fdatasync okay and slow, fsync not touching the disk). I think it is important that fsync works as expected.

[Jodi Levi (Inactive)]

Patch landed to Master.

[Andreas Dilger]

The patch http://review.whamcloud.com/8626 has not landed yet.

[Jodi Levi (Inactive)]

Patch has landed to Master. Please reopen if more work is needed in this ticket.

[Zhenyu Xu]

patch tracking at http://review.whamcloud.com/8684

[Jinshan Xiong (Inactive)]

I see. So this is a misunderstanding of @datasync parameter. This bug may exist in all series of lustre client implementation. Thanks for explanation.

[Andreas Dilger]

Looking at the comments around vfs_fsync() and vfs_fsync_range()
http://lxr.free-electrons.com/source/fs/sync.c#L178

 * vfs_fsync - perform a fsync or fdatasync on a file
 * @file:               file to sync
 * @datasync:           only perform a fdatasync operation
 *
 * Write back data and metadata for @file to disk.  If @datasync is
 * set only metadata needed to access modified file data is written.

It does appear that datasync=1 means sync data and enough metadata to get the data back (e.g. i_size and block allocations) while datasync=0 means sync everything. See also http://man7.org/linux/man-pages/man2/fsync.2.html

fdatasync() is similar to fsync(), but does not flush modified
metadata unless that metadata is needed in order to allow a
subsequent data retrieval to be correctly handled. For example,
changes to st_atime or st_mtime (respectively, time of last access
and time of last modification; see stat(2)) do not require flushing
because they are not necessary for a subsequent data read to be
handled correctly. On the other hand, a change to the file size
(st_size, as made by say ftruncate(2)), would require a metadata
flush.

So if datasync=0 then we should always be sending OST_SYNC RPCs, but if datasync=1 then that is only needed if the size is changed or if blocks were allocated. That is a difficult/impossible decision for a client to make, so the OST_SYNC RPC should be sent to the OST with a DATASYNC flag and let the OSD decide. For older branches it probably makes sense to always send OST_SYNC.

[Zhenyu Xu]

A little bit confused about the datasync parameter of fsync(), is it datasync=1 means only sync data while datasync=0 means sync data+metadata?