Niu
Could you please look into this one?
Thanks
Peter
It is normal not to see any modification while the write is being done on the OSTs but my question is: shouldn't MDT see the modification once the file is closed? Because if applications must do a SETATTR then I'm not sure that changelog can be used to detect any kind of event.
Without SOM (size on MDS), the MTIME is stored on OSTs, and MDT will not see the modification on file close neither (we only update ATIME on flie close). So, with current design, I think it's not a good idea to detect file data changes by changelog.
- May be it is time to finish SOM
- if changelog is not good to detect file data changes, how can we ?
Looking at the changelog code, I suspect a matter in the changelog mechanism
in the mdd_changelog_data_store() function (mdd_object.c, line 1362 in Lustre 2.1).
Indeed, all "time" events are lost as long as there still ANY other record related to the same file in the log.
I understand this avoids "*TIME" records storms in the log.
So I'd suggest replacing the current test by: don't log the xTIME event if there is still a *TIME record related to the same file in the log.
Thomas
May be it is time to finish SOM
Probably. 
if changelog is not good to detect file data changes, how can we ?
I'm not sure, probably the tool can check the file itself according the open/close record in changelog?
Niu,
- What about the proposal by Thomas Leibovici discarding TIME events only when another TIME event related with the file is present? Actually I'm not sure that these TIME events are being sent from the OSTs to the MDT. What's the behavior between the OSTs and MDT's ChangeLog when a client is writing on an OST?
- What do you mean by checking the open/close in the changelog? At least for lustre 2.1 there is no CLOSE event in the changelog when we close the file. Are we supposed to see it?
Hi, Diego
What about the proposal by Thomas Leibovici discarding TIME events only when another TIME event related with the file is present? Actually I'm not sure that these TIME events are being sent from the OSTs to the MDT. What's the behavior between the OSTs and MDT's ChangeLog when a client is writing on an OST?
As far as I know, OSTs don't send the TIME to MDT, and there isn't any changelogs on OSTs. When client writing to OST, only the objects on OST is written, no MDT involved.
What do you mean by checking the open/close in the changelog? At least for lustre 2.1 there is no CLOSE event in the changelog when we close the file. Are we supposed to see it?
Ah, you are right, CL_CLOSE (close event) isn't recorded in 2.1.
I'm not sure if it's worth improving the changelog to make it achieve your requirement, since I beleive that could have been resolved in HSM project. How about detecting the file changes in user level (for lustre 2.1)? for example, scanning the file changes by a background daemon?
CLOSE event could be optionally logged since lustre 2.2, landed as LU-579.
mdt_mfd_close seems to filter out mtime updates from the close for SOM.
Thomas, what does Robinhood do to detect mtime updates?
Note LU-579 added CL_CLOSE events, landed for Lustre 2.2. However, looking at current Master, it seems to have been removed.
Since this LU was opened, robinhood now relies on both MTIME and CLOSE events to detect file modifications (Lustre 2.2+ and robinhood 2.4.1).
Tracking CLOSE events fixes the issue Diego Moreno reported.
Nathan Rutman added a comment - 25/Mar/13 8:07 PM
> mdt_mfd_close seems to filter out mtime updates from the close for SOM.
wrong, SOM is not related here, it did not change the original logic. this filtering existed since the beginning.
Nathan Rutman added a comment - 25/Mar/13 8:56 PM
> Note LU-579 added CL_CLOSE events, landed for Lustre 2.2.
> However, looking at current Master, it seems to have been removed.
afaics, it is still in place, otherwise sanity test 160 would not work.
From Thomas:
I made a few tests on Lustre 2.1 and Lustre 2.4 (master) and I get the same behavior:
I seams that MTIME event is raised on open(O_TRUNC) but not for appending.
echo 1 > /mnt/lustre/dir/test
lfs changelog lustre ; lfs changelog_clear lustre cl1 0
55585 01CREAT 07:58:30.990467019 2013.03.27 0x0 t=[0x200000400:0x73a6:0x0] p=[0x200000400:0x73a5:0x0] test
echo 1 > /mnt/lustre/dir/test
lfs changelog lustre ; lfs changelog_clear lustre cl1 0
55586 17MTIME 07:58:36.765666783 2013.03.27 0x6 t=[0x200000400:0x73a6:0x0]
echo 1 >> /mnt/lustre/dir/test
lfs changelog lustre ; lfs changelog_clear lustre cl1 0
<nothing>
dd if=/dev/urandom of=/mnt/lustre/dir/test bs=1k count=50000
lfs changelog lustre ; lfs changelog_clear lustre cl1 0
55587 17MTIME 07:58:49.788543052 2013.03.27 0x6 t=[0x200000400:0x73a6:0x0]
dd if=/dev/urandom of=/mnt/lustre/dir/test bs=1k count=50000 conv=notrunc
lfs changelog lustre ; lfs changelog_clear lustre cl1 0
<nothing>
Notice: we get CL_CLOSE for the "nothing" cases in Lustre 2.2 and later.
So handling the CL_CLOSE record is a suitable solution for these versions.
The issue of Diego (also reported by James Silva, LLNL) is the robinhood DB do not reflect the actual file sizes
because it is never aware of the file change (with Lustre 2.1):
In Lustre 2.1, CL_CLOSE is not implemented and MTIME event is not triggered
in the modification cases I stated previously.
I do agree with Thomas that managing CLOSE events should be enough for Robinhood. Concerning the MTIME event, I think it's hard for users or applications reading changelog to rely on this event because, as Nathan already commented, you cannot see MTIME in case of append to an existing file.
It seems that TRUNC events are not raised at all. Should there be a separate ticket created for that?
Is this still true?
CLOSE event has been logged since 2.2, it can satisfy customer's requirement. Close this old 2.1 issue.