[#LU-543] Missing UNLINK record on overwritting rename

[LU-543] Missing UNLINK record on overwritting rename Created: 27/Jul/11 Updated: 08/Apr/12 Resolved: 14/Dec/11
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 2.0.0, Lustre 2.1.0
Fix Version/s:	Lustre 2.2.0, Lustre 2.1.2

Type:

Bug

Priority:

Minor

Reporter:

Thomas LEIBOVICI - CEA (Inactive)

Assignee:

Niu Yawei (Inactive)

Resolution:

Fixed

Votes:

Labels:

None

Severity:

Rank (Obsolete):

4806

Description

If there are 2 files A and B, and if B is removed this way: rename(A,B)
there is no information in MDT changelog to indicate that B was unlinked.
I think an UNLINK record about B would be appropriate in this case.

Comments

Comment by Peter Jones [ 28/Jul/11 ]

Niu

Could you please look into this one?

Thanks

Peter

Comment by Niu Yawei (Inactive) [ 28/Jul/11 ]

Hi, Thomas

I don't quite see why you want a UNLINK record for the rename, the rename changelog isn't enough for lustre_rsync to replay the changes? Could you explain it a little bit more? Thanks.

Comment by Thomas LEIBOVICI - CEA (Inactive) [ 29/Jul/11 ]

Hi Niu,

Initially, if you have 2 existing files dir1/A (fid1) and dir2/B (fid2),
rename(A,B) as 2 consequences:

deleting the object with fid2 (whose name was dir2/B)
renaming entry with fid1, from dir1/A to dir2/B

In the changelog, you only see that entry with fid1 is moved (RNMFROM and RNMTO records)
but there is absolutely no reference to the fact that entry with fid2 has been deleted from filesystem.
This can cause troubles when using changelogs for tracking UNLINK events to make some cleaning of file-related resources.
eg. in HSM, UNLINK are tracked to clean copies in storage backend after the file was removed from Lustre.

Comment by Niu Yawei (Inactive) [ 29/Jul/11 ]

Hi, Thomas

Thanks a lot for your clarification, as far as I know, the changelog is used for lustre_rsync to replicate a filesystem, and current rename changelog entry is sufficient for lustre_rsync to replicate the rename operation on target fs.(and the fid2 on target fs will be deleted when the rename is replicated).

For your use case (tracking UNLINK events to make some cleaning work), I think the proper solution might be fixing the tool itself. I'm not the changelog & HSM expert, maybe Andreas is the best person we refer to.

Hi, Andreas
Could you take a look at this problem? Thank you.

Comment by Niu Yawei (Inactive) [ 11/Oct/11 ]

Hi, Thomas

After discussing with Andreas, I realized that the unlink record has a CLF_UNLINK_LAST flag indicating whether it's the last nlink, which is different from the rename record.

My question is how do your application use the changelog? Will it check the CLF_UNLINK_LAST flag or it just replicate all unlink operations? You know, just simply adding an extra unlink record for rename will confuse the replcating applications (such as lustre_rsync), so I want to make certain how your application use the changelog first, then try to figure out a thorough solution.

Thanks
Niu

Comment by jacques-charles lafoucriere [ 12/Oct/11 ]

hello

In this case renaming a file does 2 things:

unlink old file (remove file)
rename file
so this is 2 independent events but done atomically. Today we have a silent unlink, we lose the information : old FID is no more used.
lustre_rsync works only if the replicate FS has the same behaviour (rename do unlink)
if lustre triggers 2 events : UNLINK + RENAME, lustre_rsync should also works, it will first remove the old one and after replace it with the new one

I think the discussion is more on : what should do Lustre than how applications are using changelog today

Comment by Niu Yawei (Inactive) [ 12/Oct/11 ]

so this is 2 independent events but done atomically

hmm, from the fs perspective, rename is one syscall, unlink + rename are two syscalls, they are different.

Today we have a silent unlink, we lose the information : old FID is no more used.

Could we scan the rename records as well? If there is any important information missed in rename record, I think we can fix it up.

I think the discussion is more on : what should do Lustre than how applications are using changelog today

I think the purpose of changelog is to record the performed fs operations on source fs and replicate them on target fs, if we record rename as UNLINK + RENAME, then an atomic rename syscall will be translated into two syscalls (unlink + rename) on target fs, that's why I hesitate to make this chanage.

Comment by Thomas LEIBOVICI - CEA (Inactive) [ 13/Oct/11 ]

If there is any important information missed in rename record, I think we can fix it up.

This is the point: when a rename results in removing an entry from the filesystem, the rename record should report the fid of the removed entry.

from the fs perspective, rename is one syscall, unlink + rename are two syscalls, they are different.

Even if it is an atomic operation for the filesystem, rename is already reported as 2 different records (RNMFRM and RNMTO).
I guess this is because current log record structure cannot contain all info about the rename operation (both source and target).
In case of an overwritting rename, this could be completed with a third event: UNLINK record with a special flag. If this flag is set, llreplicate can ignore the record, but if another tool is interested in deleted entries, the information is not lost.

Also, using UNLINK record sementics would make it possible to use existing flags of UNLINK record like CLF_UNLINK_LAST.

Comment by Niu Yawei (Inactive) [ 13/Oct/11 ]

This is the point: when a rename results in removing an entry from the filesystem, the rename record should report the fid of the removed entry.

Ok, if the target fid is necessary, I think we should put it in the second record of rename. Source fid is stored in the second record currently, looks it should be replaced with the target fid. Will provide a patch later. Thanks.

Comment by jacques-charles lafoucriere [ 13/Oct/11 ]

I think the purpose of changelog is to record the performed fs operations on source fs and replicate them on target fs, if we record rename as UNLINK + RENAME, then an atomic rename syscall will be translated into two syscalls (unlink + rename) on target fs, that's why I hesitate to make this chanage.

Changelog is a generic mechanism used to track what happen in a Lustre FS in an event based mode.
It can be used for many more things than FS replication (statistics, traces, audit, ....)

Comment by Niu Yawei (Inactive) [ 13/Oct/11 ]

Changelog is a generic mechanism used to track what happen in a Lustre FS in an event based mode.
It can be used for many more things than FS replication (statistics, traces, audit, ....)

Totally agree. Thanks, jacques-charles.

Comment by Niu Yawei (Inactive) [ 05/Nov/11 ]

http://review.whamcloud.com/1652