Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20671

Changelog Extension and Undo Semantics

XMLWordPrintable

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      The Changelog Extension Module augments existing Lustre per-MDT changelogs with undo payloads and SCN metadata. It does not replace the changelog; it extends record formats and mask handling so that setattr, setxattr, rename, unlink, rmdir, mkdir, create, link, truncate, and other reversible events carry enough information to undo the operation. The module also adds filtering support so that directory-level realms can extract relevant records without scanning unrelated MDT trees.

       

      The existing Lustre changelog records namespace and metadata events with target and parent FIDs, target name, and timestamp. The extended record adds SCN, directory realm hints, operation flags, old-value payloads, and cross-MDT dependency metadata.
      The old-value payload is critical. setattr()/setxattr() must store old values so the MDT can restore them during flashback. Therefore SATTR records store the previous mode, uid, gid, size-related attributes, and time fields relevant to the change; XATTR records store the previous xattr value or a tombstone indicating absence; TRUNC records store the previous file size; RENME records store both old and new parent FIDs and names; UNLNK/RMDIR records store enough naming and linkage metadata to decide whether the last link is being removed.

      The new-value payload is stored for audit and dry-run display, but undo primarily uses old values. For operations such as CREATE/MKDIR, the old value may be a null tombstone because the target did not previously exist. For UNLNK, the old value includes the namespace linkage information needed to recreate the directory entry and, if the last link is removed, to keep the inode reachable via trash or a hidden recovery namespace.

       

      Changelog Extension and Undo Semantics

      Lustre changelogs are per-MDD persistent logs of namespace change events and remain persistent until explicitly cleared. That persistence is a strong foundation for Flashback, but the current record set is primarily an audit and replication log; it is not by itself a complete undo log. The extension must therefore add undo-specific payloads while preserving existing record formats and APIs for non-flashback consumers.

      Namespace operations are extended as follows. CREATE records store the created name and parent linkage as current state and use a null old value because no prior entry existed. MKDIR records are symmetric: undo is rmdir only if the directory is empty or can be made empty through cascaded undo of descendant records. UNLNK records store the parent FID, name, target FID, and last-link flag. If the operation removed the last link, the old value includes the Trash reference or hidden recovery linkage needed to make the inode reachable again. RMDIR records store the directory’s name, parent, and a recursive descendant summary used to validate undo. RENME/RNMFM/RNMTO records store both source and destination namespace bindings so that rename can be inverted as a reverse rename or, if the source no longer exists, as a recreation in the source directory.

      Attribute operations require old values. SATTR records store a mask of changed attributes and the prior values for those attributes. XATTR records store the xattr name, previous value, and a flag indicating whether the xattr previously existed. TRUNC records store the previous file size. These extensions directly satisfy the requirement that setattr()/setxattr() store old values so the MDT can restore them.

      Undo semantics are intentionally conservative. A record is UNDOABLE only if all dependencies required to invert it are present and the target object’s current state is compatible with undo. For example, a CREATE can be undone by UNLNK only if the current directory still contains the name and the target FID matches the recorded target. A UNLNK can be undone only if the target FID is still reachable through trash or a recovery namespace, or if a prior CREATE/MKDIR in the same recovery window can recreate it without conflicting with surviving entries [2][3]. A SATTR can be undone only if no subsequent attribute change has overwritten the same attribute in a way that makes the old value unsafe to restore; in such cases, the engine records a warning and either skips the record or restores it as a best-effort historical value.

      The recovery engine must distinguish between idempotent and non-idempotent undo. Recreating a directory entry is idempotent if the name/FID binding is missing and the target FID is available. Restoring an old xattr value is idempotent if the xattr namespace still exists. Restoring a truncated file size is not always safe if the recovered data is no longer present; file mirror snapshots are therefore the preferred data-recovery path for TRUNC-related content recovery.

            qian_wc Qian Yingjin
            qian_wc Qian Yingjin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: