Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-21127

HSM dirty change not reported in changelog when a client is evicted while holding a file open for write

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • Lustre 2.15.8
    • Client: lustre-client 2.15.6 on Linux 6.18 (x86_64). Servers: Lustre 2.15, ZFS-backed MDT, HSM with changelog-driven copytool/archiver.
    • 3
    • 9223372036854775807

      When a client is evicted by the MDT while it holds an archived file open for write, the final modifications made by that client are never reported to changelog consumers (e.g. a copytool policy engine), so the archive copy stays stale.
        
        Sequence:
        1. Client opens file F for write, writes data. F is archived while open (hsm_state: exists archived).
        2. MDT evicts the client (e.g. lock callback timeout, or lctl set_param mdt.*.evict_client=nid:<client>). mdt_export_cleanup() -> mdt_ctxt_add_dirty_flag() sets HS_DIRTY on F (LU-3677), but the session ucred is zero-initialized, so uc_enable_audit == 0 and mdd_changelog_enabled() suppresses the CL_HSM record. F becomes "exists dirty archived" with no changelog record.
        3. The client reconnects; the application keeps writing (data goes to OSTs) and then closes F.
        4. The close RPC carries MDS_DATA_MODIFIED with the pre-eviction open handle. mdt_close_internal() returns -ESTALE ("no handle for file close"). mdc_close() resets -ESTALE to 0 because the open was committed (bug 3633 logic). ll_prepare_close() already cleared LLIF_DATA_MODIFIED, so the modification is lost. Even if the MDT had processed the close, HS_DIRTY is already set, so no CL_HSM record would be emitted.
           
        Result: no CL_HSM (dirty) record is ever written for the post-archive writes;  a changelog-driven archiver never re-archives F. Archive copy size < file size.
        
        Reproduced on 2.15.6 client / 2.15 server:
          before: archive 8192 B, file 70656 B after close, state 0xb, no changelog after eviction
          with proposed patch: "mdc close failed: rc = -116" logged, CL_HSM dirty emitted on
          close, re-archived 59392 B == file size, state 0x9
      
        Proposed fix (client side):
        - mdc_close(): do not reset -ESTALE to 0 when the close carries MDS_DATA_MODIFIED.
        - ll_close_inode_openhandle(): on close failure, restore LLIF_DATA_MODIFIED.
        - ll_md_close(): on last write close, if LLIF_DATA_MODIFIED is still set, clear and
          then set HS_DIRTY via ll_hsm_state_set() so a CL_HSM record is emitted.
      
        Alternative considered: set uc_enable_audit = 1 in mdt_ctxt_add_dirty_flag() so the
        eviction-time dirty is logged. Rejected as the only fix: it triggers a re-archive while
        the client may still be writing, and still misses writes made after the eviction.
      
        Patch and sanity-hsm test to follow.
      
      

            gaurani Anirudh Gaur
            gaurani Anirudh Gaur
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: