Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14719

"lfs migrate -m" creates broken agent inodes when target MDT full

Details

    • 3
    • 9223372036854775807

    Description

      mds02 kernel: LustreError: 8471:0:(osd_handler.c:3892:osd_create_local_agent_inode()) lfs1-MDT0001: [0x200026991:0x560d:0x0] add dot dotdot error: rc = -28
      mds02 kernel: LustreError: 8471:0:(out_lib.c:1190:out_tx_index_delete_undo()) lfs1-MDT0001-osd: Oops, can not rollback index_delete yet: rc = -524
      mds02 kernel: LustreError: 8471:0:(out_handler.c:915:out_tx_end()) lfs1-MDT0001-osd: undo for lustre/target/out_handler.c:454: rc = -524
      mds02 kernel: LustreError: 22380:0:(osd_handler.c:3892:osd_create_local_agent_inode()) lfs1-MDT0001: [0x200026991:0x5613:0x0] add dot dotdot error: rc = -28
      mds02 kernel: LustreError: 22380:0:(osd_handler.c:3892:osd_create_local_agent_inode()) Skipped 2 previous similar messages
      mds02 kernel: LustreError: 22380:0:(out_lib.c:1190:out_tx_index_delete_undo()) lfs1-MDT0001-osd: Oops, can not rollback index_delete yet: rc = -524
      mds02 kernel: LustreError: 22380:0:(out_lib.c:1190:out_tx_index_delete_undo()) Skipped 5 previous similar messages
      mds02 kernel: LustreError: 22380:0:(out_handler.c:915:out_tx_end()) lfs1-MDT0001-osd: undo for lustre/target/out_handler.c:454: rc = -524
      

      When e2fsck is run on the filesystem (LU-14710) it reports that the "." and ".." entries are corrupted, and can also report that the HTree index is corrupted:

      Directory entry for '.' in ... (1032783) is big.
      Split? no
      Second entry '3.3.0' (inode=538027 fid=[0x380020941:0x4c38:0x0]) in directory inode 1032783 should be '..'
      Fix? no
      

      This should be handled better by the MDS:

      • check that the target MDT has enough space before starting migration
      • if ENOSPC is returned before directory is migrated then clean up unused agent inodes

      Attachments

        Issue Links

          Activity

            [LU-14719] "lfs migrate -m" creates broken agent inodes when target MDT full
            gerrit Gerrit Updater added a comment - - edited

            "Andreas Dilger <adilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49217
            Subject: LU-14719 tests: find replay-single/111g breakage
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 2c47c196635aef7e911b1717332d38a321b7c1ac

            gerrit Gerrit Updater added a comment - - edited "Andreas Dilger <adilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49217 Subject: LU-14719 tests: find replay-single/111g breakage Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 2c47c196635aef7e911b1717332d38a321b7c1ac
            gerrit Gerrit Updater added a comment - - edited

            "Andreas Dilger <adilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49216
            Subject: LU-14719 tests: fix replay-single/111g version check
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 4065ec7afbed4f3aaf94a1c4a0026701267d2963

            gerrit Gerrit Updater added a comment - - edited "Andreas Dilger <adilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49216 Subject: LU-14719 tests: fix replay-single/111g version check Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 4065ec7afbed4f3aaf94a1c4a0026701267d2963

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/47039/
            Subject: LU-14719 lod: distributed transaction check space
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 6aee406c84b6b8fddf08b560acfcdf7c13c97e63

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/47039/ Subject: LU-14719 lod: distributed transaction check space Project: fs/lustre-release Branch: master Current Patch Set: Commit: 6aee406c84b6b8fddf08b560acfcdf7c13c97e63

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47128/
            Subject: LU-14719 osp: add inode watermark
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 336eb696299e1c9731bd1443f05e5d814314ed36

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47128/ Subject: LU-14719 osp: add inode watermark Project: fs/lustre-release Branch: master Current Patch Set: Commit: 336eb696299e1c9731bd1443f05e5d814314ed36

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47040/
            Subject: LU-14719 utils: dir migration stop on error
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 9ca348e8769d2c613082eeaeaf2775e22625e970

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/47040/ Subject: LU-14719 utils: dir migration stop on error Project: fs/lustre-release Branch: master Current Patch Set: Commit: 9ca348e8769d2c613082eeaeaf2775e22625e970

            "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47128
            Subject: LU-14719 osp: add inode watermark
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: e34b90897c1f47ec692b34c11b4d2482366fa90c

            gerrit Gerrit Updater added a comment - "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47128 Subject: LU-14719 osp: add inode watermark Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: e34b90897c1f47ec692b34c11b4d2482366fa90c

            "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47040
            Subject: LU-14719 utils: dir migration stop on error
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 5eff26f75e239fb9764cd9947d7f08deab001bbe

            gerrit Gerrit Updater added a comment - "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47040 Subject: LU-14719 utils: dir migration stop on error Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 5eff26f75e239fb9764cd9947d7f08deab001bbe

            "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47039
            Subject: LU-14719 lod: distributed transaction check space
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 633dec326173f72cb3b451ec43e08119a553e5df

            gerrit Gerrit Updater added a comment - "Lai Siyao <lai.siyao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/47039 Subject: LU-14719 lod: distributed transaction check space Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 633dec326173f72cb3b451ec43e08119a553e5df

            People

              laisiyao Lai Siyao
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: