Details

    • Technical task
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 9223372036854775807

    Description

      Use "lfs resync" command to calculate and fill parity code in parity component from reading the data of the corresponding data components.

      Attachments

        Activity

          [LU-12668] EC: resync parity components
          adilger Andreas Dilger added a comment - - edited

          As part of the resync and/or recovery, it is would be useful to be able to recover the size of the file based only on the data and parity stripes, even if the OST stripe holding the highest offset byte is lost. With FLR, the SOM xattr will cache the file size and blocks on the MDT inode, but it would add a layer of redundancy to also encode the file size into the EC stripes in some manner.

          If the EC component is sized exactly with the number of parity bytes written (not rounded to the next stripe boundary), then it would be possible to use this to compute the file size within M bytes of the actual size (rounded up). It would be further possible to mostly determine the file size by subtracting up to M-1 zero bytes in the last stripe, but this is not 100% robust if the file actually has zeroes at the end of the file.  Having multiple parity stripes doesn't really help, since the number of parity bytes written to each stripe would be the same.

          adilger Andreas Dilger added a comment - - edited As part of the resync and/or recovery, it is would be useful to be able to recover the size of the file based only on the data and parity stripes, even if the OST stripe holding the highest offset byte is lost. With FLR, the SOM xattr will cache the file size and blocks on the MDT inode, but it would add a layer of redundancy to also encode the file size into the EC stripes in some manner. If the EC component is sized exactly with the number of parity bytes written (not rounded to the next stripe boundary), then it would be possible to use this to compute the file size within M bytes of the actual size (rounded up). It would be further possible to mostly determine the file size by subtracting up to M-1 zero bytes in the last stripe, but this is not 100% robust if the file actually has zeroes at the end of the file.  Having multiple parity stripes doesn't really help, since the number of parity bytes written to each stripe would be the same.

          "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51492
          Subject: LU-12668 ec: add API to resync EC file
          Project: fs/lustre-release
          Branch: master
          Current Patch Set: 1
          Commit: 51acd2b7617d90dd2c7a9586255305f5ca7f773b

          gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51492 Subject: LU-12668 ec: add API to resync EC file Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 51acd2b7617d90dd2c7a9586255305f5ca7f773b

          "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51491
          Subject: LU-12668 ec: resync parity component
          Project: fs/lustre-release
          Branch: master
          Current Patch Set: 1
          Commit: d87772aec56e6b9fd07c6075c769a9ae20a2397c

          gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51491 Subject: LU-12668 ec: resync parity component Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: d87772aec56e6b9fd07c6075c769a9ae20a2397c
          bobijam Zhenyu Xu added a comment -

          https://review.whamcloud.com/#/c/38065/

          LU-12668 ec: user tool to resync EC file

          lfs resync is used to resync parity code of EC file.

          Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
          Change-Id: I89eec2dd86c9cbe93ca45d0cb458dd1c02825c7e

          bobijam Zhenyu Xu added a comment - https://review.whamcloud.com/#/c/38065/ LU-12668 ec: user tool to resync EC file lfs resync is used to resync parity code of EC file. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I89eec2dd86c9cbe93ca45d0cb458dd1c02825c7e
          bobijam Zhenyu Xu added a comment -

          https://review.whamcloud.com/#/c/38064/

          LU-12668 ec: resync parity component

          Calculate parity code from recovery group of data and write
          into parity objects.

          Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
          Change-Id: I3b4eaafdb29e72f6bc07b2d0ef2951d2d28bf9b2

          bobijam Zhenyu Xu added a comment - https://review.whamcloud.com/#/c/38064/ LU-12668 ec: resync parity component Calculate parity code from recovery group of data and write into parity objects. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I3b4eaafdb29e72f6bc07b2d0ef2951d2d28bf9b2

          People

            bobijam Zhenyu Xu
            bobijam Zhenyu Xu
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: