Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-3647 HSM _not only_ small fixes and to do list goes here
  3. LU-3932

HSM can only handle ~4G requests at most

Details

    • Technical task
    • Resolution: Won't Fix
    • Major
    • None
    • Lustre 2.5.0
    • 10389

    Description

      In current implementation, a catalog llog can only have ~4G entries in its lifetime which means HSM can only handle ~4G requests at most.

      We have to improve catalog. It seems reasonable for me to roll catalog index back when it reaches maximum, assuming that it's impossible to have ~4G entries on the same time.

      Attachments

        Activity

          [LU-3932] HSM can only handle ~4G requests at most
          jhammond John Hammond made changes -
          Status Original: Resolved [ 5 ] New: Closed [ 6 ]
          jhammond John Hammond made changes -
          Resolution New: Won't Fix [ 2 ]
          Status Original: Reopened [ 4 ] New: Resolved [ 5 ]
          jay Jinshan Xiong (Inactive) made changes -
          Assignee Original: Jinshan Xiong [ jay ] New: John Hammond [ jhammond ]
          adilger Andreas Dilger added a comment -

          It is worth noting that this only affects the case of the MDS crashes or is shut down while the first and last ChangeLog catalog llogs are straddling the start of the catalog. That would return ChangeLog entries in a slightly wrong order. That is not a problem for HSM, but may be for lustre-rsync and similar.

          adilger Andreas Dilger added a comment - It is worth noting that this only affects the case of the MDS crashes or is shut down while the first and last ChangeLog catalog llogs are straddling the start of the catalog. That would return ChangeLog entries in a slightly wrong order. That is not a problem for HSM, but may be for lustre-rsync and similar.
          jay Jinshan Xiong (Inactive) made changes -
          Resolution Original: Won't Fix [ 2 ]
          Status Original: Closed [ 6 ] New: Reopened [ 4 ]
          jay Jinshan Xiong (Inactive) added a comment -

          Andreas says that the catalog should store the catalog index for the first (or last) index that is used. When processing the ChangeLog, it should start at the first (or one after the last) catalog index so that the individual llog files are processed in order.

          jay Jinshan Xiong (Inactive) added a comment - Andreas says that the catalog should store the catalog index for the first (or last) index that is used. When processing the ChangeLog, it should start at the first (or one after the last) catalog index so that the individual llog files are processed in order.
          jlevi Jodi Levi (Inactive) made changes -
          Resolution New: Won't Fix [ 2 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]
          jlevi Jodi Levi (Inactive) added a comment -

          After speaking to Jinshan, this will not be fixed.

          jlevi Jodi Levi (Inactive) added a comment - After speaking to Jinshan, this will not be fixed.
          jlevi Jodi Levi (Inactive) made changes -
          Fix Version/s Original: Lustre 2.7.0 [ 10631 ]
          adilger Andreas Dilger made changes -
          Labels New: hsm

          People

            jhammond John Hammond
            jay Jinshan Xiong (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: