Details

    • Technical task
    • Resolution: Fixed
    • Major
    • None
    • None
    • None
    • 9223372036854775807

    Description

      Current layout add replay is flawed, because layout add replies whole file LOVEA to client, and when replay, client sends the whole LOVEA back to server, server can't know which components were newly created by this layout add, and the replay would have to calls lod_sub_object_declare_create()/lod_sub_object_create() against all components, even for the pre-existing components.

      A proposal fix is to add one more 'buffer' parameter to dt_xattr_set(), and it's passed down to lod_xattr_set() -> lod_striping_create(), in lod_striping_create(), the 'buffer' will be filled with the LOVEA for the newly created components, at the end, the 'buffer' will be copied to reply buffer and returned to client, so that on replay, client will send back the partial LOVEA which being newly created, and server knows which components needs be re-processed.

      In a word, layout add creates only partial layout, so server should return the partial layout to client but not the whole LOVEA.

      Attachments

        Activity

          [LU-9185] flawed layout add replay

          I may miss something - but isn't newly added components deduced as (replay components)^(existing components)?

          I think you are right, that fix requires much less code changes than my original proposal. Anyway, since new layout add won't instantiate objects, now we need only to remove all that replay handling code.

          niu Niu Yawei (Inactive) added a comment - I may miss something - but isn't newly added components deduced as (replay components)^(existing components)? I think you are right, that fix requires much less code changes than my original proposal. Anyway, since new layout add won't instantiate objects, now we need only to remove all that replay handling code.

          I may miss something - but isn't newly added components deduced as (replay components)^(existing components)?

          jay Jinshan Xiong (Inactive) added a comment - I may miss something - but isn't newly added components deduced as (replay components)^(existing components)?

          Why couldn't it filter out the existing components at replay time?

          Because server can't tell which are pre-existing components, unless client tells more information.

          With dynamic layout introduced, layout add won't instantiate objects anymore (I think the semantic of layout add would be changed a little bit), so this problem can be fixed in a much easier way: remove all the code of instantiate objects, replay with same instantiated objects. Layout add now becomes a simple xattr set.

          niu Niu Yawei (Inactive) added a comment - Why couldn't it filter out the existing components at replay time? Because server can't tell which are pre-existing components, unless client tells more information. With dynamic layout introduced, layout add won't instantiate objects anymore (I think the semantic of layout add would be changed a little bit), so this problem can be fixed in a much easier way: remove all the code of instantiate objects, replay with same instantiated objects. Layout add now becomes a simple xattr set.

          Why couldn't it filter out the existing components at replay time?

          jay Jinshan Xiong (Inactive) added a comment - Why couldn't it filter out the existing components at replay time?

          Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/25824
          Subject: LU-9185 pfl: fix replay of component add
          Project: fs/lustre-release
          Branch: pfl
          Current Patch Set: 1
          Commit: 35d06135c9b7a127668f4705c2857a0beeb8a52e

          gerrit Gerrit Updater added a comment - Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/25824 Subject: LU-9185 pfl: fix replay of component add Project: fs/lustre-release Branch: pfl Current Patch Set: 1 Commit: 35d06135c9b7a127668f4705c2857a0beeb8a52e

          People

            bobijam Zhenyu Xu
            niu Niu Yawei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: