[LU-13375] lfs mirror extend does not take file layout into account

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.14.0, Lustre 2.12.5
• Fix Version/s: Lustre 2.14.0
• Type: Bug
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: Emoly Liu
• Resolution: Fixed
• Severity: 3

Description

Running "lfs mirror extend" on an existing file doesn't use the existing layout for the new file, nor the default filesystem layout on the root directory. Instead it seems to always create a plain layout using the filesystem default stripe_count and stripe_size. For example, the lcme_mirror_id: 1 components were initially created from the filesystem default layout, and the lcme_mirror_id: 2 component was created with "lfs mirror extend -N1" (default stripe_count=4):

/mnt/testfs/pfl0.2
  lcm_mirror_count:  2
  lcm_entry_count:   4
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   134217728
      lmm_stripe_count:  1
      lmm_stripe_size:   1048576
      lmm_pool:          ddn_ssd
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x12a:0x0] }
                       
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 134217728
    lcme_extent.e_end:   4294967296
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pool:          ddn_ssd
      lmm_objects:
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x109:0x0] }
      - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x12b:0x0] }
        
    lcme_mirror_id:      1
    lcme_flags:          0
    lcme_extent.e_start: 4294967296
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  -1
      lmm_stripe_size:   1048576
      lmm_pool:          ddn_hdd

    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  4
      lmm_stripe_size:   1048576
      lmm_pool:          ddn_hdd
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x12c:0x0] }
      - 1: { l_ost_idx: 3, l_fid: [0x100030000:0xc8:0x0] }
      - 2: { l_ost_idx: 2, l_fid: [0x100020000:0x88:0x0] }
      - 3: { l_ost_idx: 0, l_fid: [0x100000000:0x10a:0x0] }

At a minimum, "lfs mirror extend -N file" should use the stripe_count from the last initialized component, since it is 99% likely that the file will never be modified after it is first written and closed. It would be even better to copy the stripe_size as well, but not the pool.

Comments

[Emoly Liu]

This issue happens because the mirror layout doesn't inherit any stripe option from the file's component before running mirror_extend() in lfs_setstripe_internal(). I have made some changes and now the patch can work as follows:

[root@centos7-3 tests]# lfs setstripe -E 6M -S 1M -c 3 -E -1 -S 2m -c 2 /mnt/lustre/pfl
[root@centos7-3 tests]# lfs mirror extend -N1 /mnt/lustre/pfl 
[root@centos7-3 tests]# lfs getstripe /mnt/lustre/pfl 
/mnt/lustre/pfl
  lcm_layout_gen:    3
  lcm_mirror_count:  2
  lcm_entry_count:   3
    lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   6291456
      lmm_stripe_count:  3
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x3:0x0] }
      - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x2:0x0] }
      - 2: { l_ost_idx: 2, l_fid: [0x100020000:0x4:0x0] }

    lcme_id:             65538
    lcme_mirror_id:      1
    lcme_flags:          0
    lcme_extent.e_start: 6291456
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  2
      lmm_stripe_size:   2097152
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: -1

    lcme_id:             131073
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  2
      lmm_stripe_size:   2097152
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x4:0x0] }
      - 1: { l_ost_idx: 1, l_fid: [0x100010000:0x3:0x0] }

One thing I want to make sure is that the man page of "lfs mirror extend" says "If setstripe_options are not specified, then the stripe options inherited from the previous component will be used." So, we should inherit the stripe options from "the last initialized component" but not the last component, right? (The example above used the last component.)

[Andreas Dilger]

I think there are two reasonable behaviors for "lfs mirror extend -N1" on an existing file:

• copy the whole PFL layout of the existing file so that each mirror is the same. This is probably mostly what users "expect" when mirroring a file.
• copy the layout of the last initialized (used) component of the existing file. This is more efficient, from the point of view that it doesn't create extra objects for early components that are not very helpful, and it avoids e.g. DoM/flash components at the start of the file that would not work (DoM) or waste valuable resources (flash).

The chance of a file being modified after it is written and then mirrored is very low, so if the simpler/better solution is to use only the last initialized component then I'm happy with that for now.

[Gerrit Updater]

Emoly Liu ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38079
Subject: LU-13375 flr: inherit the correct layout by "mirror extend -N"
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 15c47ef03a0a1ff978efa96c9238a87c0b984580

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/38079/
Subject: LU-13375 flr: inherit the correct layout by "mirror extend -N"
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: f2a1c6e06f4dc5250cb72b1a6f4720f20486d502

[Peter Jones]

Landed for 2.14