[LU-14897] "lfs setstripe/migrate" should check if pool exists for PFL components - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Unresolved
Priority: Minor
Fix Version/s: None
Affects Version/s: None
Labels:
None

Severity:
3
Rank (Obsolete):
9223372036854775807

Description

The following scenario is possible:

[root@client client]# lfs setstripe -E100M -c1 -p flash -E-1  -p not_a_pool test 
[root@client client]# lfs getstripe test                                         
test                                                                             
  lcm_layout_gen:    2                                                           
  lcm_mirror_count:  1                                                           
  lcm_entry_count:   2                                                           
    lcme_id:             1                                                       
    lcme_mirror_id:      0                                                       
    lcme_flags:          init                                                    
    lcme_extent.e_start: 0                                                       
    lcme_extent.e_end:   104857600                                               
      lmm_stripe_count:  1                                                       
      lmm_stripe_size:   1048576                                                 
      lmm_pattern:       raid0                                                   
      lmm_layout_gen:    0                                                       
      lmm_stripe_offset: 1                                                       
      lmm_pool:          flash                                                   
      lmm_objects:                                                               
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x83:0x0] }                       
                                                                                 
    lcme_id:             2                                                       
    lcme_mirror_id:      0                                                       
    lcme_flags:          0                                                       
    lcme_extent.e_start: 104857600                                               
    lcme_extent.e_end:   EOF                                                     
      lmm_stripe_count:  1                                                       
      lmm_stripe_size:   1048576                                                 
      lmm_pattern:       raid0                                                   
      lmm_layout_gen:    0                                                       
      lmm_stripe_offset: -1                                                      
      lmm_pool:          not_a_pool

"lfs" should check if the specified pool "not_a_pool" exists before generate and write the layout (trusted.lov) on the filesystem.

The issue does not exist with plain layout:

[root@client client]# lfs setstripe -c-1 -p not_a_pool test                    
lfs setstripe: pool 'lustrefs.not_a_pool' does not exist: Invalid argument (22)

Attachments

Issue Links

is related to

LU-11264 llapi_* routines demonstrate poor performance

Open

Activity

[LU-14897] "lfs setstripe/migrate" should check if pool exists for PFL components

Andreas Dilger made changes - 30/Jul/21 6:10 PM

Link

New: This issue is related to LU-11264 [ LU-11264 ]

Andreas Dilger added a comment - 30/Jul/21 6:10 PM

Note that there have been complains like LU-11264 that the setstripe code is already too heavy with checks. I pushed patch https://review.whamcloud.com/35028 "LU-11264 llapi: avoid pool verification overhead" to reduce the overhead of checking the pools for each file, but didn't have time to finish it off.

It would be best to start with updating that patch (maybe split into 2-3 smaller patches to separate the different changes) to reduce the overhead of checking pool validity before adding even more checks to setstripe.

Note that Lustre itself doesn't care if the pool is wrong or missing. This can still happen even if setstripe validates the input, if the pool is deleted afterward. It will allocate as if no pool is specified in that case.

Andreas Dilger added a comment - 30/Jul/21 6:10 PM Note that there have been complains like LU-11264 that the setstripe code is already too heavy with checks. I pushed patch https://review.whamcloud.com/35028 " LU-11264 llapi: avoid pool verification overhead " to reduce the overhead of checking the pools for each file, but didn't have time to finish it off. It would be best to start with updating that patch (maybe split into 2-3 smaller patches to separate the different changes) to reduce the overhead of checking pool validity before adding even more checks to setstripe. Note that Lustre itself doesn't care if the pool is wrong or missing. This can still happen even if setstripe validates the input, if the pool is deleted afterward. It will allocate as if no pool is specified in that case.

Peter Jones made changes - 30/Jul/21 5:59 PM

Assignee

Original: WC Triage [ wc-triage ]

New: Etienne Aujames [ eaujames ]

Etienne Aujames made changes - 30/Jul/21 5:21 PM

Description

Original: The following scenario is possible:
{code:java}
[root@client client]# lfs setstripe -E100M -c1 -p flash -E-1 -p not_a_pool test
[root@client client]# lfs getstripe test
test
  lcm_layout_gen: 2
  lcm_mirror_count: 1
  lcm_entry_count: 2
    lcme_id: 1
    lcme_mirror_id: 0
    lcme_flags: init
    lcme_extent.e_start: 0
    lcme_extent.e_end: 104857600
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: 1
      lmm_pool: flash
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x83:0x0] }

    lcme_id: 2
    lcme_mirror_id: 0
    lcme_flags: 0
    lcme_extent.e_start: 104857600
    lcme_extent.e_end: EOF
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: -1
      lmm_pool: not_a_pool

{code}
"lfs" should check if the specified pool "not_a_pool" exist before generate and write the layout (trusted.lov) on the filesystem.

The issue does not exist with plain layout:
{code:java}
[root@client client]# lfs setstripe -c-1 -p not_a_pool test
lfs setstripe: pool 'lustrefs.not_a_pool' does not exist: Invalid argument (22)
{code}

New: The following scenario is possible:
{code:java}
[root@client client]# lfs setstripe -E100M -c1 -p flash -E-1 -p not_a_pool test
[root@client client]# lfs getstripe test
test
  lcm_layout_gen: 2
  lcm_mirror_count: 1
  lcm_entry_count: 2
    lcme_id: 1
    lcme_mirror_id: 0
    lcme_flags: init
    lcme_extent.e_start: 0
    lcme_extent.e_end: 104857600
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: 1
      lmm_pool: flash
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x83:0x0] }

    lcme_id: 2
    lcme_mirror_id: 0
    lcme_flags: 0
    lcme_extent.e_start: 104857600
    lcme_extent.e_end: EOF
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: -1
      lmm_pool: not_a_pool

{code}
"lfs" should check if the specified pool "not_a_pool" exists before generate and write the layout (trusted.lov) on the filesystem.

The issue does not exist with plain layout:
{code:java}
[root@client client]# lfs setstripe -c-1 -p not_a_pool test
lfs setstripe: pool 'lustrefs.not_a_pool' does not exist: Invalid argument (22)
{code}

Etienne Aujames made changes - 30/Jul/21 5:21 PM

Description

New: The following scenario is possible:
{code:java}
[root@client client]# lfs setstripe -E100M -c1 -p flash -E-1 -p not_a_pool test
[root@client client]# lfs getstripe test
test
  lcm_layout_gen: 2
  lcm_mirror_count: 1
  lcm_entry_count: 2
    lcme_id: 1
    lcme_mirror_id: 0
    lcme_flags: init
    lcme_extent.e_start: 0
    lcme_extent.e_end: 104857600
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: 1
      lmm_pool: flash
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x83:0x0] }

    lcme_id: 2
    lcme_mirror_id: 0
    lcme_flags: 0
    lcme_extent.e_start: 104857600
    lcme_extent.e_end: EOF
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: -1
      lmm_pool: not_a_pool

{code}
"lfs" should check if the specified pool "not_a_pool" exist before generate and write the layout (trusted.lov) on the filesystem.

The issue does not exist with plain layout:
{code:java}
[root@client client]# lfs setstripe -c-1 -p not_a_pool test
lfs setstripe: pool 'lustrefs.not_a_pool' does not exist: Invalid argument (22)
{code}

Etienne Aujames made changes - 30/Jul/21 5:20 PM

Description

New: The following scenario is possible:
{code:java}
[root@client client]# lfs setstripe -E100M -c1 -p flash -E-1 -p not_a_pool test
[root@client client]# lfs getstripe test
test
  lcm_layout_gen: 2
  lcm_mirror_count: 1
  lcm_entry_count: 2
    lcme_id: 1
    lcme_mirror_id: 0
    lcme_flags: init
    lcme_extent.e_start: 0
    lcme_extent.e_end: 104857600
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: 1
      lmm_pool: flash
      lmm_objects:
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0x83:0x0] }

    lcme_id: 2
    lcme_mirror_id: 0
    lcme_flags: 0
    lcme_extent.e_start: 104857600
    lcme_extent.e_end: EOF
      lmm_stripe_count: 1
      lmm_stripe_size: 1048576
      lmm_pattern: raid0
      lmm_layout_gen: 0
      lmm_stripe_offset: -1
      lmm_pool: not_a_pool

{code}
"lfs" should check if the specified pool "not_a_pool" exist before generate and write the layout (trusted.lov) on the filesystem.

The issue does not exist with plain layout:
{code:java}
// code placeholder
{code}

Etienne Aujames made changes - 30/Jul/21 5:08 PM

Summary

Original: "lfs setstripe/migrate" should check if pool exist for PFL components

New: "lfs setstripe/migrate" should check if pool exists for PFL components

Etienne Aujames created issue - 30/Jul/21 5:07 PM

People

Assignee:: Etienne Aujames

Reporter:: Etienne Aujames

Votes:: 1 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 30/Jul/21 5:07 PM

Updated:: 30/Jul/21 6:10 PM