[LU-9362] PFL: Files with multiple components should not return multiple values for ‘lfs getstripe’

• Type: Technical task
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.10.0
• Affects Version/s: Lustre 2.10.0
• Labels: pfl
• Assignee: Zhenyu Xu
• Reporter: James Nunez (Inactive)

Description

Recent testing on master with files with multiple components show that several tests fail due to ‘lfs getstripe’ with the index or the stripe count flags return multiple lines of output. If this impacts the Lustre test suites, this could impact exiting tools that use ‘lfs getstripe’ and expect output of a single value.

To demonstrate this, we have a file with three components:

[root@eagle-48vm6 ]# lfs getstripe /lustre/scratch/d9.sanity/d2/d3
/lustre/scratch/d9.sanity/d2/d3
  lcm_layout_gen:  0
  lcm_entry_count: 3
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 0
    lcme_extent.e_end:   2097152
      stripe_count:  1       stripe_size:   1048576       stripe_offset: -1
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 2097152
    lcme_extent.e_end:   16777216
      stripe_count:  2       stripe_size:   2097152       stripe_offset: -1
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 16777216
    lcme_extent.e_end:   EOF
      stripe_count:  4       stripe_size:   4194304       stripe_offset: -1
[root@eagle-48vm6 ]# lfs getstripe -i /lustre/scratch/d9.sanity/d2/d3
-1
-1
-1
[root@eagle-48vm6 ]# lfs getstripe -c /lustre/scratch/d9.sanity/d2/d3
1
2
4

Andreas Dilger has commented on this issue:

"lfs getstripe -c" should return a single value for the stripe count. In the design, this was the stripe count of the last instantiated component, since that is the best estimate for the number of stripes for the given file. In the implementation (AFAIK) it was the sum of all the instantiated stripe counts, which is also reasonable.

The following tests fail due to expecting a single value from ‘lfs getstripe’.
sanity test 27i, 27w, 27A - https://testing.hpdd.intel.com/test_sets/5cc43d3c-23b8-11e7-8920-5254006e85c2

Issue Links

• has to be finished together with: LU-9359 sanity test_27p: /mnt/lustre/d27p.sanity/f27p.sanity has size 4, not 80000004
• is related to: LU-9424 ‘lfs getstripe --component-count’ return nothing for non-composite files
• is related to: LU-8998 Progressive File Layout (PFL)

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26947/
Subject: LU-9362 lfs: getstripe print last init-ed component info
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: da47e9c0320c9da342b7366e45a010e5a9ba239d

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26947
Subject: LU-9362 lfs: getstripe print last init-ed component info
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b7e74920805416ba5eef6caa556d7c9a217bbd37

[Andreas Dilger]

Bobijam, can you please provide a fix for this issue. As mentioned above, I think when asking for a simple value that the result from the last instantiated component.

[Andreas Dilger]

On further reflection, I think that always returning the parameters from the last instantiated component makes more sense than returning the sum of all the components. For example, it doesn't make sense to return the sum of the stripe sizes, or the sum of the pools (what would that even be?), or the sum of the stripe indices. The only one item that is even practical is the sum of the stripe counts, but I think this exception would be confusing, and even then it isn't clearly the best option.

Returning the stripe count (and size, pool) of the last instantiated component allows applications that don't know about PFL or are copying the file to a non-PFL filesystem to create a file layout that has (approximately) the same performance as the the (normally) largest part of the file, and what would be used if more data is being written to the file (at least until it grows into the next component). This would preserve the "use best layout based on file size" property of PFL even if the file has to be flattened to use a plain layout.

[Zhenyu Xu]

The patch of LU-9359 causes this issue.