[LU-9324] sanity-pfl test 10 needs to reset the file system default layout

• Type: Technical task
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.11.0
• Affects Version/s: Lustre 2.10.0
• Labels: pfl test
• Assignee: Zhenyu Xu
• Reporter: James Nunez (Inactive)

Description

sanity-pfl test_10 sets the default layout on $MOUNT and does not return the file system to the original state. For users running the Lustre test suites as a sanity check on their file systems, this is an unexpected side effect. We should return the file system to the state it was in before running sanity-pfl.

From sanity-pfl, the following code in test 10 sets the default layout:

        $LFS setstripe -d $MOUNT || error "clear root layout"

	# set root composite layout                                             
	$LFS setstripe -E 2M -c 1 -S 1M -E 16M -c2 -S 2M \
		-E -1 -c 4 -S 4M $MOUNT ||
                error "Set root layout failed"

Here's what the file system layout looks like before and after sanity-pfl test 10:

# lfs getstripe /lustre/scratch
/lustre/scratch
stripe_count:  1 stripe_size:   1048576 stripe_offset: -1
[root@eagle-48vm6 tests]# ./auster -k -v sanity-pfl --only 10
Started at Wed Apr 12 14:54:34 UTC 2017
…
./auster: completed with rc 0
[root@eagle-48vm6 tests]# lfs getstripe /lustre/scratch
/lustre/scratch
  lcm_layout_gen:  0
  lcm_entry_count: 3
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 0
    lcme_extent.e_end:   2097152
      stripe_count:  1       stripe_size:   1048576       stripe_offset: -1
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 2097152
    lcme_extent.e_end:   16777216
      stripe_count:  2       stripe_size:   2097152       stripe_offset: -1
    lcme_id:             N/A
    lcme_flags:          0
    lcme_extent.e_start: 16777216
    lcme_extent.e_end:   EOF
      stripe_count:  4       stripe_size:   4194304       stripe_offset: -1

Issue Links

• is related to: LU-10353 parallel-scale* tests fail with ‘No space left on device’
• is related to: LU-7723 few tests in sanity.sh fail because of STRIPES_PER_OBJ=-1
• is related to: LU-9897 Tracking of lustre library improvements.
• is related to: LU-9486 sanity test 27D failed with 'llapi_layout_test failed'
• is related to: LU-11918 Allow setting default file layout on root directory at mkfs time
• is related to: LU-8998 Progressive File Layout (PFL)

Comments

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26859
Subject: LU-9324 lnet: move cyaml.h under lnet/include/
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: adc09ec2bcb8222072e6ed0d1162f8d932f81fe1

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26578/
Subject: LU-9324 test: get layout parameters
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 183f0a19969ce1fc10b8bf5156beaa1fc29bbd0b

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch https://review.whamcloud.com/26634/
Subject: LU-9324 test: get layout parameters
Project: fs/lustre-release
Branch: pfl
Current Patch Set:
Commit: 261406b2e2f9c52752067cf4040477152faac6f2

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26708
Subject: LU-9324 lfs: output stripe info in YAML format
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c9e8576dce58837227bc35f468df6c7e2937f7a3

[Andreas Dilger]

Sorry, I didn't see comments here.

I agree that since "lfs getstripe" for composite files already produces YAML output, it would be useful to save this to a file, allow it to be edited, then use it as a template for a new file. There would need to be a new option like "lfs getstripe --yaml" (already default for composite files) that also dumps non-composite files in the same YAML format so that this works with all kinds of files. There would be some work needed for parsing the YAML input into a struct llapi_layout.

It would still be useful to allow "lfs setstripe --copy=/path/to/template/file [other options] /path/to/new/file" so that there is not a need to store the layout into a temporary file and edit it by the user. This would only get the layout from the existing file, optionally modify it, then use it as the template for the new file while ignoring all per-file details of the instantiated file (FID, object IDs, stripe indices), but keeping the generic parts (component count, stripe counts, stripe sizes, pools).

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26634
Subject: LU-9324 test: get layout parameters
Project: fs/lustre-release
Branch: pfl
Current Patch Set: 1
Commit: 5a1c12fb6f69dd31601fd6f3b1fcd294cf251532

[Niu Yawei (Inactive)]

Bobi, I think the first two commands should be enough.
1. 'lfs getstripe' to dump layout into a config file; (--dump-layout)
2. 'lfs setstripe' to accept config file as input argument; (--layout-config)

Given that 'lfs getstripe' output is already in YAML format, we may even just redirect the output of 'lfs getstripe' into a config file.

[Zhenyu Xu]

Is it more convenient to have some thing like

lfs getstripe --param-file=template_param_file lustre_file_or_dir

and then

lfs setstripe --param-file=template_param_file lustre_file_or_dir

than

lfs setstripe --copy=lustre_old_file lustre_new_file

?

User can manually edit the template_param_file to adjust the layout configuration before using it.

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26578
Subject: LU-9324 test: get layout parameters
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cab6eb227bd348002fba1bdd817a2db3e8bbd190

[Gerrit Updater]

James Nunez ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26576
Subject: LU-9324 test: Enable default file layout in testing
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 30fce57cef946ffcf980c62c6a7d732de503dd07

[John Hammond]

Bobijam or Niu, could one of you take a look at this?