PFL known issues tracking ticket
(LU-9349)
|
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.10.0 |
| Fix Version/s: | Lustre 2.11.0 |
| Type: | Technical task | Priority: | Major |
| Reporter: | James Nunez (Inactive) | Assignee: | Zhenyu Xu |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | pfl, test | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||||||||||||||||||
| Description |
|
sanity-pfl test_10 sets the default layout on $MOUNT and does not return the file system to the original state. For users running the Lustre test suites as a sanity check on their file systems, this is an unexpected side effect. We should return the file system to the state it was in before running sanity-pfl. From sanity-pfl, the following code in test 10 sets the default layout: $LFS setstripe -d $MOUNT || error "clear root layout" # set root composite layout $LFS setstripe -E 2M -c 1 -S 1M -E 16M -c2 -S 2M \ -E -1 -c 4 -S 4M $MOUNT || error "Set root layout failed" Here's what the file system layout looks like before and after sanity-pfl test 10: # lfs getstripe /lustre/scratch
/lustre/scratch
stripe_count: 1 stripe_size: 1048576 stripe_offset: -1
[root@eagle-48vm6 tests]# ./auster -k -v sanity-pfl --only 10
Started at Wed Apr 12 14:54:34 UTC 2017
…
./auster: completed with rc 0
[root@eagle-48vm6 tests]# lfs getstripe /lustre/scratch
/lustre/scratch
lcm_layout_gen: 0
lcm_entry_count: 3
lcme_id: N/A
lcme_flags: 0
lcme_extent.e_start: 0
lcme_extent.e_end: 2097152
stripe_count: 1 stripe_size: 1048576 stripe_offset: -1
lcme_id: N/A
lcme_flags: 0
lcme_extent.e_start: 2097152
lcme_extent.e_end: 16777216
stripe_count: 2 stripe_size: 2097152 stripe_offset: -1
lcme_id: N/A
lcme_flags: 0
lcme_extent.e_start: 16777216
lcme_extent.e_end: EOF
stripe_count: 4 stripe_size: 4194304 stripe_offset: -1
|
| Comments |
| Comment by Andreas Dilger [ 12/Apr/17 ] |
|
I used to try and keep some tests like sanity.sh clean enough to run on an existing filesystem, but the number of tests that try to reformat the filesystem or make other configuration changes to the filesystem is high, so I don't know if running tests on real filesystems is advisable.
That said, tests should return the filesystem configuration to the original state, which poses a separate but interesting problem - how does one copy/save/restore a PFL layout? Before PFL it was enough to store the stripe count and sometimes stripe size, but now this isn't enough. Should we allow "lfs setstripe" to accept the YAML-formatted layout from "lfs getstripe" to apply to a new file or directory? Some other mechanism to do something similar (e.g. generating command-line args for "lfs setstripe", which I don't really like)? IMHO, we should also consider a patch to change test-framework to set the default layout to PFL for all tests, make sure that all tests pass, then land this patch. After that, we should consider how to make a PFL layout the default at format time for all new filesystems (i.e. outside of test-framework) since this is the kind of feature that everyone wants, and especially users that don't even know what it is. |
| Comment by John Hammond [ 12/Apr/17 ] |
|
Bobijam or Niu, could one of you take a look at this? |
| Comment by Gerrit Updater [ 13/Apr/17 ] |
|
James Nunez (james.a.nunez@intel.com) uploaded a new patch: https://review.whamcloud.com/26576 |
| Comment by Gerrit Updater [ 13/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26578 |
| Comment by Zhenyu Xu [ 13/Apr/17 ] |
|
Is it more convenient to have some thing like lfs getstripe --param-file=template_param_file lustre_file_or_dir and then lfs setstripe --param-file=template_param_file lustre_file_or_dir than lfs setstripe --copy=lustre_old_file lustre_new_file ? User can manually edit the template_param_file to adjust the layout configuration before using it. |
| Comment by Niu Yawei (Inactive) [ 13/Apr/17 ] |
|
Bobi, I think the first two commands should be enough. Given that 'lfs getstripe' output is already in YAML format, we may even just redirect the output of 'lfs getstripe' into a config file. |
| Comment by Gerrit Updater [ 14/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26634 |
| Comment by Andreas Dilger [ 18/Apr/17 ] |
|
Sorry, I didn't see comments here. I agree that since "lfs getstripe" for composite files already produces YAML output, it would be useful to save this to a file, allow it to be edited, then use it as a template for a new file. There would need to be a new option like "lfs getstripe --yaml" (already default for composite files) that also dumps non-composite files in the same YAML format so that this works with all kinds of files. There would be some work needed for parsing the YAML input into a struct llapi_layout. It would still be useful to allow "lfs setstripe --copy=/path/to/template/file [other options] /path/to/new/file" so that there is not a need to store the layout into a temporary file and edit it by the user. This would only get the layout from the existing file, optionally modify it, then use it as the template for the new file while ignoring all per-file details of the instantiated file (FID, object IDs, stripe indices), but keeping the generic parts (component count, stripe counts, stripe sizes, pools). |
| Comment by Gerrit Updater [ 18/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26708 |
| Comment by Gerrit Updater [ 19/Apr/17 ] |
|
Andreas Dilger (andreas.dilger@intel.com) merged in patch https://review.whamcloud.com/26634/ |
| Comment by Gerrit Updater [ 26/Apr/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/26578/ |
| Comment by Gerrit Updater [ 27/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26859 |
| Comment by Gerrit Updater [ 27/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26860 |
| Comment by Gerrit Updater [ 28/Apr/17 ] |
|
Bobi Jam (bobijam@hotmail.com) uploaded a new patch: https://review.whamcloud.com/26879 |
| Comment by Gerrit Updater [ 11/May/17 ] |
|
Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: https://review.whamcloud.com/27066 |
| Comment by Gerrit Updater [ 12/May/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/26708/ |
| Comment by Gerrit Updater [ 12/May/17 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/26859/ |
| Comment by Gerrit Updater [ 03/Mar/18 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/26860/ |
| Comment by Gerrit Updater [ 06/Mar/18 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/26879/ |
| Comment by Peter Jones [ 07/Mar/18 ] |
|
Several patches landed for 2.11. Residual work will be tracked under a new ticket |
[LU-9324]