Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
Lustre 2.4.0
-
1
-
3
-
2760
Description
Commit: ORI-293
Maloo: https://maloo.whamcloud.com/test_sets/b22fa740-0832-11e1-b0d9-52540025f9af
The tests should show the actual lsattr outputs to facilitate diagnosis.
== sanity test 52a: append-only flag test (should return errors) ======= 23:01:45 (1320386505) cp: cannot create regular file `/mnt/lustre/d52a/foo': Operation not permitted rename returned -1: Operation not permitted UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 9984896 8576 9974272 0% /mnt/lustre[MDT:0] lustre-OST0000_UUID 9985024 6784 9976064 0% /mnt/lustre[OST:0] lustre-OST0001_UUID 9984768 4992 9977728 0% /mnt/lustre[OST:1] lustre-OST0002_UUID 9984768 1408 9981312 0% /mnt/lustre[OST:2] lustre-OST0003_UUID 9984384 6528 9975808 0% /mnt/lustre[OST:3] filesystem summary: 39938944 19712 39910912 0% /mnt/lustre osp.lustre-OST0000-osp-MDT0000.sync_changes=0 osp.lustre-OST0000-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0000-osp-MDT0000.sync_in_progress=1 osp.lustre-OST0001-osp-MDT0000.sync_changes=0 osp.lustre-OST0001-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0001-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0002-osp-MDT0000.sync_changes=0 osp.lustre-OST0002-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0002-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0003-osp-MDT0000.sync_changes=0 osp.lustre-OST0003-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0003-osp-MDT0000.sync_in_progress=1 osp.lustre-OST0000-osp-MDT0000.prealloc_status=0 osp.lustre-OST0001-osp-MDT0000.prealloc_status=0 osp.lustre-OST0002-osp-MDT0000.prealloc_status=0 osp.lustre-OST0003-osp-MDT0000.prealloc_status=0 sanity test_52a: @@@@@@ FAIL: lsattr
== sanity test 52b: immutable flag test (should return errors) ========= 23:02:27 (1320386547) /usr/lib64/lustre/tests/sanity.sh: line 3132: /mnt/lustre/d52b/foo: Permission denied cp: cannot create regular file `/mnt/lustre/d52b/foo': Permission denied /usr/lib64/lustre/tests/sanity.sh: line 3136: /mnt/lustre/d52b/foo: Permission denied rename returned -1: Operation not permitted UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 9985024 7680 9975296 0% /mnt/lustre[MDT:0] lustre-OST0000_UUID 9985024 6784 9976064 0% /mnt/lustre[OST:0] lustre-OST0001_UUID 9984768 5120 9977600 0% /mnt/lustre[OST:1] lustre-OST0002_UUID 9984768 1408 9981312 0% /mnt/lustre[OST:2] lustre-OST0003_UUID 9984384 6528 9975808 0% /mnt/lustre[OST:3] filesystem summary: 39938944 19840 39910784 0% /mnt/lustre osp.lustre-OST0000-osp-MDT0000.sync_changes=0 osp.lustre-OST0000-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0000-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0001-osp-MDT0000.sync_changes=0 osp.lustre-OST0001-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0001-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0002-osp-MDT0000.sync_changes=0 osp.lustre-OST0002-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0002-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0003-osp-MDT0000.sync_changes=0 osp.lustre-OST0003-osp-MDT0000.sync_in_flight=0 osp.lustre-OST0003-osp-MDT0000.sync_in_progress=0 osp.lustre-OST0000-osp-MDT0000.prealloc_status=0 osp.lustre-OST0001-osp-MDT0000.prealloc_status=0 osp.lustre-OST0002-osp-MDT0000.prealloc_status=0 osp.lustre-OST0003-osp-MDT0000.prealloc_status=0 sanity test_52b: @@@@@@ FAIL: lsattr
Attachments
Activity
This patch sets attributes based on existing ZFS flags; so it only supports APPEND, NODUMP, and IMMUTABLE. Not sure if an error should be returned for trying to set a flag that isn't supported.
I'll add conversion functions in osd-zfs and use them in osd_attr_get/osd_attr_set to convert between the zfs flags and lustre flags.
Please re-triage this potential blocker. (Looks this hasn't been triaged before.)
If I understand it correctly, this issue could cause incorrect on-disk flags, which might become an interoperability issue in the future. It is more economic to resolve before the first ZFS-capable release.
Current code still write LUSTRE_IMMUTABLE_FL, LUSTRE_APPEND_FL, etc. directly into on-disk zp_flags. These may be interpreted as ZFS_ACL_PROTECTED, ZFS_ACL_DEFAULTED, etc. For example, I copied test-framework.sh into a ZFS-based Lustre file system, set it as append-only, and got
[root@linux tests]# zdb -vvvv lustre-mdt1/mdt1 179
Dataset lustre-mdt1/mdt1 [ZPL], ID 37, cr_txg 6, 2.66M, 178 objects, rootbp DVA[0]=<0:124200:200> DVA[1]=<0:2505400:200> [L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P birth=73L/73P fill=178 cksum=144b4bbe24:75995a0c043:1656808e490d0:2f287edabb8c2a
Object lvl iblk dblk dsize lsize %full type
179 1 16K 128K 1K 128K 0.00 ZFS plain file
192 bonus System attributes
dnode flags: USED_BYTES USERUSED_ACCOUNTED SPILL_BLKPTR
dnode maxblkid: 0
path /ROOT/test-framework.sh
uid 0
gid 0
atime Mon Apr 8 16:44:50 2013
mtime Mon Apr 8 16:44:50 2013
ctime Mon Apr 8 16:44:50 2013
crtime Mon Apr 8 16:44:50 2013
gen 55
mode 100644
size 0
parent 166
links 1
pflags 20
rdev 0x0000000000000000
The "pflags" contained ZFS_ACL_DEFAULTED.
Looking into this code more closely, it seems that some functionality was lost compared to b1_8 on ldiskfs as well. The current code will only handle a small number of attributes (IMMUTABLE, APPEND_ONLY, possibly a few others), while the old code would transparently pass all of the EXT4 flags (e.g. NODUMP, NOATIME, TOPDIR, SYNC, etc).
This looks like it is missing the code at the end of vattr2lu_attr() that would convert the dnode flags to LUSTRE_IMMUTABLE_FL and LUSTRE_APPEND_FL. My understanding of the ZFS code is that at least these attributes can be stored in the znode as well - see https://github.com/zfsonlinux/zfs/issues/229.
The flags that ZFS currently understands are below:
/* * Additional file level attributes, that are stored * in the upper half of zp_flags */ #define ZFS_READONLY 0x0000000100000000ull #define ZFS_HIDDEN 0x0000000200000000ull #define ZFS_SYSTEM 0x0000000400000000ull #define ZFS_ARCHIVE 0x0000000800000000ull #define ZFS_IMMUTABLE 0x0000001000000000ull #define ZFS_NOUNLINK 0x0000002000000000ull #define ZFS_APPENDONLY 0x0000004000000000ull #define ZFS_NODUMP 0x0000008000000000ull #define ZFS_OPAQUE 0x0000010000000000ull #define ZFS_AV_QUARANTINED 0x0000020000000000ull #define ZFS_AV_MODIFIED 0x0000040000000000ull #define ZFS_REPARSE 0x0000080000000000ull #define ZFS_OFFLINE 0x0000100000000000ull #define ZFS_SPARSE 0x0000200000000000ull
While the flags that Linux understands are below (for Lustre we also define LUSTRE_*_FL to equivalent values for our own internal and protocol uses):
#define FS_SECRM_FL 0x00000001 /* Secure deletion */ #define FS_UNRM_FL 0x00000002 /* Undelete */ #define FS_COMPR_FL 0x00000004 /* Compress file */ #define FS_SYNC_FL 0x00000008 /* Synchronous updates */ #define FS_IMMUTABLE_FL 0x00000010 /* Immutable file */ #define FS_APPEND_FL 0x00000020 /* writes to file may only append */ #define FS_NODUMP_FL 0x00000040 /* do not dump file */ #define FS_NOATIME_FL 0x00000080 /* do not update atime */ /* Reserved for compression usage... */ #define FS_DIRTY_FL 0x00000100 #define FS_COMPRBLK_FL 0x00000200 /* One or more compressed clusters */ #define FS_NOCOMP_FL 0x00000400 /* Don't compress */ #define FS_ECOMPR_FL 0x00000800 /* Compression error */ /* End compression flags --- maybe not all used */ #define FS_BTREE_FL 0x00001000 /* btree format dir */ #define FS_INDEX_FL 0x00001000 /* hash-indexed directory */ #define FS_IMAGIC_FL 0x00002000 /* AFS directory */ #define FS_JOURNAL_DATA_FL 0x00004000 /* Reserved for ext3 */ #define FS_NOTAIL_FL 0x00008000 /* file tail should not be merged */ #define FS_DIRSYNC_FL 0x00010000 /* dirsync behaviour (directories only) */ #define FS_TOPDIR_FL 0x00020000 /* Top of directory hierarchies*/ #define FS_EXTENT_FL 0x00080000 /* Extents */ #define FS_DIRECTIO_FL 0x00100000 /* Use direct i/o */ #define FS_NOCOW_FL 0x00800000 /* Do not cow file */
So at least FS_IMMUTABLE_FL <=> ZFS_IMMUTABLE, FS_APPEND_FL <=> ZFS_APPEND_ONLY, and FS_NODUMP_FL <=> ZFS_NODUMP are direct equivalents. To fix this bug I think it would be sufficient to convert between these flags in lu_attr2vattr() and vattr2lu_attr(), or possibly a pair of small helper functions.
It may make sense at some point to allow e.g. FS_NOCOMP_FL to disable compression on a file, and FS_COMPR_FL to enable compression on a file in ZFS, but that is for a separate patch.
Landed for 2.4