Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16920

CSDC_v2: store and report compression flag on OST and MDT objects



    • Improvement
    • Resolution: Unresolved
    • Major
    • Lustre 2.17.0
    • None
    • 3
    • 9223372036854775807


      In order to allow device-level scanners to detect compressed OST objects (and to a lesser extent compressed MDT DoM objects) during scanning, it makes sense to store a flag on each OST object that is storing compressed data. This will allow the scanner to identify compressed objects, for example to compute the compression ratio during an OST-levels scan, and/or implement other policies during scanning without having the full file layout.

      While there is an existing EXT4_COMPR_FL that could be stored on the ldiskfs inode for this purpose, it may have adverse interactions with ext4 or e2fsck now or in the future, if ext4-level compression is ever implemented.

      Instead, we should store the compressed object state with LMAC_COMPRESSED=0x00000080 in the trusted.lma xattr on each object. I don't believe that the storage of compressed objects should need an LMAI_COMPRESSED (incompatible) flag, since there is nothing on the OST that needs to be aware of this state during a read, since the decompression is handled entirely on the client.

      The OST would be informed by the client about the presence of compressed data with the OBD_BRW_COMPRESSED flag added in patch https://review.whamcloud.com/50154 "LU-16603 protocol: add OBD_BRW_COMPRESSED".

      Other than saving the flag on the OST object one time, there is no other action required by the OST in this case.

      For non-DoM files, the MDT will not see the OBD_BRW_COMPRESSED flag. The MDS could set the LMAC_COMPRESSED flag in trusted.xattr as soon as a compressed component is instantiated.

      The MDS should report the LMAC_COMPRESSED flag in the inode flags as LUSTRE_COMPR_FL=FS_COMPR_FL=0x00000004, similar to LUSTRE_ENCRYPT_FL, so that the compression state can be displayed on the client via lsattr. It should report STATX_ATTR_COMPRESSED=0x00000004 via statx(), similar to STATX_ATTR_ENCRYPTED.


        Issue Links



              bobijam Zhenyu Xu
              adilger Andreas Dilger
              0 Vote for this issue
              3 Start watching this issue