Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-11554

Make stat() work with LSOM

Details

    • Improvement
    • Resolution: Duplicate
    • Minor
    • None
    • None
    • 9223372036854775807

    Description

      Currently, stat() does not work with LSOM yet, so using "ls -l" or "mdtest" for testing is not a going to exercise the LSOM code.
      We should add mount options or proc interface (lctl set_param llite.*.lsom=1) to enable using LSOM as retruned file's size and blocks for stat() system call, and make stat() behavior selectable.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-11916 LSOM: lazy_stat mount option not recognized

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-11962 File LSOM updates to store proper size via FLR for regular stat() usage

          • Major - Major loss of function.
          • Reopened

          Improvement - An improvement or enhancement to an existing feature or task. LU-11367 integrate LSOM with lfs find

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-10934 integrate statx() API with Lustre

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-11554] Make stat() work with LSOM
            adilger Andreas Dilger added a comment -

            Closing this as a duplicate of LU-11962, which describes a way to have "LSOM" updates usable for regular stat() operations.

            adilger Andreas Dilger added a comment - Closing this as a duplicate of LU-11962 , which describes a way to have "LSOM" updates usable for regular stat() operations.
            adilger Andreas Dilger added a comment -

            I think the only thing we should do here is use known valid SOM data (not "lazy") from the MDS if it is available and can be returned to the client (e.g. from FLR files, or if a scrubbing process verifies the size and updates the state on the MDT inode like LU-11962 from userspace).

            I think some argument could be made that if a file is accessed for read but it has not been modified in some time (e.g. a day or more), then the overhead of doing an extra RPC to start modifying it is outweighed by the benefit of doing fewer RPCs to the OSTs to get the size.

            adilger Andreas Dilger added a comment - I think the only thing we should do here is use known valid SOM data (not "lazy") from the MDS if it is available and can be returned to the client (e.g. from FLR files, or if a scrubbing process verifies the size and updates the state on the MDT inode like LU-11962 from userspace). I think some argument could be made that if a file is accessed for read but it has not been modified in some time (e.g. a day or more), then the overhead of doing an extra RPC to start modifying it is outweighed by the benefit of doing fewer RPCs to the OSTs to get the size.
            adilger Andreas Dilger added a comment -

            I think this is a lot like LU-10934. The statx() API already has a mechanism to allow "lazy" size to be send to userspace applications that know they don't care about the exact size. I don't think it would be practical/possible to have all size on a mountpoint return the LSOM size via regular stat(), since that would be much more likely to cause data loss (eg. client not reading all file data because the size is too small).

            I think the integration of LSOM with "lfs find" is much higher priority. Also, FLR and DoM have totally correct SOM size/blocks, so if DoM is used to store the majority of small files, this will also avoid the majority of need for accurate SOM size.

            adilger Andreas Dilger added a comment - I think this is a lot like LU-10934 . The statx() API already has a mechanism to allow "lazy" size to be send to userspace applications that know they don't care about the exact size. I don't think it would be practical/possible to have all size on a mountpoint return the LSOM size via regular stat() , since that would be much more likely to cause data loss (eg. client not reading all file data because the size is too small). I think the integration of LSOM with " lfs find " is much higher priority. Also, FLR and DoM have totally correct SOM size/blocks, so if DoM is used to store the majority of small files, this will also avoid the majority of need for accurate SOM size.
            gerrit Gerrit Updater added a comment -

            Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/33412
            Subject: LU-11554 som: Make stat() work with LSOM
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 3eb06b38ac88e42b12de4afd81c98c202d879076

            gerrit Gerrit Updater added a comment - Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/33412 Subject: LU-11554 som: Make stat() work with LSOM Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 3eb06b38ac88e42b12de4afd81c98c202d879076

            People

              qian_wc Qian Yingjin
              qian_wc Qian Yingjin
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: