Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12337

SoM in DoM needs to work transparently

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • None
    • Lustre 2.13.0
    • None
    • master
    • 3
    • 9223372036854775807

    Description

      If all data are in DoM (no other components in OSTs), file size should be already stored into MDT object because of DoM.
      It doesn't need SoM with DoM, but 'lfs getsom' needs to work transparently same as non-DoM files.
      Here is an issue of 'lfs getsom' with DoM.

      [root@c01 ~]# mkdir /cache1/dom
      [root@c01 ~]# lfs setstripe -L mdt -E 2M /cache1/dom
      [root@c01 ~]# pdsh -w c[01-02] lctl set_param ldlm.namespaces.*.lru_size=clear
      [root@c01 ~]# pdsh -w es400nv-vm[1-4] "echo 3 > /proc/sys/vm/drop_caches"
      [root@c01 ~]# lctl get_param ldlm.namespaces.*MDT0000*.lock_count
      ldlm.namespaces.cache1-MDT0000-mdc-ffff97f3bc25d000.lock_count=0
      
      Create an file on c01
      [root@c01 ~]# dd if=/dev/zero of=/cache1/dom/file bs=1M count=1
      1+0 records in
      1+0 records out
      1048576 bytes (1.0 MB) copied, 0.00162115 s, 647 MB/s
      
      Run 'lfs getsom' on c02 (different client)
      [root@c02 ~]# lfs getsom -s /cache1/nondom/file
      /cache1/nondom/file failed to get som xattr: No such file or directory (2)
      

      Also, even 'lfs getsom' works with DoM, we don't want lock cancellation for OST objects after fetch SOM.
      Please see below SoM for non-DoM configuration.

      [root@c01 ~]# mkdir /cache1/nondom
      [root@c01 ~]# lfs setstripe -i 0 /cache1/nondom/
      [root@c01 ~]# pdsh -w c[01-02] lctl set_param ldlm.namespaces.*.lru_size=clear
      [root@c01 ~]# pdsh -w es400nv-vm[1-4] "echo 3 > /proc/sys/vm/drop_caches"
      
      [root@c01 ~]# lctl get_param ldlm.namespaces.*0000*.lock_count
      ldlm.namespaces.cache1-MDT0000-mdc-ffff97f3bc25d000.lock_count=0
      ldlm.namespaces.cache1-OST0000-osc-ffff97f3bc25d000.lock_count=0
      
      Create an file on c01
      [root@c01 ~]# dd if=/dev/zero of=/cache1/nondom/file bs=1M count=1
      1+0 records in
      1+0 records out
      1048576 bytes (1.0 MB) copied, 0.00185952 s, 564 MB/s
      [root@c01 ~]#  lctl get_param ldlm.namespaces.*0000*.lock_count
      ldlm.namespaces.cache1-MDT0000-mdc-ffff97f3bc25d000.lock_count=3
      ldlm.namespaces.cache1-OST0000-osc-ffff97f3bc25d000.lock_count=1
      
      Even run 'lfs getsom' on c02, no lock cancellation on c01
      [root@c02 ~]# lfs getsom -s /cache1/nondom/file
      [root@c01 ~]# lctl get_param ldlm.namespaces.*0000*.lock_count
      ldlm.namespaces.cache1-MDT0000-mdc-ffff97f3bc25d000.lock_count=3
      ldlm.namespaces.cache1-OST0000-osc-ffff97f3bc25d000.lock_count=1
      

      Attachments

        Issue Links

          Activity

            [LU-12337] SoM in DoM needs to work transparently

            I think this was fixed as part of the statx() patches.

            adilger Andreas Dilger added a comment - I think this was fixed as part of the statx() patches.

            Andreas, yes, MDS returns DoM size to a client

            tappro Mikhail Pershin added a comment - Andreas, yes, MDS returns DoM size to a client

            Mike, I think that this is now the case, that if the DoM component is the only initialized component of the file, then the MDS returns the size to the client for stat()?

            adilger Andreas Dilger added a comment - Mike, I think that this is now the case, that if the DoM component is the only initialized component of the file, then the MDS returns the size to the client for stat() ?

            Shuichi, IMHO "lfs getsom" shouldn't be more than a debugging tool for testing. We now have proper LSOM access via statx(), and "lfs find -lazy", and if LU-11962 is implemented we could have proper SOM for normal "stat()" calls for files that are not being modified.

            I'd rather spend effort on LU-11962 than making "lfs getsom" work for DoM files. What do you think?

            adilger Andreas Dilger added a comment - Shuichi, IMHO " lfs getsom " shouldn't be more than a debugging tool for testing. We now have proper LSOM access via statx() , and " lfs find -lazy ", and if LU-11962 is implemented we could have proper SOM for normal " stat() " calls for files that are not being modified. I'd rather spend effort on LU-11962 than making " lfs getsom " work for DoM files. What do you think?

            For lfs getsom I don't really want this to be the primary interface for accessing SOM data, but I agree it makes sense to be consistent. For DoM file's it should be possible to generate a fake trusted.som xattr on the client based on the in-memory size.

            adilger Andreas Dilger added a comment - For lfs getsom I don't really want this to be the primary interface for accessing SOM data, but I agree it makes sense to be consistent. For DoM file's it should be possible to generate a fake trusted.som xattr on the client based on the in-memory size.

            People

              wc-triage WC Triage
              sihara Shuichi Ihara
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: