Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-15642

restore server read/write latency measurements

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.16.0
    • Lustre 2.15.0
    • None
    • 3
    • 9223372036854775807

    Description

      The patch https://review.whamcloud.com/46075 "LU-12585 obdfilter: Use actual I/O bytes in stats" changed the measurement of the read/write latency stats, so that they now contain the network round-trip time, while they previously only contained the local filesystem IO time. This caused the reported latency with 46076 applied to be much higher (tens or hundreds of milliseconds) than the pre-patch latency (hundreds of microseconds for flash).

      While it may be necessary to account for the actual read/write bytes after the RPC transfer is complete, the code should account for the IO latency after the IO is complete, as it did before, rather than after the RPC is complete. The RPC stats at the OST level and on the client will include the full RPC latency, and the ofd stats should only account for the storage latency.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-16110 Make output of jobs_stats and rename_stats valid YAML

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-16231 Lustre stats header incorrectly using boot time

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-12585 obdfilter read_bytes stat reports requested read bytes and not actual read bytes

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-15642] restore server read/write latency measurements
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.16.0 [ 15190 ]
            Assignee Original: Patrick Farrell [ paf0186 ] New: Andreas Dilger [ adilger ]
            Resolution New: Fixed [ 1 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-16231 [ LU-16231 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-16110 [ LU-16110 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to DDN-2814 [ DDN-2814 ]
            chunteraa Chris Hunter (Inactive) made changes -
            Link New: This issue is related to DDN-2667 [ DDN-2667 ]
            adilger Andreas Dilger made changes -
            Assignee Original: WC Triage [ wc-triage ] New: Patrick Farrell [ paf0186 ]
            adilger Andreas Dilger made changes -
            Description Original: The patch [https://review.whamcloud.com/46076] "{{{}LU-12585 obdfilter: Use actual I/O bytes in stats{}}}" changed the measurement of the read/write latency stats, so that they now contain the network round-trip time, while they previously only contained the local filesystem IO time. This caused the reported latency with 46076 applied to be much higher (tens or hundreds of {_}milli{_}seconds) than the pre-patch latency (hundreds of {_}micro{_}seconds for flash).

            While it may be necessary to account for the actual read/write bytes after the RPC transfer is complete, the code should account for the IO latency after the IO is complete, as it did before, rather than after the RPC is complete. The RPC stats at the OST level and on the client will include the full RPC latency, and the ofd stats should only account for the storage latency.             		New: The patch [https://review.whamcloud.com/46075] "{{{}LU-12585 obdfilter: Use actual I/O bytes in stats{}}}" changed the measurement of the read/write latency stats, so that they now contain the network round-trip time, while they previously only contained the local filesystem IO time. This caused the reported latency with 46076 applied to be much higher (tens or hundreds of {_}milli{_}seconds) than the pre-patch latency (hundreds of {_}micro{_}seconds for flash).

            While it may be necessary to account for the actual read/write bytes after the RPC transfer is complete, the code should account for the IO latency after the IO is complete, as it did before, rather than after the RPC is complete. The RPC stats at the OST level and on the client will include the full RPC latency, and the ofd stats should only account for the storage latency.
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-12585 [ LU-12585 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to DDN-2801 [ DDN-2801 ]
            adilger Andreas Dilger created issue -

            People

              adilger Andreas Dilger
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: