Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12071

bypass pagecache for large files

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • Lustre 2.14.0
    • None
    • None
    • 9223372036854775807

    Description

      pagecache invalidation is quite expensive:

      pagecache enabled:

      ost  1 sz 268435456K rsz 16384K obj   16 thr   64 write 5094.16 [2287.82, 6383.46] read 5295.85 [5039.42, 5567.47]

      pagecache disabled:

      ost  1 sz 268435456K rsz 16384K obj   16 thr   64 write 4460.22 [3615.59, 5151.63] read 5218.00 [4847.41, 5551.34]

      we do already have a mechanism to bypass pagecache in place, just need to use that properly.

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. LU-11341 Do not use pagecache for I/O

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-12346 ladvise willread should work even if OSD cache is disabled

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-12016 sanity: test_156 Error: ''IN CACHE: before: 8255, after: 8258''

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. LU-13680 large allocations in osd_bufs_get() failing

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-12179 allocate continuous pages when disabled page caches

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-410 Performance concern with Shrink file_max_cache_size to alleviate the memory pressure of OST patch for LU-15

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-11347 Do not use pagecache for SSD I/O when read/write cache are disabled

          • Major - Major loss of function.
          • Resolved

          Activity

            [LU-12071] bypass pagecache for large files
            pjones Peter Jones added a comment -

            Landed for 2.14

            pjones Peter Jones added a comment - Landed for 2.14
            gerrit Gerrit Updater added a comment -

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34422/
            Subject: LU-12071 osd-ldiskfs: bypass pagecache if requested
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 3043c6f18964b0cc843b2e7866756a1d1ddd61e3

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34422/ Subject: LU-12071 osd-ldiskfs: bypass pagecache if requested Project: fs/lustre-release Branch: master Current Patch Set: Commit: 3043c6f18964b0cc843b2e7866756a1d1ddd61e3
            gerrit Gerrit Updater added a comment -

            Alex Zhuravlev (bzzz@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34422
            Subject: LU-12071 osd-ldiskfs: bypass pagecache if requested
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 4652ad0b0d65b616b11c27829c221521003c623b

            gerrit Gerrit Updater added a comment - Alex Zhuravlev (bzzz@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34422 Subject: LU-12071 osd-ldiskfs: bypass pagecache if requested Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 4652ad0b0d65b616b11c27829c221521003c623b

            People

              bzzz Alex Zhuravlev
              bzzz Alex Zhuravlev
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: