Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-18169

Better support memcg for new kernels

Details

    • Bug
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      EXA performance test script hanged on SLES15sp3 on NUMA system.
      Found that once we disabled unstable_check, the test can pass.

      lctl set_param llite.*.unstable_stats=0 # disable unstable check
      

      Finally found the root reason: we are using NR_UNSTABLE_NFS wrongly, it was deprecated (DO NOT USE) on the SLES15sp3:

      NR_UNSTABLE_NFS,        /* NFS unstable pages - DEPRECATED DO NOT USE */
      

      Moreover, the cgroups (memcg) does not work for the newer kernel,
      the reason is that NR_UNSTABLE_NFS was removed, and it is wrongly
      using NR_ZONE_WRITE_PENDING for memory accounting.
      According to the kernel patch:

      "mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK instead"
      kernel v5.8-rc1 commit: 8d92890bd6b8502d6aee4b37430ae6444ade7a8c
      

      it should account unstable pages in NR_WRITEBACK and WB_WRITEBACK.
      We should fix these accordingly.

      Attachments

        Issue Links

          Activity

            [LU-18169] Better support memcg for new kernels

            "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56722
            Subject: LU-18169 osc: CL_FSYNC_RECLAIM not need to wait fsync RPC finished
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 95639ad79867ecd599b46be4c0a1944ad6bfc4a2

            gerrit Gerrit Updater added a comment - "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56722 Subject: LU-18169 osc: CL_FSYNC_RECLAIM not need to wait fsync RPC finished Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 95639ad79867ecd599b46be4c0a1944ad6bfc4a2

            "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56710
            Subject: LU-18169 llite: wait unstable pages to be committed during umount
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: bd9591dfa5356110d195430cab5391e09144c9a1

            gerrit Gerrit Updater added a comment - "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56710 Subject: LU-18169 llite: wait unstable pages to be committed during umount Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: bd9591dfa5356110d195430cab5391e09144c9a1

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56162/
            Subject: LU-18169 osc: do not use deprecated NR_UNSTABLE_NFS
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 39a465f6b3c11cbd892d456f68fdb867044d693c

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56162/ Subject: LU-18169 osc: do not use deprecated NR_UNSTABLE_NFS Project: fs/lustre-release Branch: master Current Patch Set: Commit: 39a465f6b3c11cbd892d456f68fdb867044d693c

            "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56197
            Subject: LU-18169 osc: add accounting for WB_WRITEBACK for memcg
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 09ba9727e32121b06e5904fae5a244d592ea4c1b

            gerrit Gerrit Updater added a comment - "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56197 Subject: LU-18169 osc: add accounting for WB_WRITEBACK for memcg Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 09ba9727e32121b06e5904fae5a244d592ea4c1b

            "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56162
            Subject: LU-18169 osc: do not use deprecated NR_UNSTABLE_NFS
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 731d06c3ec25054c32b237cec97c97d31215f448

            gerrit Gerrit Updater added a comment - "Qian Yingjin <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56162 Subject: LU-18169 osc: do not use deprecated NR_UNSTABLE_NFS Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 731d06c3ec25054c32b237cec97c97d31215f448

            People

              qian_wc Qian Yingjin
              qian_wc Qian Yingjin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: