Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-21068

sanityn test_81d: client OOM forking 10000 mrename processes

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Medium Medium
    • Lustre 2.18.0
    • None
    • None
    • 3
    • 9223372036854775807

      sanityn test_81d (parallel rename file cross-dir, added by LU-17426) starts one background mrename per file with no throttle: MDSCOUNT * COUNT = 10000 processes with the default COUNT=2500 and 4 MDTs, then waits for them all.

      On a 3GB aarch64 client with 16KB PAGE_SIZE (rocky10 6.12.0-211.47.1.el10_2) the test was OOM-killed in about 1/3 of runs. At the OOM about 5700 mrename processes were alive: page tables used 650MB (16KB pages make each minimal process's page tables ~4x larger than with 4KB pages) plus 94MB of kernel stacks and transient COW copies of the forked test shell. Lustre page cache was ~2MB, so this is the test, not the filesystem.

      Out of memory: Killed process 536825 (bash) total-vm:17264kB, anon-rss:12992kB, ... pgtables:128kB
       sanityn test_81d: @@@@@@ FAIL: not crossdir: 9998 != 10000
      

      Fix: limit the number of concurrent mrename processes, while still renaming from both mounts and all MDTs at the same time.

            adilger Andreas Dilger
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: