Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-410

Performance concern with Shrink file_max_cache_size to alleviate the memory pressure of OST patch for LU-15

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: Lustre 1.8.6
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Description

      Running obdfilter-survey with LU-15 patches on DDN SFA 10K it appeared the IO from Lustre to disk was not aligned because the sizes were observed to be 1020K and 4k. As the file size exceeded cache the performance issue was very aparent. Setting vm.min_free_kbyte did not help this performance issue at all. For example using obdfilter-survey to write an 8GB file to each OST woudl show approximately 30% unaligned I/O. The alingment issue was seen by observing cache statics on the DDN SFA 10K controller.

      Once we remove the shrink file_max_cache patch [define FILTER_MAX_CACHE_SIZE (8 * 1024 * 1024)] the alignment issue goes away. The many unaligned IO seems to be caused by this change in this patch and once I changed cache_file_size to 18446744073709551615 (which is 1.8.4 and 1.8.5 deafult), all IO were comming to SFA10K as aligned I/O.

      Disabling the read cache (lctl set_param=obdfilter.*.read_cache_enable=0) doesn't help which is still very strange to me..

      The only work around we have found is changing cache_file_size to a large size is only way to avoid this issue 1.8.6WC. This could have other performance implications as well.

      We hope to post some numbers and statitics but we need additional runs to gather that information.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              di.wang Di Wang
              Reporter:
              jsalinas John Salinas (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: