Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16757

Allow max_cached_mb to be set as a percentage of system ram

Details

    • Improvement
    • Resolution: Duplicate
    • Minor
    • None
    • Lustre 2.15.2
    • 9223372036854775807

    Description

      When lustre clients have differing amounts of memory then setting the max_cached_mb  to an absolute number which is the same on all nodes is not likely to provide optimum results. Given vendors like to set this value and place it in the configuration log then allowing max_cached_mb to be a percentage and having the client caculate the exact value to use would be a useability win.

       

      We currently do this at Sanger using a ansible templated tuning script:

       

       

      {% if adjust_lustre_cache %} {% for key, value in lustre_cache.items() %}

      echo We will set the lustre filesystem cache for {{ key }} to {{  (ansible_memtotal_mb * value / 100 )| int }}
      lctl set_param llite.{{key*.max_cached_mb={{  (ansible_memtotal_mb * value / 100 )| int }} || true

      {% endfor %} {% endif %}

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. LU-17030 allow setting max_cached_mb with %

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-16757] Allow max_cached_mb to be set as a percentage of system ram
            pjones Peter Jones made changes -
            Link New: This issue duplicates LU-17030 [ LU-17030 ]
            pjones Peter Jones made changes -
            Assignee Original: WC Triage [ wc-triage ] New: Patrick Farrell [ paf0186 ]
            Resolution New: Duplicate [ 3 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            adilger Andreas Dilger made changes -
            Issue Type Original: New Feature [ 2 ] New: Improvement [ 4 ]
            pjones Peter Jones made changes -
            Labels New: lug23dd
            james beal James Beal made changes -
            Description Original: When lustre clients have differing amounts of memory then setting the max_cached_mb  to an absolute number which is the same on all nodes is not likely to provide optimum results. Given vendors like to set this value and place it in the configuration log then allowing max_cached_mb to be a percentage and having the client caculate the exact value to use would be a useability win.

             

            We currently do this at Sanger using a ansible templated tuning script:

             

             

            {% if adjust_lustre_cache %}

            {% for key, value in lustre_cache.items() %}

            echo We will set the lustre filesystem cache for \{{ key }} to \{{  (ansible_memtotal_mb * value / 100 )| int }}}}
            {{{}lctl set_param llite.{{key{}}}*.max_cached_mb=\{{  (ansible_memtotal_mb * value / 100 )| int }} || true }}

            {% endfor %}

            {% endif %}             		New: When lustre clients have differing amounts of memory then setting the max_cached_mb  to an absolute number which is the same on all nodes is not likely to provide optimum results. Given vendors like to set this value and place it in the configuration log then allowing max_cached_mb to be a percentage and having the client caculate the exact value to use would be a useability win.

             

            We currently do this at Sanger using a ansible templated tuning script:

             

             

            {% if adjust_lustre_cache %}

            {% for key, value in lustre_cache.items() %}

            echo We will set the lustre filesystem cache for \{{ key }} to \{{  (ansible_memtotal_mb * value / 100 )| int }}
            {{{}lctl set_param llite.{{key{}}}*.max_cached_mb=\{{  (ansible_memtotal_mb * value / 100 )| int }} || true

            {% endfor %}

            {% endif %}
            james beal James Beal made changes -
            Description Original: When lustre clients have differing amounts of memory then setting the max_cached_mb  to an absolute number which is the same on all nodes is not likely to provide optimum results. Given vendors like to set this value and place it in the configuration log then allowing max_cached_mb to be a percentage and having the client caculate the exact value to use would be a useability win.

             

            We currently do this at Sanger using a ansible templated tuning script:

             

            {{{% if adjust_lustre_cache %}}}
            {{{% for key, value in lustre_cache.items() %}}}
            {{echo We will set the lustre filesystem cache for \{{ key }} to \{{  (ansible_memtotal_mb * value / 100 )| int }}}}
            {{lctl set_param llite.\{{key}}*.max_cached_mb=\{{  (ansible_memtotal_mb * value / 100 )| int }} || true }}
            {{{% endfor %}}}
            {{{% endif %}}}             		New: When lustre clients have differing amounts of memory then setting the max_cached_mb  to an absolute number which is the same on all nodes is not likely to provide optimum results. Given vendors like to set this value and place it in the configuration log then allowing max_cached_mb to be a percentage and having the client caculate the exact value to use would be a useability win.

             

            We currently do this at Sanger using a ansible templated tuning script:

             

             

            {% if adjust_lustre_cache %}

            {% for key, value in lustre_cache.items() %}

            echo We will set the lustre filesystem cache for \{{ key }} to \{{  (ansible_memtotal_mb * value / 100 )| int }}}}
            {{{}lctl set_param llite.{{key{}}}*.max_cached_mb=\{{  (ansible_memtotal_mb * value / 100 )| int }} || true }}

            {% endfor %}

            {% endif %}
            james beal James Beal created issue -

            People

              paf0186 Patrick Farrell
              james beal James Beal
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: