Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-21045

Add server-side control to disable jobstats collection independently of clients

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      In cloud and managed Lustre deployments, servers and clients are often administered by different parties. The service provider manages the Lustre servers, while customers manage their clients. Server administrators therefore cannot rely on changing jobid_var on every client to control server-side jobstats collection.

      Clients can continue sending JobIDs in RPCs, including large numbers of distinct IDs. Tracking those IDs consumes kernel memory for per-job statistics and per-CPU counters, and adds lookup, allocation, accounting, and cleanup overhead. Excessive JobID counts, whether accidental or deliberate, can put unnecessary resource pressure on managed servers.

      Server administrators need an independent flag to disable jobstats accounting even when clients continue sending JobIDs. This gives the server operator control over resource consumption without requiring cooperation from every client administrator.

      Proposed behavior

      • Add a per-target job_stats_enable flag for MDTs and OSTs, exposed through mdt.<target>.job_stats_enable and obdfilter.<target>.job_stats_enable.
      • Default to 1 to preserve existing behavior.
      • When set to 0, skip jobstats accounting before looking up or allocating per-job entries. Do not create new entries or update existing jobstats counters, even when incoming RPCs contain JobIDs.
      • Continue serving metadata and I/O requests normally, including aggregate server statistics.
      • Allow collection to be re-enabled at runtime without client configuration changes or reconnects.
      • Preserve existing statistics when toggling the flag; retain the existing expiry and explicit-clear mechanisms.

      Example:

      # Disable collection on MDTs
      lctl set_param 'mdt.*.job_stats_enable=0'
      
      # Disable collection on OSTs
      lctl set_param 'obdfilter.*.job_stats_enable=0'
      
      # Re-enable collection
      lctl set_param 'mdt.*.job_stats_enable=1'
      lctl set_param 'obdfilter.*.job_stats_enable=1'
      

      Validation

      Keep client-side JobID reporting enabled and generate metadata and I/O activity. Verify that disabled targets neither create new jobstats entries nor update existing counters. Re-enable collection and confirm that accounting resumes. Normal filesystem operations and aggregate server statistics should continue throughout.

      Suggested future improvement

      Consider a configurable server-side limit on the number of distinct JobIDs tracked concurrently per MDT/OST. This would allow operators to keep jobstats enabled while bounding resource usage associated with excessive JobID counts.

      When the limit is reached, the server could skip accounting for new JobIDs until existing entries expire, while continuing to update already tracked JobIDs and serve filesystem requests normally. A diagnostic counter should expose skipped accounting so operators can identify when the configured limit affects statistics collection.

            wc-triage WC Triage
            venkata.ravi.theja.reddy.venu Venkata Ravi Theja Reddy Venuthurla
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: