Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20147

ptlrpc: replace per-RPC sepol call_usermodehelper with module-level cache

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • None
    • None
    • RHEL9.7, RHEL10.1
    • 2
    • 9223372036854775807

      Description

      The SELinux policy retrieval introduced in LU-8955 calls `l_getsepol` via `call_usermodehelper()` on every RPC from every import independently. This has two problems:

      Performance
      Each RPC that needs to attach SELinux policy info spawns a usermode helper process. On a busy client this means repeated `l_getsepol` forks even when the policy hasn't changed.

      RHEL/Rocky 10.1 breakage (related: LU-20045)
      On RHEL/Rocky 10.1, the kernel transitions `l_getsepol` into the `kernel_generic_helper_t` SELinux domain which is blocked from reading `/sys/fs/selinux/policyvers` and writing to debugfs. The UMH invocation silently fails, causing a 30-second timeout on every first RPC.

      Proposed fix

      Replace the per-import `call_usermodehelper` model with a single module-level `upcall_cache` (hashsize=1, key=0) shared across all imports:

      • `sptlrpc_sepol_get()` calls `upcall_cache_get_entry()` which fires `l_getsepol -k` once per TTL (controlled by `send_sepol`). The result is cached in the entry's `sepol_cache_data` and snapshotted into a ref-counted `sptlrpc_sepol` for the caller.
      • A new module-level debugfs file `sptlrpc/srpc_sepol` receives the downcall from `l_getsepol -k`. Per-OBD `srpc_sepol` is kept for backward compatibility.
      • `send_sepol=-1` (always-attach mode): TTL is set to `S32_MAX`; policy changes are detected by statting the SELinux policy directory mtime on each call. If changed, the cache entry is flushed to force a new upcall.
      • `send_sepol=N`: TTL-based refresh. On expiry, `sptlrpc_sepol_do_upcall()` stats the policy dir; if unchanged, `upcall_cache_revalidate()` re-validates the entry without spawning `l_getsepol`.
      • `l_getsepol` gains a `-k / --kernel` flag (writes to the module-level path) and a `-w / --watch` flag (inotify daemon for policy change detection, retained as an alternative approach – see design note below).

      Design note / request for community input

      Two approaches were considered for detecting policy changes between TTL expirations:

      • Dir-mtime check (implemented): kernel stats the SELinux policy directory on each `sptlrpc_sepol_get()` call (`send_sepol=-1`) or each TTL expiry (`send_sepol=N`). No daemon required. Cheap – dentry is typically cached.
      • inotify watch daemon (`l_getsepol -w`): persistent userspace process watches the policy dir and pushes on file change events. More responsive but requires a running daemon; crash goes undetected.

      The dir-mtime approach is implemented. The watch daemon code is preserved in the patch. Community feedback welcome on which approach is preferred.

            hnishida Hiroshi Nishida
            hnishida Hiroshi Nishida
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: