Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20147

ptlrpc: replace per-RPC sepol call_usermodehelper with module-level cache

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • RHEL9.7, RHEL10.1
    • 2
    • 9223372036854775807

    Description

      Description

      The SELinux policy retrieval introduced in LU-8955 calls `l_getsepol` via `call_usermodehelper()` on every RPC from every import independently. This has two problems:

      Performance
      Each RPC that needs to attach SELinux policy info spawns a usermode helper process. On a busy client this means repeated `l_getsepol` forks even when the policy hasn't changed.

      RHEL/Rocky 10.1 breakage (related: LU-20045)
      On RHEL/Rocky 10.1, the kernel transitions `l_getsepol` into the `kernel_generic_helper_t` SELinux domain which is blocked from reading `/sys/fs/selinux/policyvers` and writing to debugfs. The UMH invocation silently fails, causing a 30-second timeout on every first RPC.

      Proposed fix

      Replace the per-import `call_usermodehelper` model with a single module-level `upcall_cache` (hashsize=1, key=0) shared across all imports:

      • `sptlrpc_sepol_get()` calls `upcall_cache_get_entry()` which fires `l_getsepol -k` once per TTL (controlled by `send_sepol`). The result is cached in the entry's `sepol_cache_data` and snapshotted into a ref-counted `sptlrpc_sepol` for the caller.
      • A new module-level debugfs file `sptlrpc/srpc_sepol` receives the downcall from `l_getsepol -k`. Per-OBD `srpc_sepol` is kept for backward compatibility.
      • `send_sepol=-1` (always-attach mode): TTL is set to `S32_MAX`; policy changes are detected by statting the SELinux policy directory mtime on each call. If changed, the cache entry is flushed to force a new upcall.
      • `send_sepol=N`: TTL-based refresh. On expiry, `sptlrpc_sepol_do_upcall()` stats the policy dir; if unchanged, `upcall_cache_revalidate()` re-validates the entry without spawning `l_getsepol`.
      • `l_getsepol` gains a `-k / --kernel` flag (writes to the module-level path) and a `-w / --watch` flag (inotify daemon for policy change detection, retained as an alternative approach – see design note below).

      Design note / request for community input

      Two approaches were considered for detecting policy changes between TTL expirations:

      • Dir-mtime check (implemented): kernel stats the SELinux policy directory on each `sptlrpc_sepol_get()` call (`send_sepol=-1`) or each TTL expiry (`send_sepol=N`). No daemon required. Cheap – dentry is typically cached.
      • inotify watch daemon (`l_getsepol -w`): persistent userspace process watches the policy dir and pushes on file change events. More responsive but requires a running daemon; crash goes undetected.

      The dir-mtime approach is implemented. The watch daemon code is preserved in the patch. Community feedback welcome on which approach is preferred.

      Attachments

        Activity

          People

            hnishida Hiroshi Nishida
            hnishida Hiroshi Nishida
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: