Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20374

sec: SSK mount failure under sudo due to silent setperm failure in sk_load_key

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Medium Medium
    • Lustre 2.18.0
    • None
    • None
    • 3
    • 9223372036854775807

      Attempting to mount a target with skpath= under sudo fails with error:

      [vagrant@server1 ~]$ sudo lctl set_param sptlrpc.gss.lgss_keyring.debug_level=4
      sptlrpc.gss.lgss_keyring.debug_level=4
      [vagrant@server1 ~]$ sudo mkdir -p /etc/lustre/ssk                                 [vagrant@server1 ~]$ sudo lgss_sk -w /etc/lustre/ssk/reproducer.key \                          -f sktest \                                                                        -g '127.0.0.1@tcp' \                                                              -t mgs,server,client
      Reading random data for shared key from '/dev/random'
      Generating DH parameters, this can take a while...
      error: opening '/etc/lustre/ssk/reproducer.key': File exists
      [vagrant@server1 ~]$ sudo rm /etc/lustre/ssk/reproducer.key
      [vagrant@server1 ~]$ sudo lgss_sk -w /etc/lustre/ssk/reproducer.key -f sktest -g '127.0.0.1@tcp' -t mgs,server,client
      Reading random data for shared key from '/dev/random'
      Generating DH parameters, this can take a while...
      [vagrant@server1 ~]$ sudo lgss_sk -r /etc/lustre/ssk/reproducer.key | head -15 \
          | tee $ARTIFACTS/keyfile_header.txt
        0000: 297a 481f 0302 670a e315 587c 75a7 b8ef  )zH...g...X|u...
        0010: 2dc4 6edd 768f 0734 b63d 5932 0cea a0cd  -.n.v..4.=Y2....
        0000: f381 365f c632 b2fb dcb6 4d64 b642 73db  ..6_.2....Md.Bs.
        0010: 07af 1dec f07b 1254 0f9d 9fd5 bef3 668c  .....{.T......f.
        0020: d341 cac3 149f 0325 9f94 461a 463a ba4c  .A.....%..F.F:.L
        0030: 778a 9f0c c7bb 5a9d 92ac 1255 057e 2fa3  w.....Z....U.~/.
        0040: 0c20 ae52 1ff3 a6ee 9d20 bd7d ee0a 4776  . .R..... .}..Gv
        0050: ce04 3d21 5ed0 591f 55df a611 193a b409  ..=!^.Y.U....:..
        0060: f410 c49b 91b6 de4c 4671 8ab9 1a8e 1d9e  .......LFq......
        0070: 264b 2803 c115 feb5 7943 14d3 8317 c279  &K(.....yC.....y
        0080: 72ce fe7f 6977 0973 399e d0f9 b0d6 dc44  r...iw.s9......D
        0090: 8392 fff0 6f47 020f e1cb 0c05 08f9 953c  ....oG.........<
        00a0: 6d07 b79b 4b34 87fb b980 6cfc e17a e7ec  m...K4....l..z..
        00b0: e507 631e 1d69 2361 a222 27db f78a 8bc6  ..c..i#a."'.....
        00c0: 6fab 76dc 8a6d 9e95 b125 df5e ae79 6ff6  o.v..m...%.^.yo.
        00d0: 8723 8289 0a69 e52f d57b 853c e0e8 fc1e  .#...i./.{.<....
        00e0: 1a1c e04e d13d 12dc 72be 2262 ac2c 923e  ...N.=..r."b.,.>
        00f0: aff0 2d26 c524 9008 d3b2 9017 2f64 bff7  ..-&.$....../d..
      Version:        1
      Type:           mgs server client                                         
      HMAC alg:       sha256
      Crypto alg:     ctr(aes)    
      Ctx Expiration: 604800 seconds
      Shared keylen:  256 bits
      Prime length:   2048 bits
      File system:    sktest
      MGS NIDs:       127.0.0.1@tcp
      Nodemap name:   default
      Shared key:
      Prime (p):[vagrant@server1 ~]$ sudo keyctl clear @u
      [vagrant@server1 ~]$ sudo systemctl start lsvcgss.service
      [vagrant@server1 ~]$ sudo mount -t lustre -o skpath=/etc/lustre/ssk,mgssec=skn /dev/disk/by-id/virtio-OST0001 /mnt/lustre-OST0001 

      Results in error:

      LustreError: 77048:0:(gss_keyring.c:1008:gss_sec_lookup_ctx_kr()) lustre-MDT0001-mdc-ffff8fc4c64d1800: request key failed for uid 0: rc = -126 

      This means that attempting to mount a target, or a client, with SSK loaded in this way, via ‘sudo’ (commonly used by ansible environments), or very likely when mounted via a pacemaker resource-agent, the key will fail to be loaded correctly.

      This appears to have been encountered before but there doesn’t appear to be any Jira tickets created for the issue:

      http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/2020-February/016969.html

      http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/2018-June/015668.html

      Expection

      Creating and loading an SSK should work when using sudo as shown.

            mrasobarnett Matt Rásó-Barnett
            mrasobarnett Matt Rásó-Barnett
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: