Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-21181

ptlrpc: bulk descriptor too small on 64K-page servers (bd_iov_count <= bd_max_iov LBUG)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • 3
    • 9223372036854775807

      On a node with PAGE_SIZE > 4096 (aarch64, 64KiB pages), an OSS crashes in the bulk write path:

      sanity test 119p (unaligned DIO writev, offsets 0x7ffff/0x100001/0x180000) on a 64K-page node. It crashes the server every time. The same test passes on a 4K-page node with the same tree.

      LustreError: (client.c:270:__ptlrpc_add_bulk_chunk()) ASSERTION( desc->bd_iov_count <= desc->bd_max_iov ) failed:
      LBUG
        __ptlrpc_prep_bulk_page → ptlrpc_prep_bulk_page_nopin → tgt_brw_write → tgt_handle_request0

       

      Cause: __ptlrpc_prep_bulk_page() splits a page into two kiovs when it crosses an LNET_MTU MD boundary (introduced with LU-19180 and LU-20520, which key MD boundaries on the 4KiB-virtual bd_iop_len). With 4K pages this never happens, so one kiov per page is enough. A 64K page can cross boundary, and ptlrpc_new_bulk() allocates bd_vec with exactly nfrags entries. LU-20520 noted this gap in its commit message: “A large-page server needs a separate fix: ptlrpc_prep_bulk_exp() sizes the descriptor at the bare page count, with no room for the split kiovs.”

            diwa Di Wang
            diwa Di Wang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: