Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20462 Lustre Find Utility (LFU)
  3. LU-21100

LFU: resume a device scan from a bookmark

XMLWordPrintable

    • Icon: Technical task Technical task
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      A scan of a large target with llapi_scan_device() can run for hours. If the application restarts, the scan has to start again from the beginning. OI Scrub and LFSCK keep a position so they can resume; an application running a scan needs the same, without touching the scrub's own state.

      Proposal (raised by Andreas):

      • The library generates an opaque, versioned bookmark of LLAPI_SCAN_BOOKMARK_SIZE bytes and passes it to a new callback, lfsp_bookmark, each time the scan finishes more of the target.
      • The application stores it and passes it back in lfsp_resume to start the scan there after a restart.
      • The bookmark is the lowest chunk any worker still holds, so a resumed scan may deliver some objects again but misses none.
      • It names the target (ldiskfs s_uuid, ZFS pool and dataset guids) and the query (lfsp_want, LLAPI_SCAN_F_INTERNAL). Another target is refused with -ESTALE; another query or a damaged bookmark with -EINVAL.
      • llapi_scan_namespace() has no position to save and refuses both fields with -EOPNOTSUPP.

      Follow-ups, not in the first patch:

      • lfs find --device --bookmark FILE.
      • The same for the in-kernel OSD scanner (LU-20720), through the dt iterator's store() and load().

            hnishida Hiroshi Nishida
            hnishida Hiroshi Nishida
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: