[LU-6] Update the readahead logic in lustre 2.0 Created: 18/Oct/10  Updated: 29/May/17  Due: 20/Oct/10  Resolved: 29/May/17

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.0.0
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Jinshan Xiong (Inactive) Assignee: Jinshan Xiong (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Environment:

lustre 2.0


Issue Links:
Duplicate
duplicates LU-8709 parallel asynchronous readahead Resolved
Bugzilla ID: 20,294
Rank (Obsolete): 10390

 Description   

Summary of the enhancement
=============================

There are several problems in current implementation of readahead in HEAD:

  • The read pattern is determined in readpage method, this causes ras_update to be confused if it
    has already had some pages in system;
  • Can't determine the stripe boundary. Of course, we can hack the code to get it, but it violates
    the principles in clio, where the stripe info is private to lov layer; Because of this, the read
    ahead mechanism may issue read request crossing stripe boundary, this is not good and causes small
    RPCs to be sent;
  • Ugly code to check if the read ahead pages are covered by lock.
  • Determine the read ahead window and issue the readahead RPC in read thread synchronously.

To address those issues, we are going to:

  • Different read ahead mechanisms for mmap and file read; For file read, we'll detect the read
    pattern in ll_file_read, and for mmap, it will be detected in ll_nopage. We may maintain different
    data structures for them; With this update, we can know the read pattern easier and preciser. We
    even may provide an interface for loadable module so that the customers are able to customize the
    mechanism by the characteristic of their applications;
  • Read ahead window is determined by the read pattern in llite layer, but how much data will be
    read in current slide window is determined by all layers(llite, lov and osc) in clio, so definitely
    we will provide a new callback interface in cl_io to do this. Basically speaking, in lov, we're
    going to cut the read ahead size to be stripe-align boundary, and in osc, the size will be adjusted
    by the coverage of ldlm lock;
  • Async(and per read process) read ahead thread. If the read pattern is good, we then spawn a read
    ahead thread to fetch the data in current read ahead window. In this way, the read ahead is able to
    be fully streamlined;
  • Lock ahead(controversial): If the read pattern is really good, I think it might make sense to
    acquire the ldlm lock inside the read ahead window earlier;
  • WNC support. In windows, reads have to be synchronous, we may have to pass on the IRP to read
    ahead thread.

Generated at Sat Feb 10 01:02:48 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.