[LU-10602] Add file heat support for Persistent Client Cache Created: 05/Feb/18 Updated: 30/Jan/21 |
|
| Status: | Open |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Minor |
| Reporter: | Li Xi (Inactive) | Assignee: | Li Xi |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | PCC2 | ||
| Issue Links: |
|
||||||||||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||||||||||
| Description |
|
File heat is a relative attribute of files/objects which reflects the access frequency of the files/objects. The file heat can be controlled by two system-level constants, time period and loss percentage. Time is divided into time periods. The access number during each time period is counted. But the file heat is only recalculated at the end of a time period. And at the end of each time period, a percentage of the former file heat is lost according to the loss percentage. File heat mainly designed for cache management. Cache like PCC could use file heat to determine which files to removed from the cache or which files to fetch into cache.
|
| Comments |
| Comment by Gerrit Updater [ 05/Feb/18 ] |
|
Li Xi (lixi@ddn.com) uploaded a new patch: https://review.whamcloud.com/31161 |
| Comment by Peter Jones [ 05/Feb/18 ] |
|
Thanks Li Xi |
| Comment by Gerrit Updater [ 09/Aug/18 ] |
|
Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/32969 |
| Comment by Gerrit Updater [ 10/Aug/18 ] |
|
Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/32975 |
| Comment by Gerrit Updater [ 12/Mar/19 ] |
|
Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/34399 |
| Comment by Li Xi [ 22/Mar/19 ] |
|
It would be useful to implement TopN list support for file heat. After an hour discussion with Yingjin and Shilong about the possible designs and their cons and pros, we think the following design might work: 1) In order to avoid any performance impact, the sorting of the TopN will not add anything in the I/O process. 2) TopN sorting will be implemented using binary heap based on cfs_binheap. 3) A background thread scans "s_inodes" list of "struct super_block" and adds each inode into the binary heap. Examples of this are wait_sb_inodes(), add_dquot_ref(), drop_pagecache_sb() etc. 4) The scanning frequency could be a configurable parameter which depends on total inode numbers, heat decay time, resolution ratio (the rate that user-space tool dumps the list), etc. 5) Between handling of each inode, functions like cond_resched() should be called to avoid too high CPU usage or affecting normal I/O or file system access. 6) The binary heap is reverse order, meaning the top node is the file with least heat on the top. 7) Whenever an inode is adding to the binary heap, a new structure (e.g. "struct inode_heat_node") will be allocated for the inode with FIDs, inums and other fields properly set. 8) If the size of binary heap becomes larger than N (i.e. size == N+1) after inserting, an inodes will be removed from the top of the binary heap. 9) optimization: cfs_binheap_copy() will be implemented to copy the binary heap. When a user-space process starts to dump the TopN list, the binary heap will be copied so as to make the dumping quicker. cfs_binheap_copy() essentially copies element arrays, thus should be fast. 10) When user-space tool starts to dump the TopN list, the dumping binary heap shall be removed from the top to generate the TopN list with ascending order. The user-space tool shall change the list to descending order if needed. 11) Before dumping the TopN list, the administrator should configure which type of file heats (read, write, read sample or write sample) the sorting should be based on. N should also be a configurable parameter. These things could be configured through /sys or /proc interface. If memory is not a limitation, we could maitain four topN list at the same time for each type of file heats. 12) User-space tool shall read the TopN list and show in a proper way flushing the result after each time period (like what command top does). |
| Comment by Li Xi [ 22/Mar/19 ] |
|
Maybe cfs_binheap_() supports removing from the tail (not the top). Then the binary heap could be implemented as descending order, i.e. the top node has the highest heap. |
| Comment by Qian Yingjin [ 22/Mar/19 ] |
|
The command to show the TopN heat can be defined as follow: # lfs heat_top [-p $sample_period] [-i $heat_dimension] [-N $topN] Where We also should implement a llapi interface for users to get the TopN file heat value via IOCTL, it could be defined as:
struct lu_heat_item {
struct lu_fid lhi_fid;
struct lu_heat lhi_heat;
};
struct lu_topn_heats {
__u32 lth_topn;
struct lu_heat_item lth_item[0];
};
int llapi_heat_topn_get(const char *mntpt, int topn, char *sortkey, sturct lu_topn_heats *heats); Each time, an adminstrator wants to get the topN file heat information, it can call the llapi above. |
| Comment by Li Xi [ 22/Mar/19 ] |
|
> IMHO, we would better to avoid to use a background kernel thread, just need to scan the "s_inodes" list of "superblock" and sort the file heat when the user calls the llapi via IOCTL above in the first version implementation. Sounds reasonable, Yingjin. This would make the implementation simpler. |
| Comment by Gerrit Updater [ 29/Mar/19 ] |
|
Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/34540 |
| Comment by Gerrit Updater [ 01/Apr/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34399/ |
| Comment by Gerrit Updater [ 25/Apr/19 ] |
|
Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34757 |
| Comment by Gerrit Updater [ 08/May/19 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34757/ |