[LU-13853] Describe syntax for rule based auto PCC caching

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.14.0
• Type: Bug
• Priority: Minor
• Reporter: Vikentsi Lapa
• Assignee: Qian Yingjin
• Resolution: Not a Bug
• Severity: 3

Description

Where user can find patterns and syntax to describe rules?

According to the task https://jira.whamcloud.com/browse/LU-10918

user has some form of language. So it is unclear what elements and what combinations can be applied, how to form rules. This can be included in Lustre manual with examples or in other document. (wiki page? )

For instance fname={*.doc} is applicable and works,  but what about patterns such as

"fname={file*} "fname={*file} ?

 

For two last patterns rule do not work. Consider example below:

 

/usr/sbin/lctl pcc list /mnt/lustre

/usr/sbin/lctl pcc add /mnt/lustre /mnt/pccro -p "fname={file*} roid=4"
/usr/sbin/lctl pcc list /mnt/lustre 

pcc:  
-    pccpath: /mnt/pccro    
hsmtool: posix    
rwid: 4    
roid: 4    
flags: 3e    
autocache: fname={file*}

After file was created echo > /mnt/lustre/file1

 

pdsh -w trevis-59vm[10-12] lfs pcc state /mnt/lustre/file1

trevis-59vm11: file: /mnt/lustre/file1, type: none

trevis-59vm12: file: /mnt/lustre/file1, type: none

trevis-59vm10: file: /mnt/lustre/file1, type: none

 

From my point of view

Expected: /mnt/lustre/file1 in cached state

 

So it is good to have answerst to questions below:

What elements are available in fname? Is it globbing *, ?, [] syntax supported? or it is some form of custom syntax?

Is it possible to use directory names and slash "/" symbol in pattern? 
What other syntax elements are available? logical operators? (not, or, and ?) closing elements {} - are they necessary?)

What is other keywords uid, projid etc?
Does user need  to specify this rule at all? Why it rule can't be empty?

Comments

[Qian Yingjin]

I tested and got the results as expected:

[root@qian tests]# ../utils/lctl pcc add /mnt/lustre /mnt/pcc -p "fname={file*} roid=4"
[root@qian tests]# echo "djfldj" > /mnt/lustre/file1
[root@qian tests]# ../utils/lctl pcc list /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc
    hsmtool: posix
    rwid: 4
    roid: 4
    flags: 3e
    autocache: fname={file*}
[root@qian tests]# ../utils/lfs pcc state /mnt/lustre/file1 
file: /mnt/lustre/file1, type: readwrite, PCC file: /mnt/pcc/0001/0000/0401/0000/0002/0000/0x200000401:0x1:0x0, user number: 0, flags: 0

[root@qian tests]# ../utils/lctl pcc add /mnt/lustre /mnt/pcc -p "fname={file*} roid=4 ropcc=1"
[root@qian tests]# ../utils/lctl pcc list /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc
    hsmtool: posix
    rwid: 0
    roid: 4
    flags: 2e
    autocache: fname={file*}
[root@qian tests]# echo "QQQQQQ" > /mnt/lustre/file2
[root@qian tests]# ../utils/lfs pcc state /mnt/lustre/file2
file: /mnt/lustre/file2, type: readonly, PCC file: /mnt/pcc/0002/0000/0401/0000/0002/0000/0x200000401:0x2:0x0, user number: 0, flags: 0

[Qian Yingjin]

The fname currently only supports wildcard "*" matching. ? is not supported.

The Manual has described that:
"

The following command adds a PCC backend on a client:

client# lctl pcc add /mnt/lustre /mnt/pcc  --param "projid={500,1000}&fname={*.h5},uid=1001 rwid=2"
The first substring of the config parameter is the auto-cache rule, where "&" represents the logical AND operator while "," represents the logical OR operator. The example rule means that new files are only auto cached if either of the following conditions are satisfied:

The project ID is either 500 or 1000 and the suffix of the file name is "h5";

The user ID is 1001;

"

In the current implementation, the rule can not be empty.

[Vikentsi Lapa]

Thank you for answers and explanation.  I found what was missed in my command. This time it was user error (my error), I have not specified ropcc=1 option.  Do we have something like FAQ or wiki page with user interface description for PCC? I think user errors like this one is good to describe and include in docs.

[Qian Yingjin]

Sorry, we don't have FAQ or wiki page excepted Lustre manual.

I will update the Lustre manual about PCC accordingly later.

Thanks,
Qian

[Vikentsi Lapa]

I have missed ropcc= option. So I tried with option  and command works as expected.
[root@trevis-59vm9 ~]# echo testtest > /mnt/lustre/file1
[root@trevis-59vm9 ~]# lfs pcc state /mnt/lustre/file1
file: /mnt/lustre/file1, type: readonly, PCC file: /mnt/pccro/0010/0000/0401/0000/0002/0000/0x200000401:0x10:0x0, user number: 0, flags: 0