[LU-8413] sanity test_101f fails with 'misses too much pages!' Created: 18/Jul/16  Updated: 28/Mar/17  Resolved: 03/Nov/16

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.9.0
Fix Version/s: Lustre 2.9.0

Type: Bug Priority: Minor
Reporter: James Nunez (Inactive) Assignee: Zhenyu Xu
Resolution: Fixed Votes: 0
Labels: None
Environment:

autotest review-zfs and review-ldisfs

Issue Links:
Duplicate
Related
is related to LU-8709 parallel asynchronous readahead Resolved
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

sanity test 101f fails with

'misses too much pages!'

From the test_log on the client, we see

== sanity test 101f: check mmap read performance ===================================================== 02:47:36 (1468403256)
/usr/bin/iozone
Cancel LRU locks on lustre client to flush the client cache
Reset readahead stats
mmap read the file with small block size
checking missing pages
 sanity test_101f: @@@@@@ FAIL: misses too much pages!

Here is the check that is failing in the test code:

       local miss=$($LCTL get_param -n llite.*.read_ahead_stats |
                        get_named_value 'misses' | cut -d" " -f1 | calc_total)

        [ $miss -lt 3 ] || error "misses too much pages ('$miss')!"

There’s nothing interesting in the console logs.

This test does NOT fail consistently and has failed with this error message six times since 13-July-2016:
2016-07-13 - https://testing.hpdd.intel.com/test_sets/43aec3ae-48e8-11e6-8968-5254006e85c2
2016-07-13 - https://testing.hpdd.intel.com/test_sets/af1ac4e8-4948-11e6-9f8e-5254006e85c2
2016-07-14 - https://testing.hpdd.intel.com/test_sets/c663d7fa-49b3-11e6-9f8e-5254006e85c2
2016-07-14 - https://testing.hpdd.intel.com/test_sets/95d24332-4a27-11e6-bf87-5254006e85c2
2016-07-17 - https://testing.hpdd.intel.com/test_sets/c3840482-4c02-11e6-bf87-5254006e85c2
2016-07-17 - https://testing.hpdd.intel.com/test_sets/ca53ff1e-4c71-11e6-9f8e-5254006e85c2

 Comments   
Comment by nasf (Inactive) [ 19/Jul/16 ]

Another failure instance on master:
https://testing.hpdd.intel.com/test_sets/19e77d8c-4d65-11e6-8968-5254006e85c2

Comment by nasf (Inactive) [ 20/Jul/16 ]

Hit it again on master:
https://testing.hpdd.intel.com/test_sets/495b62da-4e42-11e6-8968-5254006e85c2

Comment by Niu Yawei (Inactive) [ 21/Jul/16 ]

https://testing.hpdd.intel.com/test_sets/fc84531e-4e86-11e6-a80f-5254006e85c2

Comment by Jian Yu [ 22/Jul/16 ]

+1 on master branch: https://testing.hpdd.intel.com/test_sets/d4546ed4-4f7f-11e6-8968-5254006e85c2

Comment by Jian Yu [ 22/Jul/16 ]

More instance on master branch:
https://testing.hpdd.intel.com/test_sets/6f35cd3a-4fe4-11e6-8968-5254006e85c2

This is blocking patch review testing on master branch.

Comment by Bob Glossman (Inactive) [ 11/Aug/16 ]

another on master:
https://testing.hpdd.intel.com/test_sets/c602eb38-5fdf-11e6-906c-5254006e85c2

Comment by nasf (Inactive) [ 23/Aug/16 ]

Another failure on master:
https://testing.hpdd.intel.com/test_sets/a2833bf2-68d3-11e6-b2e2-5254006e85c2

Comment by nasf (Inactive) [ 14/Sep/16 ]

+1 on master:
https://testing.hpdd.intel.com/test_sets/fc74642e-79bd-11e6-8afd-5254006e85c2

Comment by Saurabh Tandan (Inactive) [ 21/Sep/16 ]

This issue seems to have occurred around 25 times in past 30 days

Comment by Gerrit Updater [ 27/Sep/16 ]

Bobi Jam (bobijam@hotmail.com) uploaded a new patch: http://review.whamcloud.com/22753
Subject: LU-8413 debug: specify READA debug mask for sanity_101f
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a7b6cf267bd0e34c901b6337c744f58339c9415f

Comment by Gerrit Updater [ 05/Oct/16 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/22753/
Subject: LU-8413 llite: specify READA debug mask for sanity_101f
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 49db763c398c140121ccffbf2021039b72d8b8b4

Comment by Emoly Liu [ 10/Oct/16 ]

I hit this issue with this debug patch at https://testing.hpdd.intel.com/test_sets/5a6e6c66-8e89-11e6-a9b0-5254006e85c2
btw, I noticed two issues in sanity.sh from http://review.whamcloud.com/22753/, which caused the following errors

error: set_param: setting /proc/sys/lnet/debug=debug=: Invalid argument
error: set_param: param_path 'super': No such file or directory
error: set_param: param_path 'neterror': No such file or directory
error: set_param: param_path 'dlmtrace': No such file or directory
error: set_param: param_path 'emerg': No such file or directory
error: set_param: param_path 'rpctrace': No such file or directory
error: set_param: param_path 'config': No such file or directory

bobijam, could you please have a look? Thanks.

Comment by Zhenyu Xu [ 10/Oct/16 ]

yes, i'll fix the debug setting issue.

While checking the client1 log, there is something strange about the iozone read order.

$ grep "miss." sanity.test_101f.debug_log.onyx-66.1476039341.log 
19:00000080:00400000:3.0:1476039340.244511:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28935 miss
27:00000080:00400000:3.0:1476039340.244906:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28934 miss
34:00000080:00400000:3.0:1476039340.245432:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28933 miss
41:00000080:00400000:3.0:1476039340.245768:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28932 miss
55:00000080:00400000:3.0:1476039340.248705:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28931 miss
63:00000080:00400000:3.0:1476039340.249145:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28930 miss
70:00000080:00400000:3.0:1476039340.249603:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28929 miss
77:00000080:00400000:3.0:1476039340.249984:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 28928 miss
866:00000080:00400000:3.0:1476039340.252362:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29191 miss.
874:00000080:00400000:3.0:1476039340.252724:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29190 miss.
881:00000080:00400000:3.0:1476039340.253225:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29189 miss.
888:00000080:00400000:3.0:1476039340.253564:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29188 miss.
902:00000080:00400000:3.0:1476039340.259401:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29187 miss.
910:00000080:00400000:3.0:1476039340.260100:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29186 miss.
917:00000080:00400000:3.0:1476039340.260721:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29185 miss.
924:00000080:00400000:3.0:1476039340.261225:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29184 miss.
1713:00000080:00400000:3.0:1476039340.263689:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29447 miss.
1721:00000080:00400000:3.0:1476039340.264153:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29446 miss.
1728:00000080:00400000:3.0:1476039340.264671:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29445 miss.
1735:00000080:00400000:3.0:1476039340.265081:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29444 miss.
1749:00000080:00400000:3.0:1476039340.270831:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29443 miss.
1757:00000080:00400000:3.0:1476039340.271476:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29442 miss.
1764:00000080:00400000:3.0:1476039340.272161:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29441 miss.
1771:00000080:00400000:3.0:1476039340.272687:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29440 miss.
2560:00000080:00400000:3.0:1476039340.275095:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29703 miss.
2568:00000080:00400000:3.0:1476039340.275561:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29702 miss.
2575:00000080:00400000:3.0:1476039340.276086:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29701 miss.
2582:00000080:00400000:3.0:1476039340.276465:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29700 miss.
2596:00000080:00400000:3.0:1476039340.279453:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29699 miss.
2604:00000080:00400000:3.0:1476039340.279918:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29698 miss.
2611:00000080:00400000:3.0:1476039340.280509:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29697 miss.
2618:00000080:00400000:3.0:1476039340.280963:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29696 miss.
3407:00000080:00400000:3.0:1476039340.283356:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29959 miss.
3415:00000080:00400000:3.0:1476039340.283863:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29958 miss.
3422:00000080:00400000:3.0:1476039340.284340:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29957 miss.
3429:00000080:00400000:3.0:1476039340.284773:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29956 miss.
3443:00000080:00400000:3.0:1476039340.290517:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29955 miss.
3451:00000080:00400000:3.0:1476039340.291154:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29954 miss.
3458:00000080:00400000:3.0:1476039340.291795:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29953 miss.
3465:00000080:00400000:3.0:1476039340.292446:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 29952 miss.
4254:00000080:00400000:3.0:1476039340.294855:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30215 miss.
4262:00000080:00400000:3.0:1476039340.295298:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30214 miss.
4269:00000080:00400000:3.0:1476039340.295788:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30213 miss.
4276:00000080:00400000:3.0:1476039340.296149:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30212 miss.
4290:00000080:00400000:3.0:1476039340.301501:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30211 miss.
4298:00000080:00400000:3.0:1476039340.302146:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30210 miss.
4305:00000080:00400000:3.0:1476039340.302762:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30209 miss.
4312:00000080:00400000:3.0:1476039340.303338:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30208 miss.
5101:00000080:00400000:3.0:1476039340.305654:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30471 miss.
5109:00000080:00400000:3.0:1476039340.306163:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30470 miss.
5116:00000080:00400000:3.0:1476039340.306693:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30469 miss.
5123:00000080:00400000:3.0:1476039340.307059:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30468 miss.
5137:00000080:00400000:3.0:1476039340.310040:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30467 miss.
5145:00000080:00400000:3.0:1476039340.310558:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30466 miss.
5152:00000080:00400000:3.0:1476039340.311225:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30465 miss.
5159:00000080:00400000:3.0:1476039340.311752:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30464 miss.
5948:00000080:00400000:3.0:1476039340.314074:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30727 miss.
5956:00000080:00400000:3.0:1476039340.314584:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30726 miss.
5963:00000080:00400000:3.0:1476039340.315161:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30725 miss.
5970:00000080:00400000:3.0:1476039340.315666:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30724 miss.
5984:00000080:00400000:3.0:1476039340.318679:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30723 miss.
5992:00000080:00400000:3.0:1476039340.319336:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30722 miss.
5999:00000080:00400000:3.0:1476039340.320055:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30721 miss.
6006:00000080:00400000:3.0:1476039340.320570:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30720 miss.
6795:00000080:00400000:3.0:1476039340.322983:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30983 miss.
6803:00000080:00400000:3.0:1476039340.323451:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30982 miss.
6810:00000080:00400000:3.0:1476039340.323984:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30981 miss.
6817:00000080:00400000:3.0:1476039340.324406:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30980 miss.
6831:00000080:00400000:3.0:1476039340.327449:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30979 miss.
6839:00000080:00400000:3.0:1476039340.327921:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30978 miss.
6846:00000080:00400000:3.0:1476039340.328506:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30977 miss.
6853:00000080:00400000:3.0:1476039340.329048:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 30976 miss.
7642:00000080:00400000:3.0:1476039340.331307:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31239 miss.
7650:00000080:00400000:3.0:1476039340.331751:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31238 miss.
7657:00000080:00400000:3.0:1476039340.332272:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31237 miss.
7664:00000080:00400000:3.0:1476039340.332641:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31236 miss.
7678:00000080:00400000:3.0:1476039340.337696:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31235 miss.
7686:00000080:00400000:3.0:1476039340.338238:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31234 miss.
7693:00000080:00400000:3.0:1476039340.338797:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31233 miss.
7700:00000080:00400000:3.0:1476039340.339400:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31232 miss.
8489:00000080:00400000:3.0:1476039340.341637:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31495 miss.
8497:00000080:00400000:3.0:1476039340.342040:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31494 miss.
8504:00000080:00400000:3.0:1476039340.342487:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31493 miss.
8511:00000080:00400000:3.0:1476039340.342844:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31492 miss.
8525:00000080:00400000:3.0:1476039340.345785:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31491 miss.
8533:00000080:00400000:3.0:1476039340.346190:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31490 miss.
8540:00000080:00400000:3.0:1476039340.346743:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31489 miss.
8547:00000080:00400000:3.0:1476039340.347150:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31488 miss.
9336:00000080:00400000:3.0:1476039340.349417:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31751 miss.
9344:00000080:00400000:3.0:1476039340.349929:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31750 miss.
9351:00000080:00400000:3.0:1476039340.350434:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31749 miss.
9358:00000080:00400000:3.0:1476039340.350891:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31748 miss.
9372:00000080:00400000:3.0:1476039340.356803:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31747 miss.
9380:00000080:00400000:3.0:1476039340.357409:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31746 miss.
9387:00000080:00400000:3.0:1476039340.357996:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31745 miss.
9394:00000080:00400000:3.0:1476039340.358566:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 31744 miss.
10183:00000080:00400000:3.0:1476039340.360888:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32007 miss.
10191:00000080:00400000:3.0:1476039340.361390:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32006 miss.
10198:00000080:00400000:3.0:1476039340.361912:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32005 miss.
10205:00000080:00400000:3.0:1476039340.362344:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32004 miss.
10219:00000080:00400000:3.0:1476039340.368322:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32003 miss.
10227:00000080:00400000:3.0:1476039340.369056:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32002 miss.
10234:00000080:00400000:3.0:1476039340.369647:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32001 miss.
10241:00000080:00400000:3.0:1476039340.370232:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32000 miss.
11030:00000080:00400000:3.0:1476039340.372680:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32263 miss.
11038:00000080:00400000:3.0:1476039340.373255:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32262 miss.
11045:00000080:00400000:3.0:1476039340.373829:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32261 miss.
11052:00000080:00400000:3.0:1476039340.374398:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32260 miss.
11066:00000080:00400000:3.0:1476039340.377361:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32259 miss.
11074:00000080:00400000:3.0:1476039340.377944:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32258 miss.
11081:00000080:00400000:3.0:1476039340.378563:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32257 miss.
11088:00000080:00400000:3.0:1476039340.379012:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32256 miss.
11877:00000080:00400000:3.0:1476039340.381237:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32519 miss.
11885:00000080:00400000:3.0:1476039340.381723:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32518 miss.
11892:00000080:00400000:3.0:1476039340.382307:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32517 miss.
11899:00000080:00400000:3.0:1476039340.382719:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32516 miss.
11913:00000080:00400000:3.0:1476039340.385964:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32515 miss.
11921:00000080:00400000:3.0:1476039340.386522:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32514 miss.
11928:00000080:00400000:3.0:1476039340.387155:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32513 miss.
11935:00000080:00400000:3.0:1476039340.387728:0:129532:0:(rw.c:738:ras_update()) [0x2000013a0:0xcd:0x0] pages at 32512 miss.
12725:00000001:02000400:17.0:1476039340.824856:0:129636:0:(debug.c:331:libcfs_debug_mark_buffer()) DEBUG MARKER: /usr/sbin/lctl mark  sanity test_101f: @@@@@@ FAIL: misses too much pages \(\'1025\'\)! 
12726:00000001:02000400:17.0:1476039341.304106:0:129705:0:(debug.c:331:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_101f: @@@@@@ FAIL: misses too much pages ('1025')!

it read backwards occasionally and that causes the page miss.

Comment by Zhenyu Xu [ 10/Oct/16 ]

the arguments used for iozone is

iozone -i 1 -+n -r 32k -s 128m -B -f $DIR/$tfile

should we limit the number of test processes as 1 with -u and -l arguments ?

Comment by Gerrit Updater [ 10/Oct/16 ]

Bobi Jam (bobijam@hotmail.com) uploaded a new patch: http://review.whamcloud.com/23039
Subject: LU-8413 test: limit # of processes for sanity test_101f
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 169987d54867a90f6193b76481915281fcededdf

Comment by Andreas Dilger [ 14/Oct/16 ]

It would be good to fix the readahead algorithm to handle backward reads, including backward strides.

However, DDN presented at LAD about improving the readahead code (http://www.eofs.eu/_media/events/lad16/19_parallel_readahead_framework_li_xi.pdf), and I believe that this also handled backward readahead, so if that will be available for 2.10 then we should use that instead of investing time into improving the existing readahead code to do this.

Comment by Andreas Dilger [ 17/Oct/16 ]

It would also be useful to know why this test has started failing more often recently? It has failed 7x in the past 2 days but only 23x in the past 6 weeks. Is this fallout from some other patch that landed?

Comment by Gerrit Updater [ 20/Oct/16 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/23039/
Subject: LU-8413 test: limit # of processes for sanity test_101f
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 713b9f566b17de08174c34ebcaf90fb3eafd76d7

Comment by Peter Jones [ 20/Oct/16 ]

Landed for 2.9

Comment by Bob Glossman (Inactive) [ 03/Nov/16 ]

reopened by mistake

Generated at Sat Feb 10 02:17:19 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.