[LU-19836] recovery-small: 26b FAIL: Client was not evicted by OSS

• Type: Bug
• Resolution: Unresolved
• Priority: Medium
• Severity: 3
• Assignee: Keguang Xu
• Reporter: Maloo

Description

This issue was created by maloo for stancheff <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/6edac57d-e12a-467e-88c1-fb823bd83d9f

Test session details:
clients: https://build.whamcloud.com/job/lustre-reviews/120960 - 4.18.0-553.89.1.el8_10.x86_64
servers: https://build.whamcloud.com/job/lustre-reviews/120960 - 4.18.0-553.89.1.el8_lustre.x86_64

<<Please provide additional information about the failure here>>

[ 1892.805087] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == recovery-small test 26b: evict dead exports =========== 10:30:49 \(1769596249\)
[ 1893.151630] Lustre: DEBUG MARKER: == recovery-small test 26b: evict dead exports =========== 10:30:49 (1769596249)
[ 1896.161985] Lustre: DEBUG MARKER: lctl set_param at_max=20
[ 1898.854739] Lustre: DEBUG MARKER: lctl get_param -n obdfilter.lustre-OST0000.num_exports
[ 1909.828247] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 1911.095598] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
...
[ 1920.033984] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 1920.260918] Autotest: Test running for 30 minutes (lustre-reviews_review-dne-selinux-ssk-part-2_120960.63)
[ 1921.315493] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
...
[ 1997.515328] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 1998.784518] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 1999.734581] Lustre: lustre-OST0000: haven't heard from client 995a6c1e-04df-436f-9228-bef506527c2e (at 10.240.29.223@tcp) in 101 seconds. I think it's dead, and I am evicting it. exp ff28a010efbd4000, cur 1769596357 deadline 1769596356 last 1769596256
[ 2000.057997] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2000.771896] Lustre: lustre-OST0003: haven't heard from client 995a6c1e-04df-436f-9228-bef506527c2e (at 10.240.29.223@tcp) in 102 seconds. I think it's dead, and I am evicting it. exp ff28a010efbd5800, cur 1769596358 deadline 1769596356 last 1769596256
[ 2001.314405] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2002.605183] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2003.887793] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2005.131262] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2005.366330] Lustre: 375:0:(gss_svc_upcall.c:706:gss_svc_searchbyctx()) ctx hdl 0x323d37a4d77a96eb does not have mech ctx: rc = -2
[ 2005.368131] Lustre: 375:0:(gss_svc_upcall.c:706:gss_svc_searchbyctx()) Skipped 2 previous similar messages
[ 2006.379821] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2007.649212] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2008.957767] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2010.228108] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2011.482899] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2012.748659] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2014.013252] Lustre: DEBUG MARKER: lctl get_param -n *.lustre-OST0000.num_exports | cut -d' ' -f2
[ 2015.380406] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  recovery-small test_26b: @@@@@@ FAIL: Client was not evicted by OSS 
[ 2015.561673] Lustre: DEBUG MARKER: recovery-small test_26b: @@@@@@ FAIL: Client was not evicted by OSS

Update not seen after 105s: want '5' got '6'
 recovery-small test_26b: @@@@@@ FAIL: Client was not evicted by OSS 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:7317:error()
  = /usr/lib64/lustre/tests/recovery-small.sh:1170:test_26b()
  = /usr/lib64/lustre/tests/test-framework.sh:7693:run_one()
  = /usr/lib64/lustre/tests/test-framework.sh:7756:run_one_logged()
  = /usr/lib64/lustre/tests/test-framework.sh:7571:run_test()
  = /usr/lib64/lustre/tests/recovery-small.sh:1174:main() 

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
recovery-small: 26b FAIL: Client was not evicted by OSS 

Issue Links

• is related to: LU-11509 LDLM: replace client lock LRU with improved cache algorithm