[LU-1480] failure on replay-single test_74: ASSERTION( cfs_atomic_read(&d->ld_ref) == 0 ) failed: Refcount is 1

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.5.0
• Affects Version/s: Lustre 2.4.0, Lustre 2.4.1
• Labels: yuc2
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/8506fd4e-ad5b-11e1-8152-52540035b04c.

The sub-test test_74 failed with the following error:

test failed to respond and timed out

Info required for matching: replay-single 74

Issue Links

• is duplicated by: LU-3116 (lu_object.c:1140:lu_device_fini()) ASSERTION( cfs_atomic_read(&d->ld_ref) == 0 ) failed: Refcount is 1
• is related to: LU-2070 lu_object.c:1114:lu_device_fini()) ASSERTIO N( cfs_atomic_read(&d->ld_ref) == 0 ) failed: Refcount is 6
• is related to: LU-1773 Oops in lov_delete_raid0()
• is related to: LU-3228 fc18: sanity test_183: @@@@@@ FAIL: test_183 failed with 1

Comments

[Zhenyu Xu]

pushed a debug patch at http://review.whamcloud.com/6105

[Minh Diep]

I hit this very frequent using fc18 client running sanity test

[Jian Yu]

Lustre Build: https://build.whamcloud.com/job/lustre-master/1340/

Server: el6, x86_64
clients: fc18, x86_64

The issue occurred again and was reported in LU-3116.

[Peter Jones]

Dropping priority as no longer occurring regularly

[Zhenyu Xu]

haven't seen this issue for recent tests, I think we can lower the severity.

[Andreas Dilger]

Bobijam, this bug was reported hit 14 times in the past week, according to:

https://maloo.whamcloud.com/test_sets/query?utf8=%E2%9C%93&test_set[test_set_script_id]=&test_set[status]=&test_set[query_bugs]=LU-1480&test_session[test_host]=&test_session[test_group]=&test_session[user_id]=&test_session[query_date]=&test_session[query_recent_period]=&test_node[os_type_id]=&test_node[distribution_type_id]=&test_node[architecture_type_id]=&test_node[file_system_type_id]=&test_node[lustre_branch_id]=&test_node_network[network_type_id]=&commit=Update+results

The most recent is at:
https://maloo.whamcloud.com/test_sets/8ab52280-3536-11e2-918f-52540035b04c

Can you please check if the information you need is in one of these failures. If not, is there something that can be done to improve the debugging patch to capture the information you need?

[Zhenyu Xu]

Didn't find it, the failure illustrated in the above maloo report is that a client cannot finish inode sync while trying to umount the mount point, not relating to the device refcount issue.

[Andreas Dilger]

Bobijam, can you please look at the https://maloo.whamcloud.com/test_sets/be5714e2-2ce7-11e2-9af4-52540035b04c to see if your debugging patch contains the information you need to resolve this problem.

[Ian Colle (Inactive)]

https://maloo.whamcloud.com/test_sets/be5714e2-2ce7-11e2-9af4-52540035b04c

[Zhenyu Xu]

status update:

A debugging patch has landed in master branch and waiting for re-hits with the debug message.

[Andreas Dilger]

Alex reported in LU-2070:

please use http://review.whamcloud.com/4151 to debug