[LU-6624] LBUG in osc_lru_reclaim

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.7.0
• Labels: patch
• Severity: 3
• Assignee: Jinshan Xiong (Inactive)
• Reporter: Hiroya Nozaki (Inactive)

Description

According to the existing code, I can guess that cl_client_cache->ccc_lru needs spin_lock when being referred but the below code looks violating the rule.

osc_lru_reclaim

long osc_lru_reclaim(struct client_obd *cli)
{
        struct cl_env_nest nest;
        struct lu_env *env;
        struct cl_client_cache *cache = cli->cl_cache;
        long rc = 0;
        int max_scans;
        ENTRY;

        LASSERT(cache != NULL);
        LASSERT(!list_empty(&cache->ccc_lru)); <--- HERE

        .....

        spin_lock(&cache->ccc_lru_lock);
                                  <---- The LASSERT should be here, isn't it ?
        cache->ccc_lru_shrinkers++;

        ....

Actually I sometimes see LBUG in osc_lru_reclaim when running multiple WRITEs in the same time. So I'm convinced this LASSERT should be moved to the locked section, or the LASSERT can touch ccc_lru while the other is doing linked list operation on ccc_lru.

Comments

[Peter Jones]

Landed for 2.8

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/14901/
Subject: LU-6624 osc: LBUG in osc_lru_reclaim
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1fee634f1ebfeccb1770951ca7b576f8b6e733a0

[Jinshan Xiong (Inactive)]

Good point, if there is only one OSC, it could be empty temporarily.

[Hiroya Nozaki (Inactive)]

OK, I'll post the backtrace when this case is reproduced.
Btw, IMHO, if some cl_lru_osc()s are under list_move_tail() operation, ccc_lru can be empty temporarily, isn't it ?

[Jinshan Xiong (Inactive)]

cache->ccc_lru is the LRU list of all OSCs. Now that osc_lru_reclaim() is called, which means there exists at least one OSC, so this list shouldn't be NULL.

Can you post the backtrace to this ticket when you see it next time?

[Gerrit Updater]

Hiroya Nozaki ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14901
Subject: LU-6624 osc: LBUG in osc_lru_reclaim
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3782cd123a74302c22735db5f9c5cafda625280e

[Hiroya Nozaki (Inactive)]

I'll upload a trivial patch soon.