[LU-6981] obd_last_committed is not updated in tgt_reply_data_init() Created: 10/Aug/15 Updated: 24/Aug/15 Resolved: 24/Aug/15 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.8.0 |
| Fix Version/s: | Lustre 2.8.0 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Di Wang | Assignee: | Di Wang |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||
| Severity: | 3 | ||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||
| Description |
|
In tgt_reply_data_init(), it should update obd_last_committed after analyzing those reply_data, similar as tgt_server_data_init(), otherwise obd_last_committed might be smaller than the real committed transno, and then cause recovery_thread stuck in check_for_next_transno(), which rely on the correct last committed transno. |
| Comments |
| Comment by Gerrit Updater [ 13/Aug/15 ] |
|
wangdi (di.wang@intel.com) uploaded a new patch: http://review.whamcloud.com/15971 |
| Comment by Mikhail Pershin [ 14/Aug/15 ] |
|
while I agree with patch, I wonder now recovery stuck in check_for_next_transno()? Wrong last committed value shouldn't cause such thing, we might see just longer recovery maybe but not 'hung'. Could you provide more information here? |
| Comment by Di Wang [ 14/Aug/15 ] |
|
recovery will start from last_committed transno, and it will wait for the next transno to come. If we start from a smaller transno, then the next transno will never come (because it has been committed). Note: even with that waking for gap check, this will not work, because } else if (queue_len > 0 &&
queue_len == atomic_read(&obd->obd_req_replay_clients)) {
because obd->obd_req_replay_clients == client + MDS count, while queue_len might only be == client_count. especially when multiple MDTs failed at the same time. |
| Comment by Mikhail Pershin [ 16/Aug/15 ] |
|
The queue_len is just an number of replays in the queue. If all nodes are online then it should be equal to obd_req_replay_clients when every client (and other MDS) will send replay. As I undestand the other MDT can send replays with DNE2, can't it? I mean that if check_for_next_transno() is stuck for some reason then this is not right, it wasn't designed to stuck and must go on at some point, e.g. client(MDT) eviction or so. And it obviously shouldn't be blocked by other MDT except the waiting for next replay. The obd_last_committed value is not the key here, it can cause just gaps more often but gaps may occur in normal recovery as well. |
| Comment by Gerrit Updater [ 24/Aug/15 ] |
|
Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/15971/ |
| Comment by Joseph Gmitter (Inactive) [ 24/Aug/15 ] |
|
Patch has landed for 2.8. |