[LU-14876] OUT: possible concurrent execution of UPDATE request and its resent Created: 21/Jul/21 Updated: 10/Mar/22 Resolved: 18/Aug/21 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.12.7, Lustre 2.15.0 |
| Fix Version/s: | Lustre 2.12.8, Lustre 2.15.0 |
| Type: | Bug | Priority: | Major |
| Reporter: | Mikhail Pershin | Assignee: | Mikhail Pershin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||
| Severity: | 3 | ||||
| Rank (Obsolete): | 9223372036854775807 | ||||
| Description |
|
There is possible LBUG() in out_reconstruct(): lustre_update.h:246:object_update_result_insert()) LBUG Bug happened because export lcd_last_xid became the same as request rq_xid in the middle of OUT UPDATE resent processing. 1. The first update with index 0 was not equal to lcd_last_xid and was added as normal update to be processed. I am pretty sure about sequence and log messages confirm that. This revealed at least two issues with OUT UPDATE resent handling. 1. req_xid_is_last() check shouldn't be done for each update, this is always the same request with the same XID, so it is either last or not and that should be checked only once prior updates processing. 2. In step #2 of scenario the lcd_last_xid was changed and became the same as request's one. This is the real problem and means that original request was processing while resent also starts processing. For ordinary clients this is prevented by checking exp_rpc_count in target_handle_connec() but for MDS-MDS re-connection it is flawed somewhere it seems. |
| Comments |
| Comment by Gerrit Updater [ 21/Jul/21 ] |
|
Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/44362 |
| Comment by Mikhail Pershin [ 22/Jul/21 ] |
|
As I assumed in description, there is flaw in target_handle_connect() allowing concurrent execution of original request and its resent on server. I am preparing patch for this. |
| Comment by Gerrit Updater [ 23/Jul/21 ] |
|
Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/44390 |
| Comment by Gerrit Updater [ 18/Aug/21 ] |
|
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/44390/ |
| Comment by Peter Jones [ 18/Aug/21 ] |
|
Landed for 2.15 |
| Comment by Gerrit Updater [ 13/Sep/21 ] |
|
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/44362/ |