[LU-7426] DNE3: Current llog format for remote update llog

• Type: Improvement
• Resolution: Unresolved
• Priority: Major
• Affects Version/s: Lustre 2.8.0
• Severity: 3
• Assignee: WC Triage
• Reporter: Di Wang

Description

Current llog bitmap header and records structure is not very nice for recording update llog of DNE, especially for remote update log.

In DNE, update records will be written by top_trans_start()->sub_updates_write()->llog_osd_write_rec()->osp_md_write(), where the write buffer (header bitmap + record) will be packed into the RPC buffer, then it will be sent during transaction stop.
To avoid bitmap being over-written, these RPC needs be serialized and also sent with certain order.

But there are still other problems. In llog_osd_write_rec(), it will packed header information (bitmap, lgh_index, offset etc) into the RPC buffer, but the RPC will not be sent until the transaction stop. Then these information might be invalid when the RPC is being sent, especially if the previous llog update RPC fails. And these RPCs are executed on the remote MDT, it will definitly make update llog corrupted.

There are a few options to fix the problem

1. Once any llog update RPC fails, then all of the following RPC in the sending list fails, and OSP will update the header and the new RPC will be packed with updated header. This is 2.8 approach in patch http://review.whamcloud.com/16969 "LU-7039 llog: update llog header and size", and it is easy to implement but not very nice.

2. Add special llog API in OSP->OUT interface, so the OUT can handle these add llog update records independently, instead of only using dumb write API driven by llog_osd_write_rec(). For example we can add such llog updates inside the RPC,
1. append the record.
2. get the llog index.
3. update record with the index.
4. set the bitmap by index.

3.Using other format for update format.

Any thoughts?

Issue Links

• is related to: LU-8753 Recovery already passed deadline with DNE
• is related to: LU-12310 MDT Device-level Replication/Mirroring
• is related to: LU-17821 LMR1c: Improve DNE Distributed Transaction Performance
• is related to: LU-4215 Some expected improvements for OUT
• is related to: LU-7039 llog_osd.c:778:llog_osd_next_block()) ASSERTION( last_rec->lrh_index == tail->lrt_index ) failed:
• is related to: LU-4009 Add ZIL support to osd-zfs
• is related to: LU-4215 Some expected improvements for OUT
• is related to: LU-7427 DNE3: multiple entries for BATCHID