[LU-9008] Dynamic layout modification during writes

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.10.0
• Type: Technical task
• Priority: Minor
• Reporter: Joseph Gmitter (Inactive)
• Assignee: Zhenyu Xu
• Resolution: Fixed

Description

In order to simplify implementation, this project will focus on implementing composite layouts that are grown by allocating objects in non-overlapping layout extents at the end of the file, and will not implement modification of already allocated layout extents containing data.

The client IO (CLIO) layer needs to be able to manage the growth of the file layout by reconfiguring its IO stack to add new OST objects into the layout. The client will request that the MDS instantiate OST objects based on the layout template before it begins writing to a file offset beyond the currently instantiated layout components. The layout generation stored in the composite layout and in each layout extent will allow CLIO to detect whether a specific layout extent has been modified when the lock is revoked. Since the existing components of the file layout will not be modified for PFL files, any in-flight IO operations and cached data do not need to be interrupted.

Comments

[Zhenyu Xu]

This involves following changes:

• in lov_init_composite(), skip init sub object without LCME_FL_INIT layout component.
• take care of lov_dispatch[] methods of accessing un-init-ed layout component.
• issue intent RPC during write/trunc ops when try to write to a un-init-ed component (even if at the lock stage).
• initialize sub object when lov_stripe_md retrieved after intent RPC issued, and restart the IO.

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/25317
Subject: LU-9008 pfl: dynamic layout modification during write/trunc
Project: fs/lustre-release
Branch: pfl
Current Patch Set: 1
Commit: 1fab3e9b2ffd0dca32c4921b0dd7359f46c020af

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26117
Subject: LU-9008 pfl: dynamic layout modification during write/trunc
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4bd1a288b218df3b9494c4364d4f17cebdaf053d

[Gerrit Updater]

Jinshan Xiong ([e-mail]) merged in patch https://review.whamcloud.com/25317/
Subject: LU-9008 pfl: dynamic layout modification with write/truncate
Project: fs/lustre-release
Branch: pfl
Current Patch Set:
Commit: f676c7194a6095f42f23729f271b283b001f0bbc

[Joseph Gmitter (Inactive)]

Patch landed to PFL branch

[Dmitry Eremin (Inactive)]

This patch introduce the following crash in master:

 16:46:36:[14470.127738] LustreError: 22186:0:(vvp_io.c:345:vvp_io_fini()) ASSERTION( io->ci_type == CIT_WRITE || cl_io_is_trunc(io) ) failed: 
 16:46:36:[14470.132048] LustreError: 22186:0:(vvp_io.c:345:vvp_io_fini()) LBUG
 16:46:36:[14470.134194] Pid: 22186, comm: cat
 16:46:36:[14470.135970] 
 16:46:36:[14470.135970] Call Trace:
 16:46:36:[14470.139113] [<ffffffffa07107f3>] libcfs_debug_dumpstack+0x53/0x80 [libcfs]
 16:46:36:[14470.141168] [<ffffffffa0710861>] lbug_with_loc+0x41/0xb0 [libcfs]
 16:46:36:[14470.143473] [<ffffffffa0c93761>] vvp_io_fini+0x321/0x360 [lustre]
 16:46:36:[14470.145411] [<ffffffffa0beaff2>] ? lov_io_fini+0x282/0x460 [lov]
 16:46:36:[14470.147499] [<ffffffffa0805165>] cl_io_fini+0x75/0x240 [obdclass]
 16:46:36:[14470.149358] [<ffffffffa0c42f73>] ll_file_io_generic+0x2a3/0xb00 [lustre]
 16:46:36:[14470.151383] [<ffffffff81219cff>] ? touch_atime+0x12f/0x160
 16:46:36:[14470.153202] [<ffffffffa0c4409a>] ll_file_aio_read+0x34a/0x3e0 [lustre]
 16:46:36:[14470.155178] [<ffffffffa0c441fe>] ll_file_read+0xce/0x1e0 [lustre]
 16:46:36:[14470.157019] [<ffffffff811fe19e>] vfs_read+0x9e/0x170
 16:46:36:[14470.158806] [<ffffffff811fed6f>] SyS_read+0x7f/0xe0
 16:46:36:[14470.160524] [<ffffffff81696b09>] system_call_fastpath+0x16/0x1b

The code is following:

 	/**
	 * dynamic layout change needed, send layout intent
	 * RPC.
	 */
	if (io->ci_need_write_intent) {
		loff_t start = 0;
		loff_t end = 0;

		LASSERT(io->ci_type == CIT_WRITE || cl_io_is_trunc(io));
[...]
static int lov_io_rw_iter_init(const struct lu_env *env, const struct cl_io_slice *ios)
{
[...]
index = lov_lsm_entry(lsm, lio->lis_endpos - 1);
if (index > 0 && !lsm_entry_inited(lsm, index)) {
    io->ci_need_write_intent = 1;
    RETURN(io->ci_result = -ENODATA);
}

So, "io->ci_need_write_intent" can be set to "1" in read operation also.

[Dmitry Eremin (Inactive)]

This is from test failure on test suite sanity-lfsck, subtest test_18d

 

[Zhenyu Xu]

Yes, thank you for the catch, read should not trigger the layout write intent.

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26499
Subject: LU-9008 pfl: Read should not trigger layout write intent
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 298c166c6415a99af9b8dfa60a12e36989f7e88f

[Gerrit Updater]

Jinshan Xiong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26613
Subject: LU-9008 pfl: Read should not trigger layout write intent
Project: fs/lustre-release
Branch: pfl
Current Patch Set: 1
Commit: 02670a7bc3b1f3cb143e65f8b2a31331980fea16

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26622
Subject: LU-9008 pfl: Read should not trigger layout write intent
Project: fs/lustre-release
Branch: pfl
Current Patch Set: 1
Commit: a74373c9d6882991d93362d78ea17caa3bd3dd5a

[Gerrit Updater]

Jinshan Xiong ([e-mail]) merged in patch https://review.whamcloud.com/26622/
Subject: LU-9008 pfl: Read should not trigger layout write intent
Project: fs/lustre-release
Branch: pfl
Current Patch Set:
Commit: e3ae2055f629c9bfea32905cd44e22c692cd65d9

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26499/
Subject: LU-9008 pfl: Read should not trigger layout write intent
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b4b74ee1ef7a34c56ade16da892091eae28ef498