[LU-184] Test failure on test suite insanity, subtest test_0

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.1.0
• Severity: 3

Epic

recovery

• Assignee: Niu Yawei (Inactive)
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/b573b3c0-5c15-11e0-a272-52540025f9af.

The sub-test test_0 failed with the following error:

post-failover df: 1

Comments

[Build Master (Inactive)]

Integrated in lustre-master » client,ubuntu-x86_64 #27
LU-184 Keep orphan on failover umount

Oleg Drokin : 5b3eecce0bad98c81a45712594037b6ec7a9024f
Files :

• lustre/include/lustre/lustre_idl.h
• lustre/mdd/mdd_object.c
• lustre/mdt/mdt_open.c
• lustre/mdt/mdt_handler.c

[Build Master (Inactive)]

Integrated in lustre-master » client,el5-i686 #27
LU-184 Keep orphan on failover umount

Oleg Drokin : 5b3eecce0bad98c81a45712594037b6ec7a9024f
Files :

• lustre/include/lustre/lustre_idl.h
• lustre/mdt/mdt_open.c
• lustre/mdt/mdt_handler.c
• lustre/mdd/mdd_object.c

[Build Master (Inactive)]

Integrated in lustre-master-centos5 #199
LU-184 Keep orphan on failover umount

Oleg Drokin : 5b3eecce0bad98c81a45712594037b6ec7a9024f
Files :

• lustre/mdt/mdt_open.c
• lustre/mdt/mdt_handler.c
• lustre/mdd/mdd_object.c
• lustre/include/lustre/lustre_idl.h

[Build Master (Inactive)]

Integrated in lustre-master » client,el5-x86_64 #27
LU-184 Keep orphan on failover umount

Oleg Drokin : 5b3eecce0bad98c81a45712594037b6ec7a9024f
Files :

• lustre/mdt/mdt_handler.c
• lustre/include/lustre/lustre_idl.h
• lustre/mdt/mdt_open.c
• lustre/mdd/mdd_object.c

[Niu Yawei (Inactive)]

Yes, without this patch, it can be reproduced every time by runracer + insanity, and it passed the 'runrace + insanity' after patch applied.

[Mikhail Pershin]

You was able to reproduce that issue, does it disappear with that patch?

[Mikhail Pershin]

OK, got it, but I wonder shouldn't we use one check for client removal and orphan cleanup? Now obd_fail is checked to decide about client record removal but OBD_OPT_FAILOVER flag for orphans. We need common check for both cases I think

[Niu Yawei (Inactive)]

Hi, Tappro

In 1.8, the 'unlink_orphan' param of mds_mfd_close() should be false when it's called from umount, since the flag 'OBD_OPT_FAILOVER' should has been set to exp->exp_flags when it's not a force umount.

(see server_put_super() -> class_manual_cleanup() -> class_process_config() -> class_cleanup() -> class_disconnect_exports() -> class_disconnect_export_list() -> mds_disconnect() -> mds_cleanup_mfd() -> mds_mfd_close(). obd_fail is set in server_put_super(), exp_flags is set in class_disconnect_export_list(). )

[Mikhail Pershin]

Niu, yes, I agree though I can't find how 1.8 keeps orphans, can you show, please?

[Niu Yawei (Inactive)]

As far as I can see, the problem is that we have to preserve client data on failover umount for futhure recovery, however, the orphan was cleared while closing files on umount, such inconsistence would cause open replay error.

For 1.8, we don't have such problem, because the final close on failover umount doesn't clear orphan in 1.8.

My proposal is that we should keep orhpan on failover umount, just like 1.8 does. Which requires some code changes in close/orphan handling path, and the post recovery orphan cleanup as well. Tappro/Andreas/Wangdi/Oleg: Does it sound ok? If you all agree with me, I will try to make a patch to fix it in this way. Thank you.

[Mikhail Pershin]

the obd_fail is set to 1 by server_stop_servers() just before class_manual_cleanup() call. That is why we have clients in last_rcvd but no orphans.