[LU-971] Re-sent open requests cause mod_count leakages

• Status: Closed
• Project: Lustre
• Affects Version/s: Lustre 2.2.0
• Fix Version/s: Lustre 2.2.0, Lustre 2.1.5
• Type: Bug
• Priority: Minor
• Reporter: Li Wei (Inactive)
• Assignee: Li Wei (Inactive)
• Resolution: Fixed
• Severity: 3

Description

This was first observed in ORI-454 and is reproducible with the following test on a 32-bit RHEL 6 VM:

test_0() {
    cp /etc/fstab $DIR/$tfile
    zconf_umount $HOSTNAME $MOUNT
    zconf_mount $HOSTNAME $MOUNT
    stat $DIR/$tfile

    drop_ldlm_reply "cat $DIR/$tfile >/dev/null"
    rm $DIR/$tfile

    $LCTL dk $TMP/tmp.dk
}
run_test 0 "open count leakages caused by re-sent opens"

From $TMP/tmp.dk, during the processing of the re-sent open:

00000004:00010000:0.0:1326099470.493963:1320:13852:0:(mdt_handler.c:3260:mdt_intent_fixup_resent()) @@@ no existing lock with rhandle 0xb3f7591c1bdff1aa  req@caaa8000 x1390515987677363/t0(0) o101->223c9642-c38f-70c2-828b-911bdf893616@0@lo:0/0 lens 472/0 e 0 to 0 dl 1326099476 ref 1 fl Interpret:/0/ffffffff rc 0/-1

The re-sent LDLM_ENQUEUE request was being handled. Here the MSG_RESENT flags was cleared, implying "!req_xid_is_last(req)". (On master, lcd_last_xid is updated only for requests that generate transactions.)

00000004:00000001:0.0:1326099470.494115:1376:13852:0:(mdt_open.c:743:mdt_finish_open()) Process entered
00000004:00000002:0.0:1326099470.494116:1468:13852:0:(mdt_handler.c:454:mdt_pack_attr2body()) [0x200000400:0x1:0x0]: nlink=1, mode=100644, size=0
00000004:00000001:0.0:1326099470.494118:1452:13852:0:(cmm_object.c:273:cml_xattr_get()) Process entered
00000004:00000001:0.0:1326099470.494119:1552:13852:0:(mdd_object.c:820:mdd_xattr_get()) Process entered
00000004:00000001:0.0:1326099470.494121:1552:13852:0:(mdd_object.c:829:mdd_xattr_get()) Process leaving (rc=4294967235 : -61 : ffffffc3)
00000004:00000001:0.0:1326099470.494121:1452:13852:0:(cmm_object.c:275:cml_xattr_get()) Process leaving (rc=4294967235 : -61 : ffffffc3)
00000004:00000001:0.0:1326099470.494123:1508:13852:0:(mdt_open.c:605:mdt_mfd_open()) Process entered

mdt_mfd_open() was called for this re-sent request. Then during the processing of the unlink:

00000004:00080000:0.0:1326099470.505406:1188:13853:0:(mdd_dir.c:800:mdd_finish_unlink()) Object [0x200000400:0x1:0x0] is inserted into orphan list, open count = 1

Issue Links

• is duplicated by: LU-2277 Text file busy

Comments

[Mikhail Pershin]

I've found the same issue while working on LU-80 dynamic buffer patch. There is possible fix:
http://review.whamcloud.com/#change,1930

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,ofa #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el5,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,ofa #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,sles11,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el5,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,ofa #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,ofa #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el5,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el5,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,server,el6,inkernel #476
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = SUCCESS
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,server,el6,ofa #480
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = FAILURE
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » x86_64,client,el6,ofa #480
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = FAILURE
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,ofa #480
LU-971 mdt: fix open resent issue with last_xid (Revision ed5e47ac21b02abb16b10446dac7dae722259f0a)

Result = ABORTED
Oleg Drokin : ed5e47ac21b02abb16b10446dac7dae722259f0a
Files :

• lustre/mdt/mdt_open.c

[Li Wei (Inactive)]

Fixed by Mike's patch.