[LU-7582] Manual delete of oi directory in ZFS backed server causes panic

• Status: Resolved
• Project: Lustre
• Type: Bug
• Priority: Major
• Reporter: Nathaniel Clark
• Assignee: Jinshan Xiong (Inactive)
• Resolution: Not a Bug
• Labels: zfs

Environment

lustre-2.7.64
zfs-0.6.5.3-1

• Severity: 3

Description

When trying to manually delete an oi directory after manually mounting zfs filesystem created with mkfs.lustre. The following panic happens:

PANIC: zfs: link count on 43 is 1, should be at least 2
Showing stack for process 9291
Pid: 9291, comm: rmdir Tainted: P           -- ------------    2.6.32-573.12.1.el6.x86_64 #1
Call Trace:
 [<ffffffffa0237f5d>] ? spl_dumpstack+0x3d/0x40 [spl]
 [<ffffffffa0237fed>] ? vcmn_err+0x8d/0xf0 [spl]
 [<ffffffffa0299520>] ? dbuf_rele+0x40/0x50 [zfs]
 [<ffffffffa029953e>] ? dmu_buf_rele+0xe/0x10 [zfs]
 [<ffffffffa030893a>] ? zap_put_leaf+0x4a/0xc0 [zfs]
 [<ffffffffa0299520>] ? dbuf_rele+0x40/0x50 [zfs]
 [<ffffffffa029953e>] ? dmu_buf_rele+0xe/0x10 [zfs]
 [<ffffffffa030ccca>] ? zap_unlockdir+0x4a/0xc0 [zfs]
 [<ffffffffa02f1cf2>] ? zfs_panic_recover+0x52/0x60 [zfs]
 [<ffffffffa0317e00>] ? zfs_link_destroy+0x4f0/0x570 [zfs]
 [<ffffffffa02f6b65>] ? txg_rele_to_quiesce+0x35/0x50 [zfs]
 [<ffffffffa02b1f0e>] ? dmu_tx_assign+0x41e/0x570 [zfs]
 [<ffffffffa03350b6>] ? zfs_rmdir+0x546/0x650 [zfs]
 [<ffffffff811484a5>] ? vma_prio_tree_add+0x75/0xd0
 [<ffffffffa034a095>] ? zpl_rmdir+0x65/0xa0 [zfs]
 [<ffffffff811a0abe>] ? vfs_rmdir+0xbe/0xf0
 [<ffffffff8119fbca>] ? lookup_hash+0x3a/0x50
 [<ffffffff811a3cd4>] ? do_rmdir+0x184/0x1f0
 [<ffffffff810e8a57>] ? audit_syscall_entry+0x1d7/0x200
 [<ffffffff811a3d96>] ? sys_rmdir+0x16/0x20
 [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b

The following commands were used:

# zfs import tank
# mkfs.lustre --mgs --mdt --index=0 --backfstype=zfs --fsname lustre --reformat tank/foo
# mount -t lustre tank/foo /mnt/MGS/
# umount /mnt/MGS
# zfs set canmount=on tank/foo
# zfs mount tank/foo
# ls /tank/foo/oi.36/
# cd /tank/foo
# rmdir oi.36
Message from syslogd@ieel-oss07 at Dec 18 14:43:50 ...
 kernel:PANIC: zfs: link count on 43 is 1, should be at least 2

Issue Links

• is related to: LU-7585 Implement OI Scrub for ZFS

Comments

[Nathaniel Clark]

ZFS 0.6.3-1.1 exhibits the same behaviour.

[Peter Jones]

Jinshan

Could you please comment?

Thanks

Peter

[Jinshan Xiong (Inactive)]

This is not a bug. OSD-ZFS uses zap to map FID to ZFS object ID, but it doesn't actually take a reference to the objects.

[Andreas Dilger]

The disconnect here is that the OI file looks like a directory to ZPL, but it isn't really a directory.

It is also worthwhile to note that until we implement ZFS OI Scrub (LU-7585) it isn't safe to delete the OI files. There is no way to rebuild them.