[LU-6701] possible lock inversion during mkfs

Duplicate — Tue, 9 Jun 2015 18:05:10 +0000

Got this while testing with a debug kernel:

Jun  8 11:35:52 mds kernel: [ INFO: possible circular locking dependency detected ]
Jun  8 11:35:52 mds kernel: 3.10.0-229.4.2.el7.centos.x86_64.debug #1 Tainted: GF          O--------------  
Jun  8 11:35:52 mds kernel: -------------------------------------------------------
Jun  8 11:35:52 mds kernel: mkfs.lustre/31035 is trying to acquire lock:
Jun  8 11:35:52 mds kernel: (&s->s_dquot.dqio_mutex){+.+...}, at: [<ffffffff812807eb>] dquot_commit+0x2b/0xc0
Jun  8 11:35:52 mds kernel: 
but task is already holding lock:
Jun  8 11:35:52 mds kernel: (&ei->i_data_sem){++++..}, at: [<ffffffffa056f0a4>] ldiskfs_map_blocks+0x144/0x590 [ldiskfs]
Jun  8 11:35:52 mds kernel: 
which lock already depends on the new lock.

Jun  8 11:35:52 mds kernel: 
the existing dependency chain (in reverse order) is:
Jun  8 11:35:52 mds kernel: 
-> #1 (&ei->i_data_sem){++++..}:
Jun  8 11:35:52 mds kernel:       [<ffffffff810fd91f>] validate_chain.isra.43+0x49f/0x910
Jun  8 11:35:52 mds kernel:       [<ffffffff810fec26>] __lock_acquire+0x3c6/0xb60
Jun  8 11:35:52 mds kernel:       [<ffffffff810ffb99>] lock_acquire+0x99/0x1e0
Jun  8 11:35:52 mds kernel:       [<ffffffff816cd691>] down_read+0x51/0xa0
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f2a4>] ldiskfs_map_blocks+0x344/0x590 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f555>] ldiskfs_getblk+0x65/0x210 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f727>] ldiskfs_bread+0x27/0xe0 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa058114e>] ldiskfs_quota_read+0xee/0x150 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffff812848dc>] read_blk+0x4c/0x60
Jun  8 11:35:52 mds kernel:       [<ffffffff81285767>] find_tree_dqentry+0x47/0x240
Jun  8 11:35:52 mds kernel:       [<ffffffff812858ca>] find_tree_dqentry+0x1aa/0x240
Jun  8 11:35:52 mds kernel:       [<ffffffff812858ca>] find_tree_dqentry+0x1aa/0x240
Jun  8 11:35:52 mds kernel:       [<ffffffff812858ca>] find_tree_dqentry+0x1aa/0x240
Jun  8 11:35:52 mds kernel:       [<ffffffff81285aac>] qtree_read_dquot+0x14c/0x380
Jun  8 11:35:52 mds kernel:       [<ffffffff81283f0b>] v2_read_dquot+0x2b/0x30
Jun  8 11:35:52 mds kernel:       [<ffffffff8127f9b4>] dquot_acquire+0xf4/0x140
Jun  8 11:35:52 mds kernel:       [<ffffffffa0580656>] ldiskfs_acquire_dquot+0x66/0xb0 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffff81280fff>] dqget+0x3ef/0x450
Jun  8 11:35:52 mds kernel:       [<ffffffff8128110d>] __dquot_initialize+0xad/0x1c0
Jun  8 11:35:52 mds kernel:       [<ffffffff81281233>] dquot_initialize+0x13/0x20
Jun  8 11:35:52 mds kernel:       [<ffffffffa0567b5c>] ldiskfs_mkdir+0x6c/0x280 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffff81222d47>] vfs_mkdir+0xb7/0x160
Jun  8 11:35:52 mds kernel:       [<ffffffff8122965f>] SyS_mkdirat+0x6f/0xe0
Jun  8 11:35:52 mds kernel:       [<ffffffff812296e9>] SyS_mkdir+0x19/0x20
Jun  8 11:35:52 mds kernel:       [<ffffffff816dace9>] system_call_fastpath+0x16/0x1b
Jun  8 11:35:52 mds kernel: 
-> #0 (&s->s_dquot.dqio_mutex){+.+...}:
Jun  8 11:35:52 mds kernel:       [<ffffffff810fd477>] check_prevs_add+0x987/0x990
Jun  8 11:35:52 mds kernel:       [<ffffffff810fd91f>] validate_chain.isra.43+0x49f/0x910
Jun  8 11:35:52 mds kernel:       [<ffffffff810fec26>] __lock_acquire+0x3c6/0xb60
Jun  8 11:35:52 mds kernel:       [<ffffffff810ffb99>] lock_acquire+0x99/0x1e0
Jun  8 11:35:52 mds kernel:       [<ffffffff816ccb29>] mutex_lock_nested+0x99/0x520
Jun  8 11:35:52 mds kernel:       [<ffffffff812807eb>] dquot_commit+0x2b/0xc0
Jun  8 11:35:52 mds kernel:       [<ffffffffa058070c>] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa058076f>] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffff81283c11>] __dquot_alloc_space+0x191/0x260
Jun  8 11:35:52 mds kernel:       [<ffffffffa05a13f1>] ldiskfs_mb_new_blocks+0x101/0x8c0 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa0554b59>] ldiskfs_alloc_branch+0x3c9/0x440 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa0555837>] ldiskfs_ind_map_blocks+0x207/0x920 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f1e5>] ldiskfs_map_blocks+0x285/0x590 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f555>] ldiskfs_getblk+0x65/0x210 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056f727>] ldiskfs_bread+0x27/0xe0 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa0560b61>] ldiskfs_append+0x81/0x150 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa056792f>] ldiskfs_init_new_dir+0xcf/0x230 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffffa0567c74>] ldiskfs_mkdir+0x184/0x280 [ldiskfs]
Jun  8 11:35:52 mds kernel:       [<ffffffff81222d47>] vfs_mkdir+0xb7/0x160
Jun  8 11:35:52 mds kernel:       [<ffffffff8122965f>] SyS_mkdirat+0x6f/0xe0
Jun  8 11:35:52 mds kernel:       [<ffffffff812296e9>] SyS_mkdir+0x19/0x20
Jun  8 11:35:52 mds kernel:       [<ffffffff816dace9>] system_call_fastpath+0x16/0x1b
Jun  8 11:35:52 mds kernel:

In the first trace, dquot_acquire takes s->s_dquot.dqio_mutex, and later ldiskfs_map_blocks is called, which takes ei->i_data_sem.

In the second trace, it looks like ldiskfs_map_blocks takes ei->i_data_sem and later calls dquot_commit, which takes s->s_dquot.dqio_mutex.

I'm not sure if this is a real problem, but I thought I'd report it in case anyone else ever sees these traces. It's possibly related to this false positive?:
http://permalink.gmane.org/gmane.comp.file-systems.ext4/16995

Whamcloud Community JIRA

[LU-6701] possible lock inversion during mkfs