https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13119 Lustre <p>This issue was created by maloo for Andreas Dilger &lt;adilger@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/f8038c56-3208-11ea-adca-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/f8038c56-3208-11ea-adca-52540065bddc</a></p> <p>lustre-initialization failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>'trevis-42vm12 crashed during lustre-initialization-1' </pre> </div></div> <p>The stack trace on the MDS looks like:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc BUG: unable to handle kernel NULL pointer dereference at 0000000000000078 IP: [&lt;ffffffff812d5995&gt;] common_file_perm+0x15/0x180 Oops: 0000 [#1] SMP Supported: No, Unsupported modules are loaded CPU: 0 PID: 2995 Comm: mount.lustre Tainted: 4.4.180-94.100_lustre Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: security_file_permission+0x3e/0xc0 iterate_dir+0x32/0x110 osd_ios_general_scan+0x12e/0x250 [osd_ldiskfs] osd_initial_OI_scrub+0x5e/0xc00 [osd_ldiskfs] osd_scrub_setup+0x8f5/0x960 [osd_ldiskfs] osd_device_alloc+0x5ac/0x8c0 [osd_ldiskfs] obd_setup+0xb8/0x230 [obdclass] class_setup+0x468/0x7c0 [obdclass] class_process_config+0x1890/0x27b0 [obdclass] do_lcfg+0x235/0x490 [obdclass] lustre_start_simple+0x85/0x1f0 [obdclass] server_fill_super+0xe81/0x1640 [obdclass] lustre_fill_super+0x436/0x8d0 [obdclass] mount_nodev+0x48/0xa0 mount_fs+0x3a/0x170 vfs_kern_mount+0x62/0x110 do_mount+0x213/0xcd0 SyS_mount+0x85/0xd0 </pre> </div></div> <p>It could be that this is related to <tt>iterate_dir()</tt> taking a fake <tt>filp</tt> as an argument, and somehow <tt>filp</tt> is not filled in sufficiently for <tt>security_file_permission()-&gt;common_file_perm()</tt>.</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> lustre-initialization lustre-initialization - 'trevis-42vm12 crashed during lustre-initialization-1'</p> LU-13119

lustre-initialization crashed in common_file_perm() on SLES12

Bug Minor Resolved Fixed James A Simmons Maloo Thu, 9 Jan 2020 05:42:05 +0000 Thu, 23 Jan 2020 13:43:03 +0000 Thu, 23 Jan 2020 13:43:03 +0000 Lustre 2.14.0 0 5 <p>It looks like the use of <tt>iterate_dir()</tt> was introduced by patch <a href="https://review.whamcloud.com/34714" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34714</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-11832" title="ARM servers crashing on MDS startup" class="issue-link" data-issue-key="LU-11832"><del>LU-11832</del></a> ldiskfs: properly handle VFS parallel locking</tt>".</p> <p>Which SLES is this?</p> <p>According to the data on the Maloo page it is SLES12.3.</p> <p>SUSE is using app amour which has different requirements. osd-ldiskfs open codes struct file creating instead of using </p> <p>alloc_file_pseudo() so bits are missed. Looking at common_file_perm() in the SUSE kernel code it expects </p> <p>file-&gt;f_cred to set. Eventually I like to move to alloc_file_pseudo() but that is a bit tricky in the way struct file data structures are handling in ldiskfs as scratch areas.</p> <p>James Simmons (jsimmons@infradead.org) uploaded a new patch: <a href="https://review.whamcloud.com/37184" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37184</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13119" title="lustre-initialization crashed in common_file_perm() on SLES12" class="issue-link" data-issue-key="LU-13119"><del>LU-13119</del></a> osd-ldiskfs: set f_cred for app armour<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 82d5ab9c64a290791be79b83ff006c926ddc0311</p> <p>It seems that the SLES kernel is trying to apply AppArmor security policies to the Lustre filesystem (not sure why, since we , since <tt>common_file_perm()</tt> only exists in <tt>security/apparmor/lsm.c</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">static</span> <span class="code-object">int</span> common_file_perm(<span class="code-object">int</span> op, struct file *file, u32 mask) { struct aa_file_cxt *fcxt = file-&gt;f_security; struct aa_profile *profile, *fprofile = aa_cred_profile(file-&gt;f_cred); <span class="code-object">int</span> error = 0; BUG_ON(!fprofile); <span class="code-keyword">if</span> (!file-&gt;f_path.mnt || !mediated_filesystem(file-&gt;f_path.dentry)) <span class="code-keyword">return</span> 0; profile = __aa_current_profile(); /* revalidate access, <span class="code-keyword">if</span> task is unconfined, or the cached cred * doesn't match or <span class="code-keyword">if</span> the request is <span class="code-keyword">for</span> more permissions than * was granted. * * Note: the test <span class="code-keyword">for</span> !unconfined(fprofile) is to handle file * delegation from unconfined tasks */ <span class="code-keyword">if</span> (!unconfined(profile) &amp;&amp; !unconfined(fprofile) &amp;&amp; ((fprofile != profile) || (mask &amp; ~fcxt-&gt;allow))) error = aa_file_perm(op, profile, file, mask); <span class="code-keyword">return</span> error; } <span class="code-keyword">static</span> <span class="code-object">int</span> apparmor_file_permission(struct file *file, <span class="code-object">int</span> mask) { <span class="code-keyword">return</span> common_file_perm(OP_FPERM, file, mask); } <span class="code-object">int</span> iterate_dir(struct file *file, struct dir_context *ctx) { struct inode *inode = file_inode(file); bool shared = <span class="code-keyword">false</span>; <span class="code-object">int</span> res = -ENOTDIR; <span class="code-keyword">if</span> (file-&gt;f_op-&gt;iterate_shared) shared = <span class="code-keyword">true</span>; <span class="code-keyword">else</span> <span class="code-keyword">if</span> (!file-&gt;f_op-&gt;iterate) <span class="code-keyword">goto</span> out; res = security_file_permission(file, MAY_READ); : : } osd_ios_general_scan(struct osd_thread_info *info, struct osd_device *dev, struct dentry *dentry, filldir_t filldir) { struct file *filp = &amp;info-&gt;oti_file; struct inode *inode = dentry-&gt;d_inode; <span class="code-keyword">const</span> struct file_operations *fops = inode-&gt;i_fop; filp-&gt;f_pos = 0; filp-&gt;f_path.dentry = dentry; filp-&gt;f_flags |= O_NOATIME; filp-&gt;f_mode = FMODE_64BITHASH | FMODE_NONOTIFY; filp-&gt;f_mapping = inode-&gt;i_mapping; filp-&gt;f_op = fops; filp-&gt;private_data = NULL; set_file_inode(filp, inode); rc = osd_security_file_alloc(filp); <span class="code-keyword">if</span> (rc) RETURN(rc); <span class="code-keyword">do</span> { buf.oifb_items = 0; rc = iterate_dir(filp, &amp;buf.ctx); } <span class="code-keyword">while</span> (rc &gt;= 0 &amp;&amp; buf.oifb_items &gt; 0 &amp;&amp; </pre> </div></div> <p>Sigh. I didn't commit my comment until now...</p> <p>Seen on Master. <a href="https://testing.whamcloud.com/test_sets/aedd7978-3a88-11ea-80b4-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/aedd7978-3a88-11ea-80b4-52540065bddc</a></p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37184/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37184/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13119" title="lustre-initialization crashed in common_file_perm() on SLES12" class="issue-link" data-issue-key="LU-13119"><del>LU-13119</del></a> osd-ldiskfs: set f_cred for app armour<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 33082e057d214793c70085a33f1d82b3915db3a9</p> <p>Landed for 2.14</p> Related LU-11832 Development Rank 1|i00rr3: Rank (Obsolete) 9223372036854775807 Severity