https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9484 Lustre <p>sanity test_17k fails when a default composite layout is set for the Lustre mount point with</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sanity test_17k: @@@@@@ FAIL: rsync failed with xattrs enabled </pre> </div></div> <p>From the test log, we can see lsetxattr() in rsync is failing</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanity test 17k: symlinks: rsync with xattrs enabled ============================================== 03:57:01 (1494388621) striped dir -i1 -c2 /mnt/lustre/d17k.sanity striped dir -i1 -c2 /mnt/lustre/d17k.sanity.new sending incremental file list ./ f17k.sanity f17k.sanity.lnk -&gt; /mnt/lustre/d17k.sanity/f17k.sanity rsync: rsync_xal_set: lsetxattr(""/mnt/lustre/d17k.sanity.new/.f17k.sanity.qR0FHg"","lustre.lov") failed: File exists (17) rsync: rsync_xal_set: lsetxattr(""/mnt/lustre/d17k.sanity.new/.f17k.sanity.qR0FHg"","trusted.lov") failed: File exists (17) sent 1628 bytes received 48 bytes 1117.33 bytes/sec total size is 35 speedup is 0.02 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9] sanity test_17k: @@@@@@ FAIL: rsync failed with xattrs enabled </pre> </div></div> <p>Logs for recent failures are at:<br/> <a href="https://testing.hpdd.intel.com/test_sets/f5d085aa-357c-11e7-b0a8-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f5d085aa-357c-11e7-b0a8-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/2e962ed8-3534-11e7-814a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/2e962ed8-3534-11e7-814a-5254006e85c2</a></p> LU-9484

sanity test 17k fails with 'rsync failed with xattrs enabled'

Technical task LU-9349 Critical Resolved Fixed Zhenyu Xu James Nunez pfl Wed, 10 May 2017 15:07:27 +0000 Mon, 9 Oct 2017 22:15:46 +0000 Sat, 10 Jun 2017 12:40:27 +0000 Lustre 2.10.0 Lustre 2.11.0 Lustre 2.10.0 0 8 <p>Hi Bobijam,</p> <p>Can you add your thoughts here that we discussed 1:1 today?</p> <p>Thanks.<br/> Joe</p> <p>Hi Andreas,</p> <p>I uploaded two strace output of the rsync command, the difference is that for plain.txt, /mnt/lustre/d17k.sanity/f17k.sanity is created as a plain file, while pfl.txt corresponding to a PFL file.</p> <p>The difference of the strace shows that with PFL file, rsync tries to lsetxattr a temporary file, while lsetxattr does not show up for the plain file.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedHeader panelHeader" style="border-bottom-width: 1px;"><b>rsync FPL file</b></div><div class="preformattedContent panelContent"> <pre>read(5, "rsync: rsync_xal_set: lsetxattr("..., 93) = 93 write(2, "rsync: rsync_xal_set: lsetxattr("..., 92rsync: rsync_xal_set: lsetxattr(".f17k.sanity.6HKfyd","lustre.lov") failed: File exists (17)) = 92 </pre> </div></div> <p>I don't know what causes it, could it be that the xattr of PFL is too big that cause rsync tries to use a temporary file? I could not find where I can get the rsync code to have a look.</p> <p>The temporary file is always being used by rsync, so I don't think that is different. The main issue is that the strace needs to be run with "-f" to trace all the threads across fork, since it looks like the logs are incomplete (they don't show any file creates, etc)? The "<tt>lsetxattr()</tt>" that shows up in the "pfl" log is only from when it is calling <tt>write()</tt> to print the message to the console, not the actual syscall to <tt>lsetxattr()</tt>. I think both logs are missing the actual operations to the target files.</p> <p>That said, setting <tt>trusted.lov</tt> and <tt>lustre.lov</tt> should be accepted from userspace even if they already exist, since they will normally be created when the file is first opened, and then will also be "restored" by rsync (or tar or cp) after the file is written (see comments in <tt>ll_setstripe_ea_info()</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> /** * b=10667: ignore error. * Silently eat error on setting strusted.lov attribute <span class="code-keyword">for</span> * SuSE 9, it added <span class="code-keyword">default</span> option to copy all attributes in * <span class="code-quote">'cp'</span> command. */ </pre> </div></div> <p>I also see in that function:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> (lump != NULL &amp;&amp; lump-&gt;lmm_magic == LOV_USER_MAGIC_COMP_V1) { return_err = <span class="code-keyword">true</span>; <span class="code-keyword">goto</span> setstripe; } </pre> </div></div> <p>so this is what is causing PFL to fail in the rsync case and not the plain file case. Is there a reason that the PFL code needs to return an error in this case? This came from <a href="https://review.whamcloud.com/24851" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/24851</a> so you may want to look at the commit history of the master or pfl branches versions of this patch to see why <tt>setxattr("trusted.lov")</tt> needs to return an error to userspace. </p> <p>PS: I think this is pretty critical, since it will totally break rsync, cp, tar, or anything that is trying to restore the <tt>lustre.lov</tt> or <tt>trusted.lov</tt> xattrs.</p> <p>Bobi Jam (bobijam@hotmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/27126" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27126</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9484" title="sanity test 17k fails with &#39;rsync failed with xattrs enabled&#39; " class="issue-link" data-issue-key="LU-9484"><del>LU-9484</del></a> llite: eat error on setting trunsted.lov<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: bd9956f9190c1b0aca37c47802e44a3c82eba9a6</p> <p>Yes, you are right, strace -f shows that in all cases, rsync creates temporary file.</p> <p>Andreas Dilger (andreas.dilger@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/27311" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27311</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9484" title="sanity test 17k fails with &#39;rsync failed with xattrs enabled&#39; " class="issue-link" data-issue-key="LU-9484"><del>LU-9484</del></a> llite: eat -EEXIST on setting trusted.lov<br/> Project: fs/lustre-release<br/> Branch: pfl<br/> Current Patch Set: 1<br/> Commit: 44bbbfed3513798638e01f1ae2447faf80f7c15e</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/27126/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27126/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9484" title="sanity test 17k fails with &#39;rsync failed with xattrs enabled&#39; " class="issue-link" data-issue-key="LU-9484"><del>LU-9484</del></a> llite: eat -EEXIST on setting trunsted.lov<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0e90e02ceddd60f24fac9709f3ab9e9421c80315</p> <p>I see this landed already. I was testing this with my <a href="https://jira.whamcloud.com/browse/LU-9183" title="Support for linux 4.9 kernels" class="issue-link" data-issue-key="LU-9183"><del>LU-9183</del></a> xattr patch and I was seeing sanity 102a and 102n both failing. I thought it was due to my latest <a href="https://jira.whamcloud.com/browse/LU-9183" title="Support for linux 4.9 kernels" class="issue-link" data-issue-key="LU-9183"><del>LU-9183</del></a> but I traced it down to this patch. I'm looking to see what the fix will be and I place the fix in patch <a href="https://review.whamcloud.com/#/c/27240" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/27240</a>. Strange it passed testing in maloo.</p> <p>Landed for 2.10</p> <p>Never mind. I see the sanity.sh test were updated and my test suite was missing those changes. It does work.</p> Related LU-9335 Development Rank 1|hzzcd3: Rank (Obsolete) 9223372036854775807