https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2336 Lustre <p>I noticed that our MDS logs this message once in while:<br/> LustreError: 3354:0:(llog_cat.c:298:llog_cat_add_rec()) llog_write_rec -28: lh=ffff8809f8de6780<br/> LustreError: 20199:0:(llog_cat.c:298:llog_cat_add_rec()) llog_write_rec -28: lh=ffff8809c6b299c0<br/> LustreError: 32022:0:(llog_cat.c:298:llog_cat_add_rec()) llog_write_rec -28: lh=ffff8804dd524d80<br/> LustreError: 32015:0:(llog_cat.c:298:llog_cat_add_rec()) llog_write_rec -28: lh=ffff8804dd76af00</p> <br/> [<a href='mailto:root@n-mds1'>root@n-mds1</a> ~]# cat /proc/fs/lustre/version <br/> lustre: 2.2.0 <br/> kernel: patchless_client <br/> build: 2.2.0-RC2--PRISTINE-2.6.32-220.4.2.el6_lustre.x86_64 <br/> <br/> [<a href='mailto:root@n-mds1'>root@n-mds1</a> ~]# uname -r <br/> 2.6.32-220.4.2.el6_lustre.x86_64 <br/> <br/> [<a href='mailto:root@n-mds1'>root@n-mds1</a> ~]# rpm -qa|grep lustre <br/> lustre-ldiskfs-3.3.0-2.6.32_220.4.2.el6_lustre.x86_64.x86_64 <br/> lustre-2.2.0-2.6.32_220.4.2.el6_lustre.x86_64.x86_64 <br/> kernel-firmware-2.6.32-220.4.2.el6_lustre.x86_64 <br/> lustre-modules-2.2.0-2.6.32_220.4.2.el6_lustre.x86_64.x86_64 <br/> kernel-headers-2.6.32-220.4.2.el6_lustre.x86_64 <br/> kernel-2.6.32-220.4.2.el6_lustre.x86_64 <br/> kernel-devel-2.6.32-220.4.2.el6_lustre.x86_64 LU-2336

mds llog_write_rec 'No space left on device'

Bug Major Resolved Fixed Hongchao Zhang ETHz Support metadata mn1 shh Thu, 15 Nov 2012 10:07:28 +0000 Sat, 15 Mar 2014 21:01:47 +0000 Sat, 8 Mar 2014 01:22:32 +0000 Lustre 2.2.0 Lustre 2.1.2 Lustre 2.4.0 0 8 <p>Could be a problem? This is not related with the mds crashes that I have described in the <a href="https://jira.whamcloud.com/browse/LU-2323" title="mds crash" class="issue-link" data-issue-key="LU-2323"><del>LU-2323</del></a> issue.</p> <p>thanks in advance</p> <p>Hongchao</p> <p>Could you please comment on this one?</p> <p>Thanks</p> <p>Peter</p> <p>no, this should be not a problem, the PIDs in these logs are different, a new llog file will be created if the current one<br/> is full, and if there are two logs with same PID in a row, then it could indicate there is some problem. the related codes is,</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-comment">/* now let's <span class="code-keyword">try</span> to add the record */</span> rc = llog_write_rec(env, loghandle, rec, reccookie, 1, buf, -1, th); <span class="code-keyword">if</span> (rc &lt; 0) CERROR(<span class="code-quote">"llog_write_rec %d: lh=%p\n"</span>, rc, loghandle); cfs_up_write(&amp;loghandle-&gt;lgh_lock); <span class="code-keyword">if</span> (rc == -ENOSPC) { <span class="code-comment">/* <span class="code-keyword">try</span> to use next log */</span> loghandle = llog_cat_current_log(cathandle, th); LASSERT(!IS_ERR(loghandle)); <span class="code-comment">/* <span class="code-keyword">new</span> llog can be created concurrently */</span> <span class="code-keyword">if</span> (!llog_exist(loghandle)) { rc = llog_cat_new_log(env, cathandle, loghandle, th); <span class="code-keyword">if</span> (rc &lt; 0) { cfs_up_write(&amp;loghandle-&gt;lgh_lock); RETURN(rc); } } <span class="code-comment">/* now let's <span class="code-keyword">try</span> to add the record */</span> rc = llog_write_rec(env, loghandle, rec, reccookie, 1, buf, -1, th); <span class="code-keyword">if</span> (rc &lt; 0) CERROR(<span class="code-quote">"llog_write_rec %d: lh=%p\n"</span>, rc, loghandle); cfs_up_write(&amp;loghandle-&gt;lgh_lock); } </pre> </div></div> <p>We're seeing this in production here at LLNL, and it is confusing our admins. If the message is harmless, it really should be removed (or at least masked behind a CDEBUG flag).</p> <p>the patch is tracked at <a href="http://review.whamcloud.com/#change,5146" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5146</a></p> <p>With Change, 5146 landed can this ticket be closed?</p> <p>Patch provided and landed.</p> Development Epic metadata Rank 1|hzvcbr: Rank (Obsolete) 5568 Severity