https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-767 Lustre <p>We experienced an issue where an OSS took over the resources of it's cluster partner and then initiated a STONITH event and reset the partner without any notification that the event was going to happen. According to our log files for the network, the MDS, and the OSS', there appeared to be nothing wrong with the OSS that was reset. We are trying to determine why this may have happened and would like to request any assistance you may be able to provide. We have the ha.cf file configured as follows sending unicast packets only between the cluster partners:</p> <p>keepalive 6<br/> warntime 30<br/> deadtime 90<br/> initdead 180</p> <p>The keepalives are being sent through two interfaces. One is through an Infiniband switch and the other is a direct ethernet connection using a crossover cable between the two devices.</p> <p>Would upgrading to the most current Linux-HA version possibly remediate this issue or do you think it would cause other issues due to the current Lustre and Linux OS versions we are using? </p> <p>Any assistance you can provide would be greatly appreciated. Thank you.</p> Lustre 1.8.0.1 running on Red Hat Linux Enterprise 5.3 using Linux High Availability Heartbeat version 2.1.4-4.1 LU-767

OSS takes over all resources and STONITHs it's cluster partner without any warning

Bug Minor Resolved Fixed Cliff White David Martin Mon, 17 Oct 2011 12:43:50 +0000 Thu, 16 Feb 2012 14:48:06 +0000 Thu, 16 Feb 2012 14:48:06 +0000 0 2 <p>Cliff</p> <p>Could you please advise on this one?</p> <p>Thanks</p> <p>Peter</p> <p>Thanks Peter!</p> <p>The current Linux-HA should be fine. Lustre requires nothing especially fancy from failover, we work with anything that can manage a Filesystem resource.</p> <ul class="alternate" type="square"> <li>Did you have any other monitoring setup beyond the Heartbeat pinger?</li> <li>Did you check the linux-ha logs? There should be an indication there as to what happened.</li> </ul> <p>Cliff,</p> <p>We do not have any other monitoring setup beyond Heartbeat.</p> <p>We did check the linux-ha logs and there was no indication of what happened. However, I'm not so sure we have our logging level set that would provide a verbose enough log. Do you have a recommendation as to what level we should be logging?</p> <p>Dave</p> <p>I used to use level 3 as default for debug logs, basically I would tail -f the log file, and adjust according to how much goop is spewed.<br/> How often is this failover occurring? Any other events going on at that time? Any indication of a network hiccup from other nodes?<br/> If Heartbeat pinger is the only monitoring, than either pinger failed (which should put something in logs) or somehow a takeover was ordered (which should also show in logs) <br/> Have you checked the system log on the node issuing the STONITH? There should be something there.</p> <p>What is your current state? Is this still an issue?</p> <p>Cliff,</p> <p>We have not had a re-occurrence of this issue. We believe it may be a stability issue with 1.8.0.1 and RedHat 5.3. We are looking at upgrading to 1.8.6.</p> <p>Thanks.<br/> Dave</p> <p>Okay, I will close this issue, please re-open if you have further information.</p> Development Rank 1|hzvhuv: Rank (Obsolete) 6542 Severity