https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2300 Lustre <p>1.8 client on 2.1 server:<br/> &gt; mkdir 161<br/> &gt; lfs setstripe -c 161 161<br/> error: bad stripe count 161: Invalid argument (22)<br/> error: setstripe: create stripe file '161' failed</p> <p>Expectation #1: stripe count of "-1" gives a stripe width of 160 since that's all 1.8 can support. On large 1.8.x Lustre clusters -1 gives you 160 (without error, silent or otherwise) even if you have more OSTs.</p> <p>Expectation #2: stripe count of "-1" gives a stripe width of "all OSTs", regardless of whether some clients can understand it.</p> <p>Expectation #3: stripe count of "-1" gives a stripe count of 180 iff there are any clients &lt; 2.1</p> <p>Maybe we should have a MDS conf_param setting which is the "-1" stripe count value?</p> LU-2300

Wide striping leads to confusion for old clients

Improvement Minor Resolved Won't Fix WC Triage Nathan Rutman Wed, 7 Nov 2012 15:13:03 +0000 Wed, 6 Nov 2013 17:56:54 +0000 Wed, 6 Nov 2013 17:56:53 +0000 0 9 <p>Nathan, could you clarify your expectations more? I'm not sure which of the expectations you are advocating for, and it conflates "-c </p> {&gt; 160} <p>" with "-c -1" (which could mean different things).</p> <p>In #3, is 180 the number of OSTs in the system, or is this a typo and you meant 160? Also, when you write "stripe count -1", do you really mean "any number larger than 160"?</p> <p>I tried "lfs setstripe -c 161" on my local filesystem and it created a file with 5 stripes (== $OSTCOUNT).</p> <blockquote><p>I'm not sure which of the expectations you are advocating for</p></blockquote> <p>I'm not sure either <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <blockquote><p>"-c {&gt; 160}" with "-c -1" </p></blockquote> <p>I'm specifically referring to -1 = "all available OSTs"</p> <p>I kind of think #2 makes the most sense, but it leads to confusing errors on 1.8 clients: "Why did it let me set that if it can't use it?"</p> <blockquote><p>typo and you meant 160?</p></blockquote> <p>Yes. Our system in the description has more than 160 OSTs.</p> <p>The error message was confusing to our customer, and it's not clear to me what the appropriate response should actually be. </p> <p>Nathan, I think I'm open to any reasonable interpretation of usage. I recall from Oleg's early wide striping testing that even though the 1.8 MDT cannot store layouts with more than 160 stripes that the clients can handle more that this, possibly in the neighborhood of 250 stripes. </p> <p>It definitely makes sense that 1.8 clients should not create files that they cannot use. That means the MDS should limit the layout for clients. </p> <p>Do you think that the presence of 1.8 clients should limit the layout usable by 2.x clients?</p> <blockquote><p>It definitely makes sense that 1.8 clients should not create files that they cannot use. That means the MDS should limit the layout for clients.</p></blockquote> <p>It means something should. Maybe the best answer is to change lfs on the clients to explicitly change -1 to 160 before submitting?</p> <blockquote><p>Do you think that the presence of 1.8 clients should limit the layout usable by 2.x clients?</p></blockquote> <p>I don't really like that, but on the other hand someone that creates a 161+ stripe file on a 2.x client and can't access it on a 18 client may be equally confused. If we do do this, then there are more questions:</p> <ul> <li>what if a 1.8 client connects "after" a 161-stripe file is created?</li> <li>how long do we wait since the "last" 1.8 client was seen before allowing a 161+ file?</li> </ul> <p>Perhaps the simplest policy is</p> <ul> <li>MDS changes a 1.8 client -1 stripe count to min(160, ost_count)</li> <li>MDS leaves 2.x client -1 alone</li> <li>1.8 clients trying to read 161+ files get EINVAL return code from MDS</li> </ul> <p>I agree with Nathan's suggestions &#8211; 1.8 client stripe count -1 should be min(160, ost_count); 2.x clients -1 behaves as today; 1.8 clients fail on accessing files w/&gt;160 stripes. I expect we'd see user complaints on the last item. Maybe a server-side option to allow a 2.x MDS to set -1 stripe count default to 160 (admin can then choose to set that if their environment includes 1.8 clients)?</p> <p>I'm largely in agreement with having the MDS enforce the limits. I don't think it is practical to change the 1.8 clients at this point. </p> <p>However, it should be noted that 1.8 clients can actually handle more than 160 stripes. Something like 250 or so - I suspect it will be reported by Oleg in the original bug (b=4224, IIRC). I believe 1.8 clients (or possibly the MDS) will already report an error if the reply buffer is too small. </p> <p>Xyratex <a href="http://jira-nss.xy01.xyratex.com:8080/browse/CLSTR-730" class="external-link" target="_blank" rel="nofollow noopener">CLSTR-730</a></p> <p>Ok, we'll work on a patch for this as I outlined above. (Andreas - testing shows anything above 160 returns an error.)</p> <p>I don't think 1.8 interop on systems with &gt; 160 OSTs is a huge concern anymore, and no patch is available to fix this. Closing this old bug.</p> Development Rank 1|hzvbwf: Rank (Obsolete) 5493