https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11906 Lustre <p>when trying to run any Lustre tests with bash as the shell (or via auster) and ost_HOST and/or mds_HOST set to a different node, the scripts fail in yaml.sh line 11:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/home/green/git/lustre-release/lustre/tests/yaml.sh: line 11: $logdir/node.$host.yml: ambiguous redirect </pre> </div></div> <p>The line in question is</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> echo <span class="code-quote">"$line"</span> | sed <span class="code-quote">"s/^${host}: <span class="code-comment">//"</span> | sed <span class="code-quote">"s/^${host}://"</span> \ </span> &gt;&gt; $logdir/node.$host.yml; </pre> </div></div> <p>if I put quotes around that "$logdir/node.$host.yml" - the error goes away and everything works as intended.</p> <p>I guess this is the proper way to fix it?</p> <p>Also this error does not show up if I use "sh" instead of "bash" to run the scripts, but I must use bash as otherwise flakey logic does not work.</p> LU-11906

"ambiguous redirect" from yml.sh when running in multinode setup

Bug Major Open Unresolved WC Triage Oleg Drokin Wed, 30 Jan 2019 21:57:55 +0000 Thu, 31 Jan 2019 02:05:52 +0000 Lustre 2.12.0 Lustre 2.13.0 0 3 <p>also the reason for the warning is because the yaml file does not exist on the server node at the time this is executed.</p> <p>It looks like the code in question is:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> split_output() { <span class="code-keyword">while</span> read line; <span class="code-keyword">do</span> host=${line%%:*}; echo <span class="code-quote">"$line"</span> | sed <span class="code-quote">"s/^${host}: <span class="code-comment">//"</span> | sed <span class="code-quote">"s/^${host}://"</span> \ </span> &gt;&gt; $logdir/node.$host.yml; done } </pre> </div></div> <p>so it is reading "<tt>$line</tt>" from the standard input, then dropping everything after the first ":" to find "<tt>$host</tt>", so it is expecting output from <tt>pdsh</tt> or similar.</p> <p>What is in "<tt>$host</tt>" really depends on what is in "<tt>$line</tt>" when the function is called. If there is a space before the first ":" then "<tt>$host</tt>" will have a space in it, and the redirect will be "<tt>&gt;&gt; $logdir/node.foo bar.yml</tt>" or similar, and it means there are two possible output files, which is of course not allowed. Quoting the output means you'll have an output filename with a space in it, which is probably why the error goes away.</p> <p>Ok, something really strange is going on here. I added printing of the host and that's what I got:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> &gt;&gt; <span class="code-quote">"$logdir/node.$host.yml"</span>; echo we got host <span class="code-quote">"*${host}*"</span> &gt;&amp;2 </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>oleg1-server.localnet: executing yml_node we got host *Build* we got host *Build* we got host *lbats_build_id* we got host *lbats_build_name* we got host *architecture* we got host *os* we got host *os_distribution* we got host *lustre_version* we got host *lustre_build* we got host *lustre_branch* we got host *lustre_revision* we got host *kernel_version* we got host *file_system* we got host ** we got host *Node* we got host *lbats_build_id* we got host *lbats_build_name* we got host *architecture* we got host *os* we got host *os_distribution* we got host *lustre_version* we got host *lustre_build* we got host *lustre_branch* we got host *lustre_revision* we got host *kernel_version* we got host *file_system* we got host ** we got host *Node* we got host *node_name* we got host *mem_size* we got host *architecture* we got host *networks* we got host *node_name* we got host *mem_size* we got host *architecture* we got host *networks* we got host *- tcp* we got host ** we got host *LustreEntities* we got host *- tcp* we got host ** we got host *LustreEntities* </pre> </div></div> <p>Ok, so after some digging with JJames, it looks like the code depends on pdsh not being invoked with the -N option that supresses the host: prefix. I do run it with -N and so the code is utterly confusing. Not sure when did that change came to be because in the past -N was mandatory and in fact there are still checks for it in test-framework.sh</p> <p>As Oleg said, the issue here was due to using the ‘-N’ option to pdsh. The problem is that some routines in yaml.sh assume that all output from do_rpc_nodes() has the host name prepended to each line. Of course, using pdsh/ssh with the ‘-N’ option removes the hostname from each line. In yaml.sh, yml_nodes_file() sends data to split_output() that is assumed to have the host name in every line parsed </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> 8 split_output() { 9 <span class="code-keyword">while</span> read line; <span class="code-keyword">do</span> 10 host=${line%%:*}; 11 echo <span class="code-quote">"$line"</span> | sed <span class="code-quote">"s/^${host}: <span class="code-comment">//"</span> | sed <span class="code-quote">"s/^${host}://"</span> \ </span> 12 &gt;&gt; $logdir/node.$host.yml; 13 done 14 } 15 16 yml_nodes_file() { 17 export logdir=$1 18 19 <span class="code-keyword">if</span> [ -f $logdir/shared ]; then 20 do_rpc_nodes $(comma_list $(all_nodes)) \ 21 <span class="code-quote">"yml_node &gt;&gt; $logdir/node.\\\$(hostname -s).yml"</span> 22 <span class="code-keyword">else</span> 23 do_rpc_nodes $(comma_list $(all_nodes)) yml_node | split_output 24 fi 25 yml_entities 26 } </pre> </div></div> <p>Using the '-N' option for pdsh is not mandatory, but we should be able to write out results when it is specified. </p> Development Rank 1|i00am7: Rank (Obsolete) 9223372036854775807 Severity