https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1665 Lustre <p>hsb_head<span class="error">&#91;0&#93;</span> can experience poor alignment on certain platforms with odd alignment requirements like the ia64. it is possible for hsb_head<span class="error">&#91;0&#93;</span> to be aligned to a 32-bit boundary, but later it could be cast to something with a stricter alignment requirement. this can/does cause hardware faults for unaligned access:</p> <p>Jul 18 19:25:20 seraph kernel: kernel unaligned access to 0xe0000069eef8f194, ip=0xa00000028342e0d1<br/> Jul 18 19:25:20 seraph kernel: kernel unaligned access to 0xe0000069eef8f19c, ip=0xa00000028342e0d1<br/> Jul 18 19:25:20 seraph kernel: kernel unaligned access to 0xe0000069eef8f1a4, ip=0xa00000028342e0d1<br/> Jul 18 19:25:20 seraph kernel: kernel unaligned access to 0xe0000069eef8f1ac, ip=0xa00000028342e0d1<br/> Jul 18 19:25:20 seraph kernel: kernel unaligned access to 0xe0000069eef8f1b4, ip=0xa00000028342e0d1</p> ia64 LU-1665

hsb_head[0] in cfs_hash_bucket_t can experience poor alignment

Bug Minor Resolved Fixed Keith Mannthey chas williams - CONTRACTOR Tue, 24 Jul 2012 10:22:40 +0000 Wed, 23 Jan 2013 14:28:04 +0000 Wed, 23 Jan 2013 14:28:04 +0000 Lustre 2.2.0 Lustre 2.1.2 Lustre 2.4.0 0 5 <p>Keith</p> <p>Can you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>The char to long change is a low risk change overall but is isn't clear to me we are not just masking an underlaying issue. </p> <p>Do you know know what part of the code that accesses this data structure is causing the unaligned access? Where "ip=0xa00000028342e0d1" is? </p> <p>for whatever reason, gcc doesnt quite align the structure optimally for ia64. it seems to aligning so that the data structure is compatible with x86/x86_64. try this program on x86_64:</p> <p>#include &lt;stdio.h&gt;<br/> #include &lt;stdlib.h&gt;</p> <p>struct long_then_charp {<br/> unsigned long value;<br/> unsigned char string<span class="error">&#91;0&#93;</span>;<br/> };</p> <p>struct int_then_charp </p> { unsigned int value; unsigned char string[0]; }; int main(int argc, char **argv) { struct long_then_charp *a; struct int_then_charp *b; unsigned long *anon; a = malloc(32); b = malloc(32); printf("&amp;a.value %p\n", &amp;a-&gt;value); printf("a.string[] %p\n", a-&gt;string); anon = (void *) a-&gt;string; *anon = 0xdeadbeefdeadbeef; printf("&amp;b.value %p\n", &amp;b-&gt;value); printf("b.string[] %p\n", b-&gt;string); anon = (void *) b-&gt;string; *anon = 0xdeadbeefdeadbeef; } <p>when you run this you will see that a.string is aligned to an 8 byte boundary in one case and a 4 byte boundary in the other case. both of these will work on x86/x86_64 without complaint (although i imagine that the x64_64 might benefit from the long being aligned to an 8 byte boundary?)</p> <p>the kernel on ia64 issues a runtime message though for this program:</p> <p>% a.out<br/> &amp;a.value 0x6000000000004010<br/> a.string[] 0x6000000000004018<br/> &amp;b.value 0x6000000000004040<br/> b.string[] 0x6000000000004044<br/> a.out(119837): unaligned access to 0x6000000000004044, ip=0x40000000000009e0</p> <p>so the b.string[] that was cast to the unsigned long and then accessed caused this fault. the program will run, but performance is somewhat reduced since the kernel has to handle this misalignment.</p> <p>in lustre, this specifically seems to be here:</p> <p>(gdb) info line *cfs_hash_bd_lookup_intent+0x80<br/> Line 630 of "/scratch1/chas/lustre-2.1.2/libcfs/libcfs/hash.c" starts at address 0x28e0 &lt;cfs_hash_bd_lookup_intent+128&gt;<br/> and ends at 0x28e1 &lt;cfs_hash_bd_lookup_intent+129&gt;.</p> <p>that's here:<br/> tatic cfs_hlist_node_t *<br/> cfs_hash_bd_lookup_intent(cfs_hash_t *hs, cfs_hash_bd_t *bd,<br/> const void *key, cfs_hlist_node_t *hnode,<br/> cfs_hash_lookup_intent_t intent)</p> <p>{<br/> cfs_hlist_head_t *hhead = cfs_hash_bd_hhead(hs, bd);<br/> cfs_hlist_node_t *ehnode;<br/> cfs_hlist_node_t *match;<br/> int intent_add = (intent &amp; CFS_HS_LOOKUP_MASK_ADD) != 0;</p> <p> /* with this function, we can avoid a lot of useless refcount ops,</p> <ul> <li>which are expensive atomic operations most time. */<br/> match = intent_add ? NULL : hnode;<br/> &gt;&gt;&gt;&gt; cfs_hlist_for_each(ehnode, hhead) {</li> </ul> <p>cfs_hash_bd_hhead() converts hsb_head<span class="error">&#91;0&#93;</span> via the appropriate cfs_hash_hlist_ops_t to a cfs_hlist_head_t. apparently hhead winds up with a poor alignment for ia64 because of this. there are a couple other places, where this happens but it is basically the same.</p> <p>The reasoning seems good to me. Can you submit the patch in git? </p> <p><a href="http://wiki.whamcloud.com/display/PUB/Submitting+Changes" class="external-link" target="_blank" rel="nofollow noopener">http://wiki.whamcloud.com/display/PUB/Submitting+Changes</a></p> <p>It makes the DCO and testing much easier. </p> <p>Do you have any questions about the process?</p> <p>no. just busy this week.</p> <p>Without an update I will close this LU at the end of the Month. </p> <p>i have a patch in gerrit <a href="http://review.whamcloud.com/#change,3603" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3603</a></p> <p>i dont know if it was merged. build testing failed but i think the error was due to problems with the backend testing facility.</p> <p>Hi Chas</p> <p>Your change has not been merged yet. It has failed autotest due to an unrelated error. Usually the ticket owner will monitor contributed patches and take action if this happens but I imagine Keith was unaware of the patch. The convention is to post a comment with a link to any related patches so that those watching the JIRA ticket can find them.</p> <p>Keith</p> <p>Can you please see whether it is best to trigger a re-test on the patch with a no score build or else rebase the patch on the latest master code.</p> <p>Thanks</p> <p>Peter</p> <p>Sorry for that patch lost issue last year. </p> <p>The patch has just been landed to master. </p> Development Rank 1|hzvgqf: Rank (Obsolete) 6360 Severity