https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10114 Lustre <p>Hello,<br/> As mentioned in Xue Wei's LAD'17 talk: <a href="https://www.eofs.eu/_media/events/lad17/08_li_xi_lcoc_lad_2017.pdf" class="external-link" target="_blank" rel="nofollow noopener">https://www.eofs.eu/_media/events/lad17/08_li_xi_lcoc_lad_2017.pdf</a> (LCOC: Lustre Cache on Client based on SSD – Xue Wei, NSCC-Wuxi and Li Xi, DDN), there is currently an upper limit of 32 HSM archives that can be registered.</p> <p>We also have a potential use-case for HSM that would greatly benefit from being able to increase this threshold, for example we would then be able to allocate a HSM archive for an individual customer's project, thus being able to colocate their HSM archived files more logically on our particular HSM backend (a tape filesystem), which would greatly improve our ability to restore large numbers of files.</p> <p>My question is just is this upper limit something that is relatively simple to increase without impacting much else?</p> <p>I've emailed Li Xi from DDN who was also listed in the talk (I haven't managed to find the email address of Xue Wei from NSCC-Wuxi) if he or Xue could comment on this too, so I'll update this if I hear back from them.</p> <p>Thanks,<br/> Matt</p> LU-10114

Feasibility of increasing upper limit of maximum HSM backends registered with MDT

Improvement Minor Resolved Fixed Teddy Matt Rásó-Barnett HSM Tue, 10 Oct 2017 11:26:47 +0000 Thu, 29 Nov 2018 23:12:26 +0000 Thu, 29 Nov 2018 23:12:26 +0000 Lustre 2.12.0 0 8 <p>Unfortunately the limit of 32 archives is part of the wire protocol:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">struct req_msg_field RMF_MDS_HSM_ARCHIVE = DEFINE_MSGF(<span class="code-quote">"hsm_archive"</span>, 0, sizeof(__u32), lustre_swab_generic_32s, NULL); EXPORT_SYMBOL(RMF_MDS_HSM_ARCHIVE); </pre> </div></div> <p>So it can be changed but it will take some time for this to be seen in a production release.</p> <p>John, typically it should be possible to increase the size of a buffer without breaking backward compatibility, so long as old clients aren't expected to access any of the fields beyond the old size of the buffer. Unfortunately, the <tt>MDS_HSM_ARCHIVE</tt> RPC is used between the client and MDS, so there needs to be some compatibility in place (probably <tt>struct obd_connect_data</tt> feature flag and max archive count) to negotiate the limits between them.</p> <p>I also see that the archive ID is used as a 32-bit value in a few structs, in particular those in <tt>lustre_user.h</tt>:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> struct hsm_user_state { <span class="code-comment">/** Current HSM states, from <span class="code-keyword">enum</span> hsm_states. */</span> __u32 hus_states; __u32 hus_archive_id; <span class="code-comment">/** The current undergoing action, <span class="code-keyword">if</span> there is one */</span> __u32 hus_in_progress_state; __u32 hus_in_progress_action; struct hsm_extent hus_in_progress_location; <span class="code-object">char</span> hus_extended_info[]; }; struct hsm_request { __u32 hr_action; <span class="code-comment">/* <span class="code-keyword">enum</span> hsm_user_action */</span> __u32 hr_archive_id; <span class="code-comment">/* archive id, used only with HUA_ARCHIVE */</span> __u64 hr_flags; <span class="code-comment">/* request flags */</span> __u32 hr_itemcount; <span class="code-comment">/* item count in hur_user_item vector */</span> __u32 hr_data_len; }; struct hsm_action_list { __u32 hal_version; __u32 hal_count; <span class="code-comment">/* number of hai's to follow */</span> __u64 hal_compound_id; <span class="code-comment">/* returned by coordinator */</span> __u64 hal_flags; __u32 hal_archive_id; <span class="code-comment">/* which archive backend */</span> __u32 padding1; <span class="code-object">char</span> hal_fsname[0]; <span class="code-comment">/* <span class="code-keyword">null</span>-terminated */</span> /* struct hsm_action_item[hal_count] follows, aligned on 8-<span class="code-object">byte</span> boundaries. See hai_zero */ } __attribute__((packed)); struct hsm_user_import { __u64 hui_size; __u64 hui_atime; __u64 hui_mtime; __u32 hui_atime_ns; __u32 hui_mtime_ns; __u32 hui_uid; __u32 hui_gid; __u32 hui_mode; __u32 hui_archive_id; }; </pre> </div></div> <p>but these all appear to be used as integer values and not bitmaps, so they should be OK without any changes.</p> <p>Also, sanity-hsm.sh <tt>test_50()</tt> and <tt>test_51()</tt> are testing that up to 32 archives can be used, so this test would need to be updated if we allow more archives.</p> <p>Also, the copytool registration to the kernel passes only a 32-bit <tt>lk_data</tt> mask to indicate which archives it is in charge of:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">struct lustre_kernelcomm { __u32 lk_wfd; __u32 lk_rfd; __u32 lk_uid; __u32 lk_group; __u32 lk_data; __u32 lk_flags; } __attribute__((packed)); </pre> </div></div> <p>Hi Li Xi, could you make Xue Wei aware of this ticket?</p> <p>Teddy Zheng (jjkky@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/32197" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32197</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10114" title="Feasibility of increasing upper limit of maximum HSM backends registered with MDT" class="issue-link" data-issue-key="LU-10114"><del>LU-10114</del></a> hsm: increasing upper limit of maximum HSM backends registered with MDT<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 733344c1d871ae7925822de8135b32900ea2c776</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/32806/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32806/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10114" title="Feasibility of increasing upper limit of maximum HSM backends registered with MDT" class="issue-link" data-issue-key="LU-10114"><del>LU-10114</del></a> hsm: add OBD_CONNECT2_ARCHIVE_ID_ARRAY to pass archive_id lists in array<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1c7e7d1243f78c72210a0ba3c22d5c84838a416e</p> <p>John L. Hammond (jhammond@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33631" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33631</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10114" title="Feasibility of increasing upper limit of maximum HSM backends registered with MDT" class="issue-link" data-issue-key="LU-10114"><del>LU-10114</del></a> hsm: noop chaneg for introp baselining<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 2b02bf4747f0e6cb5d523471a4a6df226c7a5e86</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/32197/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32197/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10114" title="Feasibility of increasing upper limit of maximum HSM backends registered with MDT" class="issue-link" data-issue-key="LU-10114"><del>LU-10114</del></a> hsm: increase upper limit of maximum HSM backends registered with MDT<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3bfb6107ba4e92d8aa02e842502bc44bac7b8b43</p> <p>Landed for 2.12</p> Related LU-10092 Development Rank 1|hzzln3: Rank (Obsolete) 9223372036854775807