https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10258 Lustre <p>It would be useful to add an "<tt>lfs open <span class="error">&#91;--ifcomp=&lt;component_id&gt;&#93;</span> <span class="error">&#91;--ofcomp=&lt;component_id&gt;&#93;</span> &lt;command&gt;</tt>" that opens the stdin and/or stdout with the specified component ID values, so that e.g. "dd" could be used as a resync tool for stale components, or other arbitrary commands (e.g. MPI file utils for parallel resync, HSM tools for archive handling, etc).</p> <p>Also useful would be to open a specific mirror number for whole-file access, instead of just a single component. That would be something like "<tt>&#45;&#45;ifrepl=</tt>" and "<tt>&#45;&#45;ofrepl=</tt>" options. </p> LU-10258

lfs mirror command to read/write specific mirror copy

Improvement Minor Resolved Fixed Zhenyu Xu Andreas Dilger FLR2 Sun, 19 Nov 2017 22:08:38 +0000 Mon, 13 Jun 2022 22:10:35 +0000 Thu, 21 Mar 2019 04:58:34 +0000 Lustre 2.13.0 Lustre 2.12.3 0 6 <blockquote> <p>Should we support the command like <tt>lfs mirror dd <span class="error">&#91;--if=&lt;mirror_id&gt;&#93;</span> <span class="error">&#91;--of=&lt;mirror_id&gt;&#93;</span></tt> instead? If <tt>-<del>if</tt> is not existing, it will read from <tt>stdin</tt>; if <tt></del><del>of</tt> is not existing, it will write to <tt>stdout</tt>. Multiple <tt></del>-off</tt> can be supported if necessary. This is more like unix and is able to use pipe to chain commands.</p></blockquote> <p>The problem with <tt>--ifcomp</tt> is that sometimes components are not started from beginning of the file; neither are components perfectly aligned. Therefore it would be hard to define the semantics clearly.</p> <p>From <a href="https://review.whamcloud.com/#/c/29100/23/lustre/doc/lfs-mirror-verify.1" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/29100/23/lustre/doc/lfs-mirror-verify.1</a></p> <p>Andreas Dilger Nov 22 9:37 PM</p> <blockquote> <p>Not for this patch, but it would be interesting to use something like:<br/> lfs open --ifrepl=N &lt;file&gt; | diff &lt;file&gt; /dev/stdin<br/> (or with cmp) to be able to find what is different between two replicas. <br/> That makes me wonder if we would want some special open syntax some time later, like:<br/> diff &lt;file&gt; "&lt;file&gt;;N"<br/> to allow opening replica N directly from the command line? This uses the same naming to be consistent with VAX VMS file versions, and has the benefit that virtually no files will have ';' in the name because this is a shell metacharacter. The drawback is that accessing non-primary components would always need to escape or quote the file name, which is probably OK if it isn't commonly used. We might use something like ",,N" to allow easier usage, but have a higher risk of collisions with existing filenames.</p></blockquote> <blockquote> <p>The problem with --ifcomp is that sometimes components are not started from beginning of the file; neither are components perfectly aligned. Therefore it would be hard to define the semantics clearly.</p></blockquote> <p>I think there are legitimate use cases for opening a component and allowing writes outside of the component start/end. For example, migrating a large DoM file from the MDT to the start of an OST object and freeing up space on the MDT.</p> <p>I wonder if we should rename this to "<tt>lfs mirror write</tt>" and rename "<tt>lfs mirror dump</tt>" to "<tt>lfs mirror read</tt>" for consistency?</p> <p>Bobi Jam (bobijam@hotmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/33219" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33219</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror write command<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 08c8accd22dc1021da3a0c37d8c6a33f1bf44607</p> <p>Bobi Jam (bobijam@hotmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/33220" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33220</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror copy command<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 075a96702fe89c37a9a4e1c11401a75d173dfdad</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33219/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33219/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror write command<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 14171e787dd00902af7c9edf10a6b54aab45f7b2</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33220/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33220/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror copy command<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: c6e7c0788d7cd766880d12eae6679782283dc479</p> <p>Landed for 2.13</p> <p>Jian Yu (yujian@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36193" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36193</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror copy command<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 00a5d4a04710b5c939b81c422e7b800471ffd83d</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36193/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36193/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10258" title="lfs mirror command to read/write specific mirror copy" class="issue-link" data-issue-key="LU-10258"><del>LU-10258</del></a> lfs: lfs mirror copy command<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 24f181513fb14f111455ec113d69a6ce5b50cf56</p> Related LU-5170 LU-9771 LU-11245 LU-10177 LU-11421 LU-13831 Development Rank 1|hzzo1r: Rank (Obsolete) 9223372036854775807