https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14179 Lustre <p>Running a simple <tt>lfs find -i 11 /oak</tt> did generate a few of these messages today. I don't know exactly on which files sorry, just wanted to report it as you might be interested to investigate. It's with Lustre 2.12.5 client and server.</p> CentOS 7.6 Lustre 2.12.5 LU-14179

lfs find: error: llapi_semantic_traverse: string buffer too small

Bug Minor Resolved Fixed Yang Sheng Stephane Thiell Thu, 3 Dec 2020 17:24:51 +0000 Fri, 26 Aug 2022 05:37:24 +0000 Mon, 22 Mar 2021 18:33:19 +0000 Lustre 2.12.5 Lustre 2.15.0 0 5 <p>Yang Sheng</p> <p>Is there anything you could suggest here?</p> <p>Peter</p> <p>One possibility is that there are paths longer than <tt>PATH_MAX</tt> in the filesystem? I was testing this recently, and <em>some</em> things work in this case. It is possible to run something like "<tt>while mkdir subdir; do cd dir; done</tt>" to create a very deep directory tree using relative pathnames, but some of the interfaces assume <tt>PATH_MAX</tt> is the longest possible pathname.</p> <p>It is a trivial change "<tt>lfs find</tt>" to have a larger buffer, but I'm not sure if this is enough to solve the problem(s).</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> diff --git a/lustre/utils/liblustreapi.c b/lustre/utils/liblustreapi.c --- a/lustre/utils/liblustreapi.c +++ b/lustre/utils/liblustreapi.c @@ -2240,7 +2240,7 @@ <span class="code-keyword">static</span> <span class="code-object">int</span> param_callback(<span class="code-object">char</span> *path, semantic_func_t sem_init, param-&gt;fp_depth = 0; - ret = llapi_semantic_traverse(buf, PATH_MAX + 1, NULL, sem_init, + ret = llapi_semantic_traverse(buf, 2 * PATH_MAX + 1, NULL, sem_init, sem_fini, param, NULL); out: find_param_fini(param); </pre> </div></div> <p> Some of the GNU tools handle subdirectory trees larger than <tt>PATH_MAX</tt> by "cd" into some subdirectory, then run operations using relative pathnames. Something similar would be needed for e.g. <tt>FID2PATH</tt>, among other things. I haven't looked at it in any detail.</p> <p>Since we use one buffer to travel path. It must fail on 'small buffer' if the level of directory deep enough. So use relative path looks like a reasonable solution.</p> <p>This the error is coming from such path:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-read_genome_tsv/execution/script lfs find: error: llapi_semantic_traverse: string buffer too small </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@oak-gw02 ~]# ls -l /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-read_genome_tsv/execution/ total 20 -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 2 Feb 25 2019 rc -rwxr--r--+ 1 fwim oak_scg-lab_bpulend 2076 Feb 25 2019 script -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 956 Feb 25 2019 script.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stderr -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stderr.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stdout -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 6 Feb 25 2019 stdout.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 1102 Feb 25 2019 tmp.tsv </pre> </div></div> <p>Looks like the path length is 3776 so still within PATH_MAX which is 4096.</p> <blockquote> <p>Looks like the path length is 3776 so still within PATH_MAX which is 4096.</p></blockquote> <p>In addition to the pathname, the fid2path code stores a header at the start of the struct. Have you tried increasing the allocated buffer size to see if that solves the problem? I tested <tt>fid2path</tt> successfully on a pathname of 4089 bytes in length (2.12.6), and I'm not sure there were any patches in this area, so it isn't clear what is different in your setup.</p> <p>Hello! Actually, it looks like we have a few paths that are &gt; 4096 bytes in length. So I guess we can close this ticket as not being a bug in lfs at all.  I'm a bit surprised that users can create such long paths which might break sysadmin commands, but at least there is a way to detect them thanks to such error. Thank you both for having a look at this and sorry for the noise!</p> <p> </p> <p>A few commands with one of the very long paths:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@oak-h01v08 ~]# path=/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8 [root@oak-h01v08 ~]# echo $path | wc 1 1 4107 [root@oak-h01v08 ~]# cd $path -bash: cd: /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8: File name too long [root@oak-h01v08 ~]# cd $(dirname $path) [root@oak-h01v08 atac]# pwd | wc 1 1 4070 [root@oak-h01v08 atac]# cd e489978a-965b-4848-8a75-997f2f90e3f8/ [root@oak-h01v08 e489978a-965b-4848-8a75-9]# pwd | wc 1 1 4107 </pre> </div></div> <p> <br/> Both `find` and `lfs find` are reporting the same error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@oak-h01v08 ~]# find $path find: ‘/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8’: File name too long [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# lfs find $path lfs find: Path name '/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8' is too long: Invalid argument (22) error: find failed for /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8. </pre> </div></div> <p>I just noticed a slight difference in behavior: if I run a (lfs) find using the parent directory, even with your patch  Andreas, <tt>find</tt> works but  <tt>lfs find</tt> reports an error. It might be absolutely OK as we're beyond <tt>PATH_MAX</tt> anyway.</p> <p> </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@oak-h01v08 ~]# find $(dirname $path) /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8 [root@oak-h01v08 ~]# echo $? 0 [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# lfs find $path lfs find: Path name '/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8' is too long: Invalid argument (22) error: find failed for /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8. [root@oak-h01v08 ~]# echo $? 22 </pre> </div></div> <p> </p> <p>ok - thanks Stephane</p> <p>I wouldn't mind keeping this open as a low-priority ticket to fix this issue, unless that messes with Peter's issue tracking, in which case I'll open a separate LU ticket.</p> <p>I think the GNU tools work around this limitation by changing into a subdirectory and then using a relative pathname to keep the path shorter than <tt>PATH_MAX</tt>. It looks like increasing the userspace buffer size for the <tt>ioctl()</tt> is fairly easily done, but I haven't checked what else would be needed (if anything) to make this work. Obviously, we can't increase <tt>PATH_MAX</tt> everywhere, but very few places actually need a full pathname, only the relative pathname is enough. Symlinks couldn't be larger than what ldiskfs can store (blocksize - 1 = 4095). We already have to split <tt>fid2path</tt> into chunks for DNE to handle crossing MDT boundaries, so we might be able to do the same thing for long pathnames (just return the partial pathname and let the client stitch it together.</p> <p>That said, it looks to me like the user with the very long pathnames has a bug in their application, since it is creating such deep pathnames with a repeating pattern "<tt>call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/...</tt>".</p> <p>Fine with me to reopen</p> <p>Hi Andreas,</p> <p>We hit this (minor) problem again on another directory tree when using <tt>lfs find</tt>. Like the first time, the root cause is clearly a bug in the user's application or script, as the path has a repetitive pattern in it. However, sometimes, surprisingly, we found valid files at the very end of the path, whose absolute paths are above 4096 bytes, so could be useful if <tt>lfs find</tt> does actually report all of them (eg. when we want to empty an OST and make sure no files are remaining on it). I tried your simple patch to avoid <tt>lfs find</tt> failure and I confirm it's working to get rid of the error.</p> <p>Alternatively, one could decide that this error is actually valid and properly reported, and that these paths should be fixed first, as they are indeed too long for PATH_MAX. In that case, I think it would be nice to add the path of the directory to the error message, otherwise currently it's difficult to identify these long paths; something simple like this does seem to work for me:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>@@ -2014,8 +2014,8 @@ static int llapi_semantic_traverse(char *path, int size, DIR *parent, path[len] = 0; if ((len + dent-&gt;d_reclen + 2) &gt; size) { llapi_err_noerrno(LLAPI_MSG_ERROR, - "error: %s: string buffer too small", - __func__); + "error: %s: string buffer too small for %s", + __func__, path); break; } strcat(path, "/"); </pre> </div></div> <p>Feel free to put that into a patch, ideally with a test case. </p> <p>Fir the user's sake, they should probably fix their application to avoid creating a directory like that. </p> <p>Stephane Thiell (sthiell@stanford.edu) uploaded a new patch: <a href="https://review.whamcloud.com/41337" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41337</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14179" title="lfs find: error: llapi_semantic_traverse: string buffer too small" class="issue-link" data-issue-key="LU-14179"><del>LU-14179</del></a> lfs: avoid lfs find error with long paths<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 72d703753eda6abe14e7ceae7d973ddeab65b22a</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/41337/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41337/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14179" title="lfs find: error: llapi_semantic_traverse: string buffer too small" class="issue-link" data-issue-key="LU-14179"><del>LU-14179</del></a> lfs: avoid lfs find error with long paths<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: a6a76df19db61a2015f4cc78f88060f249c955f2</p> <p>Landed for 2.15</p> Development Rank 1|i01gb3: Rank (Obsolete) 9223372036854775807 Severity