[LU-14179] lfs find: error: llapi_semantic_traverse: string buffer too small Created: 03/Dec/20 Updated: 26/Aug/22 Resolved: 22/Mar/21 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.12.5 |
| Fix Version/s: | Lustre 2.15.0 |
| Type: | Bug | Priority: | Minor |
| Reporter: | Stephane Thiell | Assignee: | Yang Sheng |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Environment: |
CentOS 7.6 Lustre 2.12.5 |
||
| Severity: | 4 |
| Rank (Obsolete): | 9223372036854775807 |
| Description |
|
Running a simple lfs find -i 11 /oak did generate a few of these messages today. I don't know exactly on which files sorry, just wanted to report it as you might be interested to investigate. It's with Lustre 2.12.5 client and server. |
| Comments |
| Comment by Peter Jones [ 03/Dec/20 ] |
|
Yang Sheng Is there anything you could suggest here? Peter |
| Comment by Andreas Dilger [ 04/Dec/20 ] |
|
One possibility is that there are paths longer than PATH_MAX in the filesystem? I was testing this recently, and some things work in this case. It is possible to run something like "while mkdir subdir; do cd dir; done" to create a very deep directory tree using relative pathnames, but some of the interfaces assume PATH_MAX is the longest possible pathname. It is a trivial change "lfs find" to have a larger buffer, but I'm not sure if this is enough to solve the problem(s). diff --git a/lustre/utils/liblustreapi.c b/lustre/utils/liblustreapi.c --- a/lustre/utils/liblustreapi.c +++ b/lustre/utils/liblustreapi.c @@ -2240,7 +2240,7 @@ static int param_callback(char *path, semantic_func_t sem_init, param->fp_depth = 0; - ret = llapi_semantic_traverse(buf, PATH_MAX + 1, NULL, sem_init, + ret = llapi_semantic_traverse(buf, 2 * PATH_MAX + 1, NULL, sem_init, sem_fini, param, NULL); out: find_param_fini(param); Some of the GNU tools handle subdirectory trees larger than PATH_MAX by "cd" into some subdirectory, then run operations using relative pathnames. Something similar would be needed for e.g. FID2PATH, among other things. I haven't looked at it in any detail. |
| Comment by Yang Sheng [ 04/Dec/20 ] |
|
Since we use one buffer to travel path. It must fail on 'small buffer' if the level of directory deep enough. So use relative path looks like a reasonable solution. |
| Comment by Stephane Thiell [ 04/Dec/20 ] |
|
This the error is coming from such path: /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-read_genome_tsv/execution/script lfs find: error: llapi_semantic_traverse: string buffer too small [root@oak-gw02 ~]# ls -l /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-read_genome_tsv/execution/ total 20 -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 2 Feb 25 2019 rc -rwxr--r--+ 1 fwim oak_scg-lab_bpulend 2076 Feb 25 2019 script -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 956 Feb 25 2019 script.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stderr -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stderr.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 0 Feb 25 2019 stdout -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 6 Feb 25 2019 stdout.submit -rw-r--r--+ 1 fwim oak_scg-lab_bpulend 1102 Feb 25 2019 tmp.tsv Looks like the path length is 3776 so still within PATH_MAX which is 4096. |
| Comment by Andreas Dilger [ 10/Dec/20 ] |
In addition to the pathname, the fid2path code stores a header at the start of the struct. Have you tried increasing the allocated buffer size to see if that solves the problem? I tested fid2path successfully on a pathname of 4089 bytes in length (2.12.6), and I'm not sure there were any patches in this area, so it isn't clear what is different in your setup. |
| Comment by Stephane Thiell [ 13/Dec/20 ] |
|
Hello! Actually, it looks like we have a few paths that are > 4096 bytes in length. So I guess we can close this ticket as not being a bug in lfs at all. I'm a bit surprised that users can create such long paths which might break sysadmin commands, but at least there is a way to detect them thanks to such error. Thank you both for having a look at this and sorry for the noise!
A few commands with one of the very long paths: [root@oak-h01v08 ~]# path=/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8
[root@oak-h01v08 ~]# echo $path | wc
1 1 4107
[root@oak-h01v08 ~]# cd $path
-bash: cd: /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8: File name too long
[root@oak-h01v08 ~]# cd $(dirname $path)
[root@oak-h01v08 atac]# pwd | wc
1 1 4070
[root@oak-h01v08 atac]# cd e489978a-965b-4848-8a75-997f2f90e3f8/
[root@oak-h01v08 e489978a-965b-4848-8a75-9]# pwd | wc
1 1 4107
[root@oak-h01v08 ~]# find $path find: ‘/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8’: File name too long [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# lfs find $path lfs find: Path name '/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8' is too long: Invalid argument (22) error: find failed for /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8. |
| Comment by Stephane Thiell [ 13/Dec/20 ] |
|
I just noticed a slight difference in behavior: if I run a (lfs) find using the parent directory, even with your patch Andreas, find works but lfs find reports an error. It might be absolutely OK as we're beyond PATH_MAX anyway.
[root@oak-h01v08 ~]# find $(dirname $path) /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8 [root@oak-h01v08 ~]# echo $? 0 [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# [root@oak-h01v08 ~]# lfs find $path lfs find: Path name '/oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8' is too long: Invalid argument (22) error: find failed for /oak/stanford/scg/lab_bpulend/wimmers/mo_atacScr_v1/cromwell-executions/atac/d40d2b02-4a21-4f6a-a0f5-443e903a40ab/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8. [root@oak-h01v08 ~]# echo $? 22
|
| Comment by Peter Jones [ 14/Dec/20 ] |
|
ok - thanks Stephane |
| Comment by Andreas Dilger [ 14/Dec/20 ] |
|
I wouldn't mind keeping this open as a low-priority ticket to fix this issue, unless that messes with Peter's issue tracking, in which case I'll open a separate LU ticket. I think the GNU tools work around this limitation by changing into a subdirectory and then using a relative pathname to keep the path shorter than PATH_MAX. It looks like increasing the userspace buffer size for the ioctl() is fairly easily done, but I haven't checked what else would be needed (if anything) to make this work. Obviously, we can't increase PATH_MAX everywhere, but very few places actually need a full pathname, only the relative pathname is enough. Symlinks couldn't be larger than what ldiskfs can store (blocksize - 1 = 4095). We already have to split fid2path into chunks for DNE to handle crossing MDT boundaries, so we might be able to do the same thing for long pathnames (just return the partial pathname and let the client stitch it together. That said, it looks to me like the user with the very long pathnames has a bug in their application, since it is creating such deep pathnames with a repeating pattern "call-trim_adapter/shard-1/inputs/1458923937/mo_atacScr.tmp/cromwell-executions/atac/e489978a-965b-4848-8a75-997f2f90e3f8/...". |
| Comment by Peter Jones [ 14/Dec/20 ] |
|
Fine with me to reopen |
| Comment by Stephane Thiell [ 24/Jan/21 ] |
|
Hi Andreas, We hit this (minor) problem again on another directory tree when using lfs find. Like the first time, the root cause is clearly a bug in the user's application or script, as the path has a repetitive pattern in it. However, sometimes, surprisingly, we found valid files at the very end of the path, whose absolute paths are above 4096 bytes, so could be useful if lfs find does actually report all of them (eg. when we want to empty an OST and make sure no files are remaining on it). I tried your simple patch to avoid lfs find failure and I confirm it's working to get rid of the error. Alternatively, one could decide that this error is actually valid and properly reported, and that these paths should be fixed first, as they are indeed too long for PATH_MAX. In that case, I think it would be nice to add the path of the directory to the error message, otherwise currently it's difficult to identify these long paths; something simple like this does seem to work for me: @@ -2014,8 +2014,8 @@ static int llapi_semantic_traverse(char *path, int size, DIR *parent,
path[len] = 0;
if ((len + dent->d_reclen + 2) > size) {
llapi_err_noerrno(LLAPI_MSG_ERROR,
- "error: %s: string buffer too small",
- __func__);
+ "error: %s: string buffer too small for %s",
+ __func__, path);
break;
}
strcat(path, "/");
|
| Comment by Andreas Dilger [ 24/Jan/21 ] |
|
Feel free to put that into a patch, ideally with a test case. Fir the user's sake, they should probably fix their application to avoid creating a directory like that. |
| Comment by Gerrit Updater [ 27/Jan/21 ] |
|
Stephane Thiell (sthiell@stanford.edu) uploaded a new patch: https://review.whamcloud.com/41337 |
| Comment by Gerrit Updater [ 22/Mar/21 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/41337/ |
| Comment by Peter Jones [ 22/Mar/21 ] |
|
Landed for 2.15 |