Details
-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
3
-
9223372036854775807
Description
It would be useful for "lfs find" to perform directory scans with multiple threads in parallel. It could (potentially) fork a new thread (or put a work item into a pool) for each subdirectory so that they can be scanned in parallel.
There is "libcircle" and "libpcircle" that can perform workload sharing to speed up directory traversal. Also, the pfind code in IO500 is also doing efficient parallel directory traversal, including splitting up large directories by hash index to traverse in parallel.
Integrating one of these algorithms into "lfs find" with pthreads would allow a many-fold improvement in directory scanning performance.
Attachments
Issue Links
- is related to
-
-
- Open
-