Details
-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
Split from LU-16789.
By Andreas:
It isn't clear whether there is a need to have many threads if there can be multiple AIO requests active at one time. It would be enough to have two or three threads active at once - one to create one file in advance, and the others to issue AIO read/write requests for the its file until it hits the AIO limit. If enough AIO requests have been submitted to read (and in turn write) the whole file size, then the remaining AIO requests would go to a new file/thread, while the current thread waits for the pending AIO completions and closing the file before taking the next file from the input to process.
That way, only as many threads are started as needed to fill the AIO queue. It makes sense to have at least 2 or 3 threads actively reading/writing to different files (OSTs) so that they do not get blocked waiting for one OST that is slow or has hit max_rpcs_in_flight, but it also doesn't help to have thousands of in-flight IO requests to avoid clogging the network and OSTs. The --threads parameter would be the maximum number of threads started, if that is necessary (eg. for files <= 1MiB in size), but that probably wouldn't be helpful for large files.
Ideally this can be self-balancing in some way, so that only as many AIO requests are submitted on a single file as needed to keep it busy, and the others are used for other files. Similarly, the number of AIO requests are balanced to hit peak performance (or the specified bandwidth limit), and not thousands of requests that are all running slowly.
Attachments
Issue Links
- is related to
-
-
- Open
-