[LU-8626] limit number of items in HSM action queue

• Type: Improvement
• Resolution: Unresolved
• Priority: Major
• Labels: hsm
• Assignee: Quentin Bouget (Inactive)
• Reporter: Andreas Dilger

Description

Several presentations at RUG'16 mentioned that Lustre has poor performance when there are very large numbers of HSM actions outstanding on the coordinator.

Firstly, having a /proc file that exposes the number of entries currently in the HSM action list would allow RBH and monitoring scripts to easily monitor the number of enties.

Issue Links

• is related to: LU-7988 HSM: high lock contention for cdt_llog_lock

Comments

[Andreas Dilger]

2.11 has already been released.

[Thomas Leibovici]

About already landed https://review.whamcloud.com/30336/:

Given the implemented counters refers to the contents of "active_requests" list, they should rather be named "active_archive_count", "active_restore_count", ... instead of "archive_count", etc... to be more explicit and avoid any confusion with the contents of hsm/actions that contains all requested actions.

This change should be done before releasing 2.11 to avoid changing names in /proc later after the feature is released.

[Peter Jones]

ok

[Quentin Bouget (Inactive)]

Hi Peter,

I am not sure this issue should be marked as resolved yet.
The patches that landed only provide information about how many requests the coordinator is currently handling, there are no built-in limitations yet.

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/30336/
Subject: LU-8626 hsm: expose the number of active hsm requests per type
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 42e40555f250b83730d233dc5e22fd1f9396ccfe

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/28677/
Subject: LU-8626 hsm: count the number of started requests of each type
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 973759d1ff3bbcb217754bd9942fdf670dec2d96

[Gerrit Updater]

Quentin Bouget ([e-mail]) uploaded a new patch: https://review.whamcloud.com/30336
Subject: LU-8626 hsm: expose the number of active hsm requests per type
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 294a0e4ff77fb09ac643b5e15af224027ded4aee

[Gerrit Updater]

Quentin Bouget ([e-mail]) uploaded a new patch: https://review.whamcloud.com/28677
Subject: LU-8626 hsm: count the number of started requests of each type
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4e84ba8aa669554b2d1b77459ebe79770aa4ad37

[Andreas Dilger]

I think we should stick with a single value per file, since this is required when moving stats into /sys/fs/lustre, so something like action_archive_started_count, action_archive_succeed_count, action_restore_started_count, action_restore_succeed_count.