https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4691 Lustre <p>Most archives fail (depends on MDT index). Successfully released files cannot be restored.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># export MOUNT_2=y # export MDSCOUNT=4 # llmount.sh # for ((i = 0; i &lt; MDSCOUNT; i++)); do mdt_name=$(printf 'lustre-MDT%04x' $i) lctl conf_param $mdt_name.mdt.hsm_control=enabled done # mount $HOSTNAME@tcp:/lustre /mnt/lustre-hsm -t lustre -o user_xattr,flock # lhsmtool_posix -vvvv --hsm_root=$HSM_ROOT --daemon /mnt/lustre-hsm 2&gt; /tmp/hsm.log # lfs mkdir -i0 -c4 /mnt/lustre/d0 # cd /mnt/lustre/d0 # echo &gt; f0 # echo &gt; f1 # echo &gt; f2 # echo &gt; f3 # lfs hsm_archive f0 # lfs hsm_state f0 f0: (0x00000001) exists, archive_id:1 # lfs hsm_release f0 Cannot send HSM request (use of f0): Operation not permitted # # lfs hsm_archive f1 # lfs hsm_archive f2 # lfs hsm_archive f3 # lfs hsm_release f1 # lfs hsm_release f2 Cannot send HSM request (use of f2): Operation not permitted # lfs hsm_release f3 Cannot send HSM request (use of f3): Operation not permitted # lfs hsm_state * f0: (0x00000001) exists, archive_id:1 f1: (0x0000000d) released exists archived, archive_id:1 f2: (0x00000001) exists, archive_id:1 f3: (0x00000001) exists, archive_id:1 # cat f1 ^C </pre> </div></div> LU-4691

HSM operations mostly fail in striped directories

Bug Critical Closed Fixed WC Triage John Hammond HSM dne2 mdt Sat, 1 Mar 2014 00:05:39 +0000 Thu, 31 Jul 2014 15:19:06 +0000 Thu, 31 Jul 2014 15:18:47 +0000 Lustre 2.6.0 Lustre 2.7.0 0 7 <p><tt>lfs hsm_state</tt> clearly shows that only <tt>f1</tt> was successfully archived, that's the reason why files could not be released/restored.</p> <p>Could you display what is in copytool log <tt>/tmp/hsm.log</tt>, MDTs logs and client logs?</p> <p>Hmm, I tried to reproduce the problem, but seems works for me, though it seems the daemon can not be started correctly after the mount, and I have to start again.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@mds tests]# sh -vx tests.sh umount /mnt/lustre-hsm + umount /mnt/lustre-hsm MOUNT_2=y MDSCOUNT=4 sh llmount.sh + MOUNT_2=y + MDSCOUNT=4 + sh llmount.sh Stopping clients: mds /mnt/lustre (opts:) Stopping client mds /mnt/lustre opts: Stopping clients: mds /mnt/lustre2 (opts:) Stopping client mds /mnt/lustre2 opts: Stopping /mnt/mds1 (opts:-f) on mds Stopping /mnt/mds2 (opts:-f) on mds Stopping /mnt/mds3 (opts:-f) on mds Stopping /mnt/mds4 (opts:-f) on mds Stopping /mnt/ost1 (opts:-f) on mds Stopping /mnt/ost2 (opts:-f) on mds waited 0 for 31 ST ost OSS OSS_uuid 0 Loading modules from /work/lustre-release_new/lustre-release/lustre/tests/.. detected 8 online CPUs by sysfs libcfs will create CPU partition based on online CPUs debug=vfstrace rpctrace dlmtrace neterror ha config ioctl super subsystem_debug=all -lnet -lnd -pinger gss/krb5 is not supported Formatting mgs, mds, osts Format mds1: /tmp/lustre-mdt1 Format mds2: /tmp/lustre-mdt2 Format mds3: /tmp/lustre-mdt3 Format mds4: /tmp/lustre-mdt4 Format ost1: /tmp/lustre-ost1 Format ost2: /tmp/lustre-ost2 Checking servers environments Checking clients mds environments Loading modules from /work/lustre-release_new/lustre-release/lustre/tests/.. detected 8 online CPUs by sysfs libcfs will create CPU partition based on online CPUs debug=vfstrace rpctrace dlmtrace neterror ha config ioctl super subsystem_debug=all -lnet -lnd -pinger gss/krb5 is not supported Setup mgs, mdt, osts Starting mds1: -o loop /tmp/lustre-mdt1 /mnt/mds1 Started lustre-MDT0000 Starting mds2: -o loop /tmp/lustre-mdt2 /mnt/mds2 Started lustre-MDT0001 Starting mds3: -o loop /tmp/lustre-mdt3 /mnt/mds3 Started lustre-MDT0002 Starting mds4: -o loop /tmp/lustre-mdt4 /mnt/mds4 Started lustre-MDT0003 Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1 Started lustre-OST0000 Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2 Started lustre-OST0001 Starting client: mds: -o user_xattr,flock mds@tcp:/lustre /mnt/lustre Starting client: mds: -o user_xattr,flock mds@tcp:/lustre /mnt/lustre2 Using TIMEOUT=20 disable quota as required /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0000.mdt.hsm_control=enabled + /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0000.mdt.hsm_control=enabled /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0001.mdt.hsm_control=enabled + /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0001.mdt.hsm_control=enabled /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0002.mdt.hsm_control=enabled + /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0002.mdt.hsm_control=enabled /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0003.mdt.hsm_control=enabled + /work/lustre-release_new/lustre-release/lustre/utils/lctl conf_param lustre-MDT0003.mdt.hsm_control=enabled mount -t lustre -o user_xattr,flock mds:/lustre /mnt/lustre-hsm + mount -t lustre -o user_xattr,flock mds:/lustre /mnt/lustre-hsm sleep 5 + sleep 5 /work/lustre-release_new/lustre-release/lustre/utils/lhsmtool_posix -vvvv --hsm_root=/mnt/lustre --daemon /mnt/lustre-hsm 2&gt; /tmp/hsm.log + /work/lustre-release_new/lustre-release/lustre/utils/lhsmtool_posix -vvvv --hsm_root=/mnt/lustre --daemon /mnt/lustre-hsm /work/lustre-release_new/lustre-release/lustre/utils/lfs mkdir -i0 -c4 /mnt/lustre/d0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs mkdir -i0 -c4 /mnt/lustre/d0 cd /mnt/lustre/d0 + cd /mnt/lustre/d0 echo &gt; f0 + echo echo &gt; f1 + echo echo &gt; f2 + echo echo &gt; f3 + echo [root@mds tests]# vim /tmp/hsm.log [root@mds tests]# sh -vx test_1.sh cd /mnt/lustre/d0 + cd /mnt/lustre/d0 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f0 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f0 f0: (0x00000000) /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f0 Cannot send HSM request (use of f0): Operation not permitted /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f1 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f1 f1: (0x00000000) /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f1 Cannot send HSM request (use of f1): Operation not permitted /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f2 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f2 f2: (0x00000000) /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f2 Cannot send HSM request (use of f2): Operation not permitted /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f3 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f3 f3: (0x00000000) /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f3 Cannot send HSM request (use of f3): Operation not permitted </pre> </div></div> <p>It seems because the daemon is not being start up correctly, then I start the daemon again, it works</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@mds tests]# /work/lustre-release_new/lustre-release/lustre/utils/lhsmtool_posix -vvvv --hsm_root=/mnt/lustre --daemon /mnt/lustre-hsm lhsmtool_posix[25558]: action=0 src=(null) dst=(null) mount_point=/mnt/lustre-hsm [root@mds tests]# lhsmtool_posix[25559]: waiting for message from kernel lhsmtool_posix[25559]: copytool fs=lustre archive#=1 item_count=1 lhsmtool_posix[25559]: waiting for message from kernel lhsmtool_posix[25560]: '[0x300000401:0x1:0x0]' action ARCHIVE reclen 72, cookie=0x5399b0c2 lhsmtool_posix[25560]: processing file 'd0/f3' lhsmtool_posix[25560]: archiving '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' lhsmtool_posix[25560]: saving stripe info of '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' in /mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp.lov lhsmtool_posix[25560]: going to copy data from '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' lhsmtool_posix[25560]: Going to copy 1 bytes /mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0 -&gt; /mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp lhsmtool_posix[25560]: data archiving for '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' done lhsmtool_posix[25560]: attr file for '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' lhsmtool_posix[25560]: fsetxattr of 'trusted.hsm' on '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25560]: fsetxattr of 'trusted.link' on '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25560]: fsetxattr of 'trusted.lov' on '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25560]: fsetxattr of 'trusted.lma' on '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25560]: fsetxattr of 'lustre.lov' on '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25560]: xattr file for '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0_tmp' lhsmtool_posix[25560]: symlink '/mnt/lustre/shadow/d0/f3' to '../../0001/0000/0401/0000/0003/0000/0x300000401:0x1:0x0' done lhsmtool_posix[25560]: Action completed, notifying coordinator cookie=0x5399b0c2, FID=[0x300000401:0x1:0x0], hp_flags=0 err=0 lhsmtool_posix[25560]: llapi_hsm_action_end() on '/mnt/lustre-hsm/.lustre/fid/0x300000401:0x1:0x0' ok (rc=0) lhsmtool_posix[25559]: copytool fs=lustre archive#=1 item_count=1 lhsmtool_posix[25559]: waiting for message from kernel lhsmtool_posix[25562]: '[0x340000401:0x1:0x0]' action ARCHIVE reclen 72, cookie=0x5399b0c2 lhsmtool_posix[25562]: processing file 'd0/f0' lhsmtool_posix[25562]: archiving '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' lhsmtool_posix[25562]: saving stripe info of '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' in /mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp.lov lhsmtool_posix[25562]: going to copy data from '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' lhsmtool_posix[25562]: Going to copy 1 bytes /mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0 -&gt; /mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp lhsmtool_posix[25562]: data archiving for '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' done lhsmtool_posix[25562]: attr file for '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' lhsmtool_posix[25562]: fsetxattr of 'trusted.hsm' on '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25562]: fsetxattr of 'trusted.link' on '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25562]: fsetxattr of 'trusted.lov' on '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25562]: fsetxattr of 'trusted.lma' on '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25562]: fsetxattr of 'lustre.lov' on '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25562]: xattr file for '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0_tmp' lhsmtool_posix[25562]: symlink '/mnt/lustre/shadow/d0/f0' to '../../0001/0000/0401/4000/0003/0000/0x340000401:0x1:0x0' done lhsmtool_posix[25562]: Action completed, notifying coordinator cookie=0x5399b0c2, FID=[0x340000401:0x1:0x0], hp_flags=0 err=0 lhsmtool_posix[25562]: llapi_hsm_action_end() on '/mnt/lustre-hsm/.lustre/fid/0x340000401:0x1:0x0' ok (rc=0) lhsmtool_posix[25559]: copytool fs=lustre archive#=1 item_count=1 lhsmtool_posix[25559]: waiting for message from kernel lhsmtool_posix[25563]: '[0x280000400:0x2:0x0]' action ARCHIVE reclen 72, cookie=0x5399b0c2 lhsmtool_posix[25563]: processing file 'd0/f1' lhsmtool_posix[25563]: archiving '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' to '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' lhsmtool_posix[25563]: saving stripe info of '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' in /mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp.lov lhsmtool_posix[25563]: going to copy data from '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' to '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' lhsmtool_posix[25563]: Going to copy 1 bytes /mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0 -&gt; /mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp lhsmtool_posix[25563]: data archiving for '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' to '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' done lhsmtool_posix[25563]: attr file for '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' saved to archive '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' lhsmtool_posix[25563]: fsetxattr of 'trusted.hsm' on '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' rc=0 (Success) lhsmtool_posix[25563]: fsetxattr of 'trusted.link' on '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' rc=0 (Success) lhsmtool_posix[25563]: fsetxattr of 'trusted.lov' on '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' rc=0 (Success) lhsmtool_posix[25563]: fsetxattr of 'trusted.lma' on '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' rc=0 (Success) lhsmtool_posix[25563]: fsetxattr of 'lustre.lov' on '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' rc=0 (Success) lhsmtool_posix[25563]: xattr file for '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' saved to archive '/mnt/lustre/0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0_tmp' lhsmtool_posix[25563]: symlink '/mnt/lustre/shadow/d0/f1' to '../../0002/0000/0400/8000/0002/0000/0x280000400:0x2:0x0' done lhsmtool_posix[25563]: Action completed, notifying coordinator cookie=0x5399b0c2, FID=[0x280000400:0x2:0x0], hp_flags=0 err=0 lhsmtool_posix[25563]: llapi_hsm_action_end() on '/mnt/lustre-hsm/.lustre/fid/0x280000400:0x2:0x0' ok (rc=0) lhsmtool_posix[25559]: copytool fs=lustre archive#=1 item_count=1 lhsmtool_posix[25559]: waiting for message from kernel lhsmtool_posix[25564]: '[0x2c0000401:0x1:0x0]' action ARCHIVE reclen 72, cookie=0x5399b0c2 lhsmtool_posix[25564]: processing file 'd0/f2' lhsmtool_posix[25564]: archiving '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' lhsmtool_posix[25564]: saving stripe info of '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' in /mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp.lov lhsmtool_posix[25564]: going to copy data from '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' lhsmtool_posix[25564]: Going to copy 1 bytes /mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0 -&gt; /mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp lhsmtool_posix[25564]: data archiving for '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' to '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' done lhsmtool_posix[25564]: attr file for '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' lhsmtool_posix[25564]: fsetxattr of 'trusted.hsm' on '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25564]: fsetxattr of 'trusted.link' on '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25564]: fsetxattr of 'trusted.lov' on '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25564]: fsetxattr of 'trusted.lma' on '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25564]: fsetxattr of 'lustre.lov' on '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' rc=0 (Success) lhsmtool_posix[25564]: xattr file for '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' saved to archive '/mnt/lustre/0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0_tmp' lhsmtool_posix[25564]: symlink '/mnt/lustre/shadow/d0/f2' to '../../0001/0000/0401/c000/0002/0000/0x2c0000401:0x1:0x0' done lhsmtool_posix[25564]: Action completed, notifying coordinator cookie=0x5399b0c2, FID=[0x2c0000401:0x1:0x0], hp_flags=0 err=0 lhsmtool_posix[25564]: llapi_hsm_action_end() on '/mnt/lustre-hsm/.lustre/fid/0x2c0000401:0x1:0x0' ok (rc=0) [root@mds tests]# sh -vx test_1.sh cd /mnt/lustre/d0 + cd /mnt/lustre/d0 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f0 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f0 f0: (0x00000009) exists archived, archive_id:1 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f0 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f0 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f1 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f1 f1: (0x00000009) exists archived, archive_id:1 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f1 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f1 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f2 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f2 f2: (0x00000009) exists archived, archive_id:1 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f2 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_archive f3 sleep 2 + sleep 2 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_state f3 f3: (0x00000009) exists archived, archive_id:1 /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f3 + /work/lustre-release_new/lustre-release/lustre/utils/lfs hsm_release f3 [root@mds tests]# </pre> </div></div> <p>So I suspect this is related with daemon, not about striped directory.</p> <p>I don't see any verification that restore either via lfs hsm_restore or by file access works as expected.</p> <p>On current master I see that restore in a striped directory doesn't usually work.</p> <p><a href="http://review.whamcloud.com/10866" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10866</a> add striped dir to sanity-hsm.sh</p> <p>Please note that landing <a href="http://review.whamcloud.com/#/c/10866/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10866/</a> does not mean that this issue has been fixed.</p> <p>Di's patch is at <a href="http://review.whamcloud.com/#/c/10866/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10866/</a>.</p> <p>Patch landed to master.</p> Development Rank 1|hzwggf: Rank (Obsolete) 12900 Severity