https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9212 Lustre <p>Dear Support,</p> <p>One of our customer contacted us because of the following errors:<br/> <span class="error">&#91;root@mds01 ~&#93;</span># dmesg -T | grep -E "opd_pre_status -116|opd_pre_status -115"<br/> <span class="error">&#91;Thu Mar 9 14:50:45 2017&#93;</span> LustreError: 25696:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115<br/> <span class="error">&#91;Thu Mar 9 15:01:44 2017&#93;</span> LustreError: 25753:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115<br/> <span class="error">&#91;Thu Mar 9 15:11:46 2017&#93;</span> LustreError: 12529:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115<br/> <span class="error">&#91;Thu Mar 9 15:22:04 2017&#93;</span> LustreError: 25542:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115<br/> <span class="error">&#91;Thu Mar 9 15:33:14 2017&#93;</span> LustreError: 25248:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST0021-osc-MDT0000: precreate failed opd_pre_status -116<br/> <span class="error">&#91;Thu Mar 9 15:43:56 2017&#93;</span> LustreError: 25656:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115<br/> <span class="error">&#91;Thu Mar 9 15:54:21 2017&#93;</span> LustreError: 25645:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001c-osc-MDT0000: precreate failed opd_pre_status -115</p> <p>I have started focusing on one of the osts that are reporting the error: ost001b.<br/> It is currently on storage06:<br/> <span class="error">&#91;9752632.984562&#93;</span> Lustre: wurfs-OST001c-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001c0000:0x1ad9d9d:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> <span class="error">&#91;9752632.984569&#93;</span> Lustre: Skipped 127 previous similar messages<br/> <span class="error">&#91;9752877.938323&#93;</span> LustreError: 23938:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST0021: unable to precreate: rc = -116<br/> <span class="error">&#91;9752877.938334&#93;</span> LustreError: 23938:0:(ofd_dev.c:1781:ofd_create_hdl()) Skipped 266 previous similar messages<br/> <span class="error">&#91;9752884.305673&#93;</span> mpt2sas1: log_info(0x31120302): originator(PL), code(0x12), sub_code(0x0302)<br/> <span class="error">&#91;9753043.238296&#93;</span> LustreError: 42247:0:(osd_compat.c:593:osd_obj_update_entry()) wurfs-OST001c-osd: the FID <span class="error">&#91;0x580000400:0x2d2c:0x0&#93;</span> is used by two objects: 524529/2876391915 1582/813719656<br/> <span class="error">&#91;9753043.238307&#93;</span> LustreError: 42247:0:(osd_compat.c:593:osd_obj_update_entry()) Skipped 3455 previous similar messages<br/> <span class="error">&#91;9753235.723470&#93;</span> Lustre: wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> <span class="error">&#91;9753235.723478&#93;</span> Lustre: Skipped 130 previous similar messages<br/> <span class="error">&#91;9753429.709498&#93;</span> mpt2sas1: log_info(0x31120302): originator(PL), code(0x12), sub_code(0x0302)<br/> <span class="error">&#91;9753478.487827&#93;</span> LustreError: 42848:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001c: unable to precreate: rc = -115<br/> <span class="error">&#91;9753478.487835&#93;</span> LustreError: 42848:0:(ofd_dev.c:1781:ofd_create_hdl()) Skipped 263 previous similar messages<br/> <span class="error">&#91;9753648.598453&#93;</span> LustreError: 4652:0:(osd_compat.c:593:osd_obj_update_entry()) wurfs-OST001c-osd: the FID <span class="error">&#91;0x580000400:0x2d2c:0x0&#93;</span> is used by two objects: 524529/2876391915 1582/813719656<br/> <span class="error">&#91;9753648.598462&#93;</span> LustreError: 4652:0:(osd_compat.c:593:osd_obj_update_entry()) Skipped 3509 previous similar messages</p> <p>I can see these messages repeated a lot of times.<br/> <span class="error">&#91;root@storage06 ~&#93;</span># mount | grep mapper<br/> /dev/mapper/ost001b on /lustre/ost1b type lustre (ro)<br/> /dev/mapper/ost001c on /lustre/ost1c type lustre (ro)<br/> /dev/mapper/ost001d on /lustre/ost1d type lustre (ro)<br/> /dev/mapper/ost0021 on /lustre/ost21 type lustre (ro)<br/> /dev/mapper/ost0022 on /lustre/ost22 type lustre (ro)<br/> /dev/mapper/ost0023 on /lustre/ost23 type lustre (ro)</p> <p>The customer has run: lctl lfsck_start -M wurfs-OST001b but it didnt help.</p> <p>I suppose that the next step is:<br/> debugfs -c -R "ncheck 524529 1582" /dev//dev/mapper/ost001b <br/> (I wonder if it can be done while the fs is mounted.)</p> [<a href='mailto:root@storage06'>root@storage06</a> ~]# uname -r<br/> 3.10.0-327.3.1.el7_lustre.x86_64<br/> [<a href='mailto:root@storage06'>root@storage06</a> ~]# cat /etc/redhat-release <br/> Scientific Linux release 7.2 (Nitrogen)<br/> LU-9212

Lustre errors on precreation (e -115)

Bug Major Resolved Won't Fix Peter Jones cv_eng Thu, 9 Mar 2017 14:46:14 +0000 Wed, 21 Mar 2018 16:59:42 +0000 Wed, 21 Mar 2018 16:59:42 +0000 0 5 <p>I pressed enter my mistake.</p> <p>However, the next step, i think it would be:<br/> debugfs -c -R "stat &lt;result from previous command&gt;" dev/mapper/ost001b | grep fid<br/> so we can understand which are the files it complains about.</p> <p>Then, mount the OST as ldiskfs mode and remove the files?<br/> Unfortunately seems that the customer has noticed issues, only because the files were being written to the OSTs where there are no errors, so looking at the logs doesnt seem to help.</p> <p>(We have this error for multiple OSTs)</p> <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=cv_eng" class="user-hover" rel="cv_eng">cv_eng</a>,</p> <p>I see the version was selected as 2.8.0. Was that due to filing it originally as an LU ticket (moved over to your CV project now) or is the site really using 2.8.0? Also, can you tell us what site this is supporting?</p> <p>Thanks.<br/> Joe</p> <p>Hello Joe,</p> <p>The customer is currently using Lustre 2.8.0 (no IEEL).<br/> The customer i am talking about is Universiteit van Wageningen.</p> <p>I am not sure i have opened the ticket in the right place.</p> <p>Kind regards,<br/> Andrea</p> <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=cv_eng" class="user-hover" rel="cv_eng">cv_eng</a>,</p> <p>Could your provide the Confirmation of Service (COS) number for this site?</p> <p>Thank you,</p> <p>Brad</p> <p>Dear Brad,</p> <p>I am not able to get it as of now, will try monday.<br/> Is there anything else that we can do meanwhile?</p> <p>Kind regards,</p> <p>Dear Brad, all,</p> <p>Please note that this is the community version of Lustre and there is no support contract paid.</p> <p>Kind regards, </p> <p>Dear all,</p> <p>Keeping you posted.<br/> We have disabled the OSS with the OST and checked the filesystem.<br/> I have no error on the OSS.</p> <p>I have an error on the MDS when i try to create a file on that OST:<br/> <span class="error">&#91;root@node009 brokenosts&#93;</span># touch test2<br/> touch: cannot touch ‘test2’: No space left on device</p> <p>On the mds:<br/> Mar 13 16:51:05 mds01 kernel: LustreError: 25716:0:(osp_precreate.c:1243:osp_precreate_ready_condition()) wurfs-OST001b-osc-MDT0000: precreate failed opd_pre_status -115</p> <p>Shall we schedule a downtime and run a fsck on the mdt volume?</p> <p>Kind regards,</p> <p>You may find that you get more prompt community support from directing such questions to the community mailing list. More people subscribe to those than review tickets in the issue tracking systm.</p> <p>Dear all,</p> <p>Any reply would be appreciated.<br/> The current situation is the following.<br/> We have stopped pacemaker on the storage06 (which is the one that has that resource running):<br/> <span class="error">&#91;root@storage06 log&#93;</span># pcs status | grep 1b<br/> storage-ost001b (ocf::heartbeat:Filesystem): Started storage06.failover.cluster</p> <ul> <li>storage-ost001b_monitor_120000 on storage06.failover.cluster 'not running' (7): call=295, status=complete, exitreason='none'<br/> *</li> </ul> <p>And then we have tried to execute e2fsck -n /dev/mapper/ost001b.<br/> The e2fsck has reported nothing to be repaired.</p> <p>Today, i noticed that there are still errors and we can't create files on this OST:<br/> <span class="error">&#91;Mon Mar 13 18:36:44 2017&#93;</span> LustreError: 42126:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Mon Mar 13 18:46:44 2017&#93;</span> LustreError: 35949:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Mon Mar 13 18:56:44 2017&#93;</span> LustreError: 26996:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Mon Mar 13 19:06:45 2017&#93;</span> LustreError: 26989:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 03:37:13 2017&#93;</span> LustreError: 26995:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 03:47:13 2017&#93;</span> LustreError: 44782:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 03:57:14 2017&#93;</span> LustreError: 35964:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 04:07:14 2017&#93;</span> LustreError: 35964:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 04:17:14 2017&#93;</span> LustreError: 26994:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 04:27:15 2017&#93;</span> LustreError: 27006:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 04:37:15 2017&#93;</span> LustreError: 27006:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 04:47:15 2017&#93;</span> LustreError: 35964:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> <span class="error">&#91;Tue Mar 14 07:07:30 2017&#93;</span> LustreError: 35960:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115</p> <p>Llooking at cat /usr/include/asm-generic/errno.h, seems that error refers to:<br/> #define EINPROGRESS 115 /* Operation now in progress */<br/> #define ESTALE 116 /* Stale file handle */</p> <p>(on some other osts we do have error 116 as well)</p> <p>Will also try to use the mailing list.</p> <p>Kind regards,</p> <p>Dear Peter,</p> <p>UPDATE:<br/> I have noticed that on this OST (wurfs-OST001b) The IO Scrub gets launched every ~7 seconds:<br/> <span class="error">&#91;root@storage06 wurfs-OST001b&#93;</span># cat oi_scrub<br/> name: OI_scrub<br/> magic: 0x4c5fd252<br/> oi_files: 64<br/> status: completed<br/> flags:<br/> param:<br/> time_since_last_completed: 8 seconds<br/> time_since_latest_start: 8 seconds<br/> time_since_last_checkpoint: 8 seconds<br/> latest_start_position: 12<br/> last_checkpoint_position: 30515713<br/> first_failure_position: N/A<br/> checked: 3417<br/> updated: 0<br/> failed: 0<br/> prior_updated: 0<br/> noscrub: 0<br/> igif: 1<br/> success_count: 2526979<br/> run_time: 0 seconds<br/> average_speed: 3417 objects/sec<br/> real-time_speed: N/A<br/> current_position: N/A<br/> lf_scanned: 0<br/> lf_repaired: 0<br/> lf_failed: 0<br/> <span class="error">&#91;root@storage06 wurfs-OST001b&#93;</span># cat oi_scrub <br/> name: OI_scrub<br/> magic: 0x4c5fd252<br/> oi_files: 64<br/> status: completed<br/> flags:<br/> param:<br/> time_since_last_completed: 2 seconds<br/> time_since_latest_start: 2 seconds<br/> time_since_last_checkpoint: 2 seconds<br/> latest_start_position: 12<br/> last_checkpoint_position: 30515713<br/> first_failure_position: N/A<br/> checked: 3417<br/> updated: 0<br/> failed: 0<br/> prior_updated: 0<br/> noscrub: 0<br/> igif: 1<br/> success_count: 2526980<br/> run_time: 0 seconds<br/> average_speed: 3417 objects/sec<br/> real-time_speed: N/A<br/> current_position: N/A<br/> lf_scanned: 0<br/> lf_repaired: 0<br/> lf_failed: 0</p> <p>And, dumping the logs from the ring buffer i see:<br/> 00080000:02000400:24.0:1489665812.888068:0:35949:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:24.0:1489665812.888083:0:35949:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:27.0:1489665812.923388:0:40057:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:27.0:1489665812.923400:0:40057:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1<br/> 00002000:00080000:24.0:1489665822.903706:0:35949:0:(ofd_dev.c:1747:ofd_create_hdl()) wurfs-OST001b: reserve 64 objects in group 0x0 at 26893346<br/> 00100000:10000000:27.0:1489665822.903984:0:40212:0:(osd_scrub.c:660:osd_scrub_prep()) wurfs-OST001b: OI scrub prep, flags = 0x4e<br/> 00100000:10000000:27.0:1489665822.903992:0:40212:0:(osd_scrub.c:278:osd_scrub_file_reset()) wurfs-OST001b: reset OI scrub file, old flags = 0x0, add flags = 0x0<br/> 00100000:10000000:27.0:1489665822.904016:0:40212:0:(osd_scrub.c:1510:osd_scrub_main()) wurfs-OST001b: OI scrub start, flags = 0x4e, pos = 12<br/> 00080000:02000400:24.0:1489665822.904062:0:35949:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:24.0:1489665822.904079:0:35949:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:27.0:1489665822.940373:0:40212:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:27.0:1489665822.940385:0:40212:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1<br/> 00002000:00080000:8.0:1489665832.919771:0:10464:0:(ofd_dev.c:1747:ofd_create_hdl()) wurfs-OST001b: reserve 64 objects in group 0x0 at 26893346<br/> 00100000:10000000:20.0:1489665832.920031:0:40406:0:(osd_scrub.c:660:osd_scrub_prep()) wurfs-OST001b: OI scrub prep, flags = 0x4e<br/> 00100000:10000000:20.0:1489665832.920037:0:40406:0:(osd_scrub.c:278:osd_scrub_file_reset()) wurfs-OST001b: reset OI scrub file, old flags = 0x0, add flags = 0x0<br/> 00100000:10000000:20.0:1489665832.920057:0:40406:0:(osd_scrub.c:1510:osd_scrub_main()) wurfs-OST001b: OI scrub start, flags = 0x4e, pos = 12<br/> 00080000:02000400:8.0:1489665832.920094:0:10464:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:8.0:1489665832.920113:0:10464:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:20.0:1489665832.955088:0:40406:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:20.0:1489665832.955101:0:40406:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1<br/> 00002000:00080000:30.0:1489665842.935720:0:35960:0:(ofd_dev.c:1747:ofd_create_hdl()) wurfs-OST001b: reserve 64 objects in group 0x0 at 26893346<br/> 00100000:10000000:27.0:1489665842.936008:0:40553:0:(osd_scrub.c:660:osd_scrub_prep()) wurfs-OST001b: OI scrub prep, flags = 0x4e<br/> 00100000:10000000:27.0:1489665842.936015:0:40553:0:(osd_scrub.c:278:osd_scrub_file_reset()) wurfs-OST001b: reset OI scrub file, old flags = 0x0, add flags = 0x0<br/> 00100000:10000000:27.0:1489665842.936038:0:40553:0:(osd_scrub.c:1510:osd_scrub_main()) wurfs-OST001b: OI scrub start, flags = 0x4e, pos = 12<br/> 00080000:02000400:30.0:1489665842.936081:0:35960:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:30.0:1489665842.936096:0:35960:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:27.0:1489665842.972129:0:40553:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:27.0:1489665842.972141:0:40553:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1<br/> 00002000:00080000:10.0:1489665852.951770:0:35949:0:(ofd_dev.c:1747:ofd_create_hdl()) wurfs-OST001b: reserve 64 objects in group 0x0 at 26893346<br/> 00100000:10000000:18.0:1489665852.951986:0:40838:0:(osd_scrub.c:660:osd_scrub_prep()) wurfs-OST001b: OI scrub prep, flags = 0x4e<br/> 00100000:10000000:18.0:1489665852.951992:0:40838:0:(osd_scrub.c:278:osd_scrub_file_reset()) wurfs-OST001b: reset OI scrub file, old flags = 0x0, add flags = 0x0<br/> 00100000:10000000:18.0:1489665852.952017:0:40838:0:(osd_scrub.c:1510:osd_scrub_main()) wurfs-OST001b: OI scrub start, flags = 0x4e, pos = 12<br/> 00080000:02000400:10.0:1489665852.952060:0:35949:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:10.0:1489665852.952089:0:35949:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:18.0:1489665852.987792:0:40838:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:18.0:1489665852.987804:0:40838:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1<br/> 00002000:00080000:8.0:1489665862.967664:0:35949:0:(ofd_dev.c:1747:ofd_create_hdl()) wurfs-OST001b: reserve 64 objects in group 0x0 at 26893346<br/> 00100000:10000000:27.0:1489665862.967948:0:41207:0:(osd_scrub.c:660:osd_scrub_prep()) wurfs-OST001b: OI scrub prep, flags = 0x4e<br/> 00100000:10000000:27.0:1489665862.967955:0:41207:0:(osd_scrub.c:278:osd_scrub_file_reset()) wurfs-OST001b: reset OI scrub file, old flags = 0x0, add flags = 0x0<br/> 00100000:10000000:27.0:1489665862.967982:0:41207:0:(osd_scrub.c:1510:osd_scrub_main()) wurfs-OST001b: OI scrub start, flags = 0x4e, pos = 12<br/> 00080000:02000400:8.0:1489665862.968024:0:35949:0:(osd_handler.c:860:osd_fid_lookup()) wurfs-OST001b-os: trigger OI scrub by RPC for <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>, rc = 0 <span class="error">&#91;1&#93;</span><br/> 00002000:00020000:8.0:1489665862.968040:0:35949:0:(ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115<br/> 00100000:10000000:27.0:1489665863.004087:0:41207:0:(osd_scrub.c:758:osd_scrub_post()) wurfs-OST001b: OI scrub post, result = 1<br/> 00100000:10000000:27.0:1489665863.004098:0:41207:0:(osd_scrub.c:1520:osd_scrub_main()) wurfs-OST001b: OI scrub: stop, pos = 30515713: rc = 1</p> <p>I tried to see where that FID leads but seems that the file doesnt actually exist;<br/> (The customer has moved everything away from this osts)<br/> <span class="error">&#91;root@nfs01 ~&#93;</span># lfs fid2path wurfs "<span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>"<br/> ioctl err -22: Invalid argument (22)<br/> fid2path: error on FID <span class="error">&#91;0x1001b0000:0x19a5c22:0x0&#93;</span>: Invalid argument</p> <p>Any ideas?</p> <p>Saw something similar on master b3541:</p> <p><a href="https://testing.hpdd.intel.com/test_sets/7d9a7bb2-0a12-11e7-8c9f-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/7d9a7bb2-0a12-11e7-8c9f-5254006e85c2</a><br/> <a href="https://jira.whamcloud.com/browse/LU-9277" title="ost-pools test_19: createmany /mnt/lustre/d19.ost-pools/dir1/f19.ost-pools failed!" class="issue-link" data-issue-key="LU-9277"><del>LU-9277</del></a>: (ofd_dev.c:1776:ofd_create_hdl()) lustre-OST0002: unable to precreate: rc = -30<br/> <a href="https://jira.whamcloud.com/browse/LU-9212" title="Lustre errors on precreation (e -115)" class="issue-link" data-issue-key="LU-9212"><del>LU-9212</del></a>: (ofd_dev.c:1781:ofd_create_hdl()) wurfs-OST001b: unable to precreate: rc = -115</p> <p>Unsupported site running older version</p> Development Rank 1|hzz6e7: Rank (Obsolete) 9223372036854775807 Severity