[LU-9442] OST unable to precreate new objects after reaching 0xFFFFFFFF Created: 03/May/17  Updated: 03/Aug/17  Resolved: 19/Jul/17

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.10.1, Lustre 2.11.0

Type: Bug Priority: Major
Reporter: Sergey Cheremencev Assignee: WC Triage
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by LU-9631 sanity-lfsck test_18a: Expect 3 fixed... Resolved
Related
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

Issue could be simple reproduced changing lov_objid and LAST_ID to value near with 0xFFFFFFFF.
Example:

[root@devvm-2 tests]# cd /mnt/lustre
[root@devvm-2 lustre]# mkdir test1
[root@devvm-2 lustre]# lfs setstripe -i 0 -c 1 test1
[root@devvm-2 lustre]# touch test1/asdf
[root@devvm-2 lustre]# touch test1/qwerty
[root@devvm-2 lustre]# cd
[root@devvm-2 ~]# umount /mnt/lustre-mds
lustre-mds1/ lustre-mds2/ 
[root@devvm-2 ~]# umount /mnt/lustre-mds1/
[root@devvm-2 ~]# umount /mnt/lustre-ost1/
[root@devvm-2 ~]# mount -t ldiskfs -o loop /tmp/lustre-mdt1 /mnt/mds
[root@devvm-2 ~]# mount -t ldiskfs -o loop /tmp/lustre-ost1 /mnt/ost
[root@devvm-2 ~]# hexedit /mnt/mds/lov_objid 
[root@devvm-2 ~]# hexedit /mnt/ost/O/0/LAST_ID 
[root@devvm-2 ~]# hexdump -C /mnt/mds/lov_objid 
00000000  f5 ff ff ff 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000010
[root@devvm-2 ~]# hexdump -C /mnt/ost/O/0/LAST_ID 
00000000  f5 ff ff ff 00 00 00 00                           |........|
00000008
[root@devvm-2 ~]# umount /mnt/mds
[root@devvm-2 ~]# umount /mnt/ost
[root@devvm-2 ~]# mount -t lustre -o loop /tmp/lustre-mdt1 /mnt/mds
[root@devvm-2 ~]# mount -t lustre -o loop /tmp/lustre-ost1 /mnt/lustre-ost1/
[root@devvm-2 test1]# for i in `seq 1 10`; do touch file$i; done
[root@devvm-2 test1]# dmesg | tail -n 20
...
Lustre: lustre-OST0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
Lustre: lustre-OST0000: trigger OI scrub by RPC for the [0x100000001:0x0:0x0] with flags 0x4a, rc = 0
LustreError: 4279:0:(ofd_dev.c:1775:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -115
LustreError: 4263:0:(osp_precreate.c:619:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -115
LustreError: 4263:0:(osp_precreate.c:1273:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -115
LustreError: 4351:0:(ofd_dev.c:1775:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -115


According to OST's lustre logs it fails on following:

00002000:00000040:5.0:1493820390.271480:0:4351:0:(ofd_dev.c:416:ofd_object_init()) object init, fid = [0x100000001:0x0:0x0]
00080000:00000010:5.0:1493820390.271486:0:4351:0:(osd_handler.c:338:osd_object_alloc()) kmalloced 'mo': 200 at ffff8800365270c0.
00080000:00000001:5.0:1493820390.271489:0:4351:0:(osd_handler.c:993:osd_fid_lookup()) Process entered
00000001:00000001:5.0:1493820390.271491:0:4351:0:(osd_compat.c:1285:osd_obj_spec_lookup()) Process entered
00000001:00000001:5.0:1493820390.271492:0:4351:0:(osd_compat.c:875:osd_seq_load()) Process entered
00000001:00000001:5.0:1493820390.271493:0:4351:0:(osd_compat.c:883:osd_seq_load()) Process leaving (rc=18446612135418184448 : -131938291367168 : ffff8800b902c300)
00000001:00000001:5.0:1493820390.271516:0:4351:0:(osd_compat.c:1320:osd_obj_spec_lookup()) Process leaving (rc=0 : 0 : 0)
00080000:00000001:5.0:1493820390.271517:0:4351:0:(osd_handler.c:523:osd_iget_check()) Process entered
00080000:00000001:5.0:1493820390.271518:0:4351:0:(osd_handler.c:648:osd_iget_check()) Process leaving via put (rc=0 : 0 : 0x0)
00080000:00000001:5.0:1493820390.271519:0:4351:0:(osd_handler.c:727:osd_check_lma()) Process entered
00080000:00000001:5.0:1493820390.271524:0:4351:0:(osd_handler.c:791:osd_check_lma()) Process leaving (rc=18446744073709551538 : -78 : ffffffffffffffb2)



 Comments   
Comment by Sergey Cheremencev [ 15/May/17 ]

The problem is understood. In the nearest days I will push the patch.

So don't spend time on it.

Comment by Gerrit Updater [ 22/May/17 ]

Sergey Cheremencev (sergey.cheremencev@seagate.com) uploaded a new patch: https://review.whamcloud.com/27225
Subject: LU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2594290c6d550a012857b0e6a1f80a50f3b89c36

Comment by Gerrit Updater [ 19/Jul/17 ]

Oleg Drokin (oleg.drokin@intel.com) merged in patch https://review.whamcloud.com/27225/
Subject: LU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2dde01f1edac9e330f853c3ffee64eb43d82b7c1

Comment by Peter Jones [ 19/Jul/17 ]

Landed for 2.11

Comment by Gerrit Updater [ 26/Jul/17 ]

Minh Diep (minh.diep@intel.com) uploaded a new patch: https://review.whamcloud.com/28227
Subject: LU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: eab600a7927c2e45e0dde2400d31f9bf58aed2a9

Comment by Gerrit Updater [ 03/Aug/17 ]

John L. Hammond (john.hammond@intel.com) merged in patch https://review.whamcloud.com/28227/
Subject: LU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: baa1ce2e0bbcda59d7a4acb7e67cebde402f9d95

Generated at Sat Feb 10 02:26:14 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.