[LU-11186] Cannot create files on MDT1, ofd_precreate_objects()) snx11205-OST0001:0xffffffdd:11811161088 hit the OBIF_MAX_OID (1<<32)!

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.12.0, Lustre 2.10.7
• Affects Version/s: Lustre 2.11.0
• Labels: patch
• Severity: 3
• Assignee: Alexander Boyko
• Reporter: Alexander Boyko

Description

Jul  7 03:27:16 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xffffffdd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:16 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:18 snx11205n005 kernel: LustreError: 8958:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:18 snx11205n005 kernel: LustreError: 8958:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:662:osp_precreate_send()) snx11205-OST0001-osc-MDT0001: precreate fid [0x2c0000400:0xfffffffc:0x0] < local used fid [0x2c0000400:0xfffffffc:0x0]: rc = -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:1287:osp_precreate_thread()) snx11205-OST0001-osc-MDT0001: cannot precreate objects: rc = -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 13467:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 9294:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:22 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:22 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:662:osp_precreate_send()) snx11205-OST0001-osc-MDT0001: precreate fid [0x2c0000400:0xfffffffc:0x0] < local used fid [0x2c0000400:0xfffffffc:0x0]: rc = -116
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:1287:osp_precreate_thread()) snx11205-OST0001-osc-MDT0001: cannot precreate objects: rc = -116
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 9305:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:32 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:32 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28Jul  7 03:27:16 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xffffffdd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:16 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:18 snx11205n005 kernel: LustreError: 8958:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:18 snx11205n005 kernel: LustreError: 8958:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:662:osp_precreate_send()) snx11205-OST0001-osc-MDT0001: precreate fid [0x2c0000400:0xfffffffc:0x0] < local used fid [0x2c0000400:0xfffffffc:0x0]: rc = -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:1287:osp_precreate_thread()) snx11205-OST0001-osc-MDT0001: cannot precreate objects: rc = -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 13467:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:18 snx11205n003 kernel: LustreError: 9294:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:22 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:22 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:662:osp_precreate_send()) snx11205-OST0001-osc-MDT0001: precreate fid [0x2c0000400:0xfffffffc:0x0] < local used fid [0x2c0000400:0xfffffffc:0x0]: rc = -116
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 24598:0:(osp_precreate.c:1287:osp_precreate_thread()) snx11205-OST0001-osc-MDT0001: cannot precreate objects: rc = -116
Jul  7 03:27:22 snx11205n003 kernel: LustreError: 9305:0:(osp_precreate.c:1339:osp_precreate_ready_condition()) snx11205-OST0001-osc-MDT0001: precreate failed opd_pre_status -116
Jul  7 03:27:32 snx11205n005 kernel: LustreError: 9052:0:(ofd_objects.c:217:ofd_precreate_objects()) snx11205-OST0001:0xfffffffd:11811161088 hit the OBIF_MAX_OID (1<<32)!
Jul  7 03:27:32 snx11205n005 kernel: LustreError: 9052:0:(ofd_dev.c:1764:ofd_create_hdl()) snx11205-OST0001: unable to precreate: rc = -28

the reproducer
I`m changing manual last id to the end of range for ost1 and mdt2 for a less file creations.

 MDSCOUNT=2 sh lustre/tests/llmount.sh
  lfs mkdir -i1 -c1 /mnt/lustre/mdt2
 lfs setstripe -i0 -c1 /mnt/lustre/mdt2
  lustre/tests/createmany -o /mnt/lustre/mdt2/file_ 100
 MDSCOUNT=2 sh lustre/tests/llmountcleanup.sh
mount -o loop -t ldiskfs /tmp/lustre-ost1 /mnt/lustre
hexedit /mnt/lustre/O/280000400/LAST_ID   /change zeroes to ff, so it looks like 83 ff ff ff 
 umount /mnt/lustre
mount -o loop -t ldiskfs /tmp/lustre-mdt2 /mnt/lustre
 hexedit /mnt/lustre/lov_objid    //change zeroes to ff at first u32, looks like 63 ff ff ff
 umount /mnt/lustre
 NOFORMAT=1 MDSCOUNT=2 sh lustre/tests/llmount.sh
 lustre/tests/createmany -o /mnt/lustre/mdt2/file1_ 100
 lustre/tests/createmany -o /mnt/lustre/mdt2/file2_ 100

Issue Links

• is duplicated by: LU-6417 Test cases for sequence roll over.