Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-5933

FIEMAP: FIEMAP_EXTENT_LAST is not always set

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • Lustre 2.7.0
    • Lustre 2.7.0
    • None
    • 3
    • 16568

    Description

      With the FS_IOC_FIEMAP ioctl, when the exact number of required extent is used, and the file ends with a hole, FIEMAP_EXTENT_LAST is not set like it should.

      Reproducer:

      #include <stdio.h>
      #include <string.h>
      #include <stdlib.h>
      #include <sys/types.h>
      #include <sys/stat.h>
      #include <fcntl.h>
      #include <linux/fs.h>
      #include <linux/fiemap.h>
      #include <sys/ioctl.h>
      
      int main(int argc, char *argv[])
      {
      	struct fiemap *fiemap = NULL;
      	struct fiemap fiemap_init;
      	int fd;
      	size_t extents_count;
      
      	fd = open(argv[1], O_RDONLY);
      
      	/* Query to get the number of extents. */
      	memset(&fiemap_init, 0, sizeof(fiemap_init));
      	fiemap_init.fm_length = ~0;		/* all of it */
      	fiemap_init.fm_flags = FIEMAP_FLAG_SYNC; /* sync file */
      
      	if (ioctl(fd, FS_IOC_FIEMAP, &fiemap_init) < 0) {
      		perror("fiemap 1 failed");
      		return 1;
      	}
      
      	extents_count = fiemap_init.fm_mapped_extents;
      	fprintf(stderr, "(1st) file has %zd extent(s)\n", extents_count);
      
      	if (extents_count == 0) {
      		return 1;
      	}
      
      	/* Workaround for Lustre */
      	//extents_count++;
      
      	fiemap = calloc(1, sizeof(struct fiemap) +
      					sizeof(struct fiemap_extent) * extents_count);
      	if (fiemap == NULL) {
      		perror("calloc failed");
      		return 1;
      	}
      
      	/* Get the extents. */
      	fiemap->fm_length = ~0;		/* the whole file */
      	fiemap->fm_flags = FIEMAP_FLAG_SYNC; /* sync file */
      	fiemap->fm_extent_count = extents_count;
      
      	if (ioctl(fd, FS_IOC_FIEMAP, fiemap) < 0) {
      		perror("fiemap 2 failed");
      		return 1;
      	}
      
      	fprintf(stderr, "(2nd) file has %zd extent(s)\n", extents_count);
      
      	if (fiemap->fm_extents[fiemap->fm_mapped_extents-1].fe_flags & FIEMAP_EXTENT_LAST)
      		printf("Good\n");
      	else
      		printf("Bad\n");
      
      	return 0;
      }
      

      On ext4:

      # echo sdfghjk > tfile
      # ~/tsparse tfile 
      file has 1 extent(s)
      Good
      # truncate -s 10000 tfile 
      # ~/tsparse tfile 
      (1st) file has 1 extent(s)
      (2nd) file has 1 extent(s)
      Good
      

      On Lustre head of tree:

      # echo sdfghjk > tfile
      # ~/tsparse tfile 
      (1st) file has 1 extent(s)
      (2nd) file has 1 extent(s)
      Good
      # truncate -s 1000000 tfile 
      # ~/tsparse tfile 
      file has 1 extent(s)
      Bad
      

      If I uncomment the "hack" in the test code, allocating one extra extent, then it works, although only 1 extent is actually used.

      This is likely to break programs that rely on that flag to check the integrity of the returned mapping.

      I don't have a fix at this time.

      Attachments

        Issue Links

          Activity

            People

              bobijam Zhenyu Xu
              fzago Frank Zago (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: