Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-17151

sanity: test_411b Error: '(3) failed to write successfully'

Details

    • 3
    • 9223372036854775807

    Description

      This issue was created by maloo for Serguei Smirnov <ssmirnov@ddn.com>

      This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/2a3fbe0b-f784-4875-bd67-6ab32aa223a3

      Test session details:
      clients: https://build.whamcloud.com/job/lustre-reviews/99011 - 4.18.0-425.10.1.el8_7.aarch64
      servers: https://build.whamcloud.com/job/lustre-reviews/99011 - 4.18.0-477.21.1.el8_lustre.x86_64

      sanity test_411b: @@@@@@ FAIL: (3) failed to write successfully
      Trace dump:
      = /usr/lib64/lustre/tests/test-framework.sh:6700:error()
      = /usr/lib64/lustre/tests/sanity.sh:27545:test_411b()
      = /usr/lib64/lustre/tests/test-framework.sh:7040:run_one()
      = /usr/lib64/lustre/tests/test-framework.sh:7096:run_one_logged()
      = /usr/lib64/lustre/tests/test-framework.sh:6926:run_test()
      = /usr/lib64/lustre/tests/sanity.sh:27592:main()
      Dumping lctl log to /autotest/autotest-2/2023-09-27/lustre-reviews_review-ldiskfs-dne-arm_99011_29_0b05909e-9d3c-46a3-9f81-125f5c37cc5d//sanity.test_411b.*.1695797605.log
      CMD: trevis-108vm17.trevis.whamcloud.com,trevis-108vm18,trevis-72vm4,trevis-72vm5,trevis-72vm6 /usr/sbin/lctl dk > /autotest/autotest-2/2023-09-27/lustre-reviews_review-ldiskfs-dne-arm_99011_29_0b05909e-9d3c-46a3-9f81-125f5c37cc5d//sanity.test_411b.debug_log.$(hostname -s).1695797605.log;
      dmesg > /autotest/autotest-2/2023-09-27/lustre-reviews_review-ldiskfs-dne-arm_99011_29_0b05909e-9d3c-46a3-9f81-125f5c37cc5d//sanity.test_411b.dmesg.$(hostname -s).1695797605.log
      cache 19660800
      rss 0
      rss_huge 0
      shmem 0
      mapped_file 0
      dirty 2883584
      writeback 0
      swap 2949120
      pgpgin 21206
      pgpgout 20906
      pgfault 4034
      pgmajfault 417
      inactive_anon 0
      active_anon 0
      inactive_file 18022400
      active_file 1638400
      unevictable 0
      hierarchical_memory_limit 268435456
      hierarchical_memsw_limit 9223372036854710272
      total_cache 19660800
      total_rss 0
      total_rss_huge 0
      total_shmem 0
      total_mapped_file 0
      total_dirty 2883584
      total_writeback 0
      total_swap 2949120
      total_pgpgin 21206
      total_pgpgout 20906
      total_pgfault 4034
      total_pgmajfault 417
      total_inactive_anon 0
      total_active_anon 0
      total_inactive_file 18022400
      total_active_file 1638400
      total_unevictable 0

      Attachments

        Issue Links

          Activity

            [LU-17151] sanity: test_411b Error: '(3) failed to write successfully'

            Yes, it is using tmpfs on the VM host for the MDT since August or so. It is exported to the VM guest as a block device. The OSTs are still HDD I believe.

            adilger Andreas Dilger added a comment - Yes, it is using tmpfs on the VM host for the MDT since August or so. It is exported to the VM guest as a block device. The OSTs are still HDD I believe.

            do I understand correctly that the all last runs were tmpfs-based?

            bzzz Alex Zhuravlev added a comment - do I understand correctly that the all last runs were tmpfs-based?
            qian_wc Qian Yingjin added a comment -

            Should we increase the memory limit value in the test script or enforce the test only running on real environment not VM?

            qian_wc Qian Yingjin added a comment - Should we increase the memory limit value in the test script or enforce the test only running on real environment not VM?

            This is failing quite a bit on master right now.

            paf0186 Patrick Farrell added a comment - This is failing quite a bit on master right now.

            It looks like those failures are despite patch https://review.whamcloud.com/52610 "LU-17151 tests: increase sanity/411b memory limit" having landed.

            adilger Andreas Dilger added a comment - It looks like those failures are despite patch https://review.whamcloud.com/52610 " LU-17151 tests: increase sanity/411b memory limit " having landed.
            adilger Andreas Dilger added a comment - It looks like this is still failing fairly regularly in the past week: https://testing.whamcloud.com/search?status%5B%5D=FAIL&test_set_script_id=f9516376-32bc-11e0-aaee-52540025f9ae&sub_test_script_id=e877b2b8-1ca3-4655-836b-81fb1693f948&start_date=2023-11-13&end_date=2023-11-20&source=sub_tests#redirect

            There is another failure on x86_64, on the first dd invocation in the test, on latest master that includes this fix at https://testing.whamcloud.com/test_sets/36fdecc1-7e58-410d-b1eb-7345e901b7e5.

            nangelinas Nikitas Angelinas added a comment - There is another failure on x86_64, on the first dd invocation in the test, on latest master that includes this fix at https://testing.whamcloud.com/test_sets/36fdecc1-7e58-410d-b1eb-7345e901b7e5 .
            pjones Peter Jones added a comment -

            Landed for 2.16

            pjones Peter Jones added a comment - Landed for 2.16

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/52610/
            Subject: LU-17151 tests: increase sanity/411b memory limit
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 0e878390e1c8c5883bccd01758392eaa16a67f31

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/52610/ Subject: LU-17151 tests: increase sanity/411b memory limit Project: fs/lustre-release Branch: master Current Patch Set: Commit: 0e878390e1c8c5883bccd01758392eaa16a67f31

            Can we land this since this issue is very common.

            simmonsja James A Simmons added a comment - Can we land this since this issue is very common.

            "Timothy Day <timday@amazon.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52610
            Subject: LU-17151 tests: increase 411b mem limit for arm
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 8444e3f8d5d467ec7653c713529806d6123a449c

            gerrit Gerrit Updater added a comment - "Timothy Day <timday@amazon.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52610 Subject: LU-17151 tests: increase 411b mem limit for arm Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 8444e3f8d5d467ec7653c713529806d6123a449c

            People

              qian_wc Qian Yingjin
              maloo Maloo
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: