Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14713

Process hung with waiting for mmap_sem

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.15.0
    • Lustre 2.12.7, Lustre 2.12.8
    • None
    • 3
    • 9223372036854775807

    Description

      Write and Truncate IO will serialized on ll_trunc_sem::ll_trunc_{readers|waiters}, if one process quit abruptly (be killed), the other will keep waiting for the semaphore (task state be set as TASK_INTERRUPTIBLE):

       INFO: task a.out:109684 blocked for more than 120 seconds.
      Tainted: G          IOE    --------- -  - 4.18.0-240.15.1.el8_3.x86_64 #1
 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
 Call Trace:
 __schedule+0x2a6/0x700
 schedule+0x38/0xa0
 trunc_sem_down_read+0xa6/0xb0 [lustre]
 vvp_io_write_start+0x107/0xb80 [lustre]
 cl_io_start+0x59/0x110 [obdclass]
 cl_io_loop+0x9a/0x1e0 [obdclass]
 ll_file_io_generic+0x380/0xb10 [lustre]
 ll_file_write_iter+0x136/0x5a0 [lustre]
 new_sync_write+0x124/0x170
 vfs_write+0xa5/0x1a0
 ksys_write+0x4f/0xb0
 do_syscall_64+0x5b/0x1a0

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-15397 LustreError: 4585:0:(llite_mmap.c:61:our_vma()) ASSERTION( !down_write_trylock(&mm->mmap_sem) ) failed

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            [LU-14713] Process hung with waiting for mmap_sem
            ys Yang Sheng made changes -
            Link New: This issue is duplicated by DDN-2284 [ DDN-2284 ]
            eaujames Etienne Aujames made changes -
            Affects Version/s New: Lustre 2.12.8 [ 15093 ]
            Affects Version/s New: Lustre 2.12.7 [ 14793 ]
            ys Yang Sheng made changes -
            Link New: This issue is related to DDN-2891 [ DDN-2891 ]
            ys Yang Sheng made changes -
            Link New: This issue is related to LU-15397 [ LU-15397 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to EX-4212 [ EX-4212 ]
            pjones Peter Jones made changes -
            Fix Version/s New: Lustre 2.15.0 [ 14791 ]
            Resolution New: Fixed [ 1 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            pjones Peter Jones made changes -
            Link Original: This issue is related to JFC-21 [ JFC-21 ]
            pjones Peter Jones made changes -
            Link New: This issue is related to JFC-21 [ JFC-21 ]
            adilger Andreas Dilger made changes -
            Description Original: Write and Truncate IO will serialized on {{ll_trunc_sem::ll_trunc_{readers|waiters}}}, if one process quit abruptly (be killed), the other will keep waiting for the semaphore (task state be set as TASK_INTERRUPTIBLE):
            {noformat}
             INFO: task a.out:109684 blocked for more than 120 seconds.
                  Tainted: G IOE --------- - - 4.18.0-240.15.1.el8_3.x86_64 #1
             "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
             Call Trace:
             __schedule+0x2a6/0x700
             schedule+0x38/0xa0
             trunc_sem_down_read+0xa6/0xb0 [lustre]
             vvp_io_write_start+0x107/0xb80 [lustre]
             cl_io_start+0x59/0x110 [obdclass]
             cl_io_loop+0x9a/0x1e0 [obdclass]
             ll_file_io_generic+0x380/0xb10 [lustre]
             ll_file_write_iter+0x136/0x5a0 [lustre]
             new_sync_write+0x124/0x170
             vfs_write+0xa5/0x1a0
             ksys_write+0x4f/0xb0
             do_syscall_64+0x5b/0x1a0
            {noformat}             		New: Write and Truncate IO will serialized on {{ll_trunc_sem::ll_trunc_\{readers|waiters\}}}, if one process quit abruptly (be killed), the other will keep waiting for the semaphore (task state be set as {{TASK_INTERRUPTIBLE}}):
            {noformat}
             INFO: task a.out:109684 blocked for more than 120 seconds.
                  Tainted: G IOE --------- - - 4.18.0-240.15.1.el8_3.x86_64 #1
             "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
             Call Trace:
             __schedule+0x2a6/0x700
             schedule+0x38/0xa0
             trunc_sem_down_read+0xa6/0xb0 [lustre]
             vvp_io_write_start+0x107/0xb80 [lustre]
             cl_io_start+0x59/0x110 [obdclass]
             cl_io_loop+0x9a/0x1e0 [obdclass]
             ll_file_io_generic+0x380/0xb10 [lustre]
             ll_file_write_iter+0x136/0x5a0 [lustre]
             new_sync_write+0x124/0x170
             vfs_write+0xa5/0x1a0
             ksys_write+0x4f/0xb0
             do_syscall_64+0x5b/0x1a0
            {noformat}
            pjones Peter Jones made changes -
            Link New: This issue is related to DDN-2090 [ DDN-2090 ]

            People

              bobijam Zhenyu Xu
              bobijam Zhenyu Xu
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: