[LU-15882] Lustre 2.15 GPUDirect Testing IO failed Created: 23/May/22  Updated: 24/May/22

Status: Open
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.15.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Oleg Kulachenko (Inactive) Assignee: WC Triage
Resolution: Unresolved Votes: 0
Labels: None
Environment:

NVIDIA DGX A100


Attachments: Text File build_8-20220523_2138.log     Text File cufile_sanity_1_20220523_2202.log     Text File gds_8-20220523_2138.log     Text File testcases_1_20220523_2139.log     Text File testcases_sparse_1_20220523_2202.log    
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

gds sanity test found failed I/O:

 

/usr/local/gds/tools/gdsio -V -D /data/sanity/tests// -d 0  -w 8 -s 1G -i 32K:1024K:1K -x 0 -I 1 -o 1
Error: IO failed stopping traffic, fd :51 ret:-5 errno :1
Error: IO failed stopping traffic, fd :49 ret:-5 errno :1
Error: IO failed stopping traffic, fd :52 ret:-5 errno :1
Error: IO failed stopping traffic, fd :56 ret:-5 errno :1
Error: IO failed stopping traffic, fd :54 ret:-5 errno :1
Error: IO failed stopping traffic, fd :55 ret:-5 errno :1
FAILED

 

dmesg:

[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5
[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5
[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5
[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5
[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5
[Mon May 23 15:43:05 2022] nvidia-fs:write unable to flush dirty pages :-5 

 

**************************************************
Testsuite : 181 / 182 tests passed
done tests:Mon May 23 22:02:56 UTC 2022 

ddn@a100-01:~$ lctl get_param version
version=2.15.50_13_gc524079_dirty

NVIDIA-SMI 515.43.04    

Driver Version: 515.43.04    

CUDA Version: 11.7

Kernel: 5.4.0-109-generic

 


Generated at Sat Feb 10 03:22:06 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.