[#LU-231] sanity test_170: FAIL: expected 248 bad lines, but got 249

[LU-231] sanity test_170: FAIL: expected 248 bad lines, but got 249 Created: 22/Apr/11 Updated: 28/Jun/11 Resolved: 24/May/11
Status:	Resolved
Project:	Lustre
Component/s:	None
Affects Version/s:	Lustre 1.8.6
Fix Version/s:	Lustre 2.1.0, Lustre 1.8.6

Type:

Bug

Priority:

Minor

Reporter:

Jian Yu

Assignee:

Lai Siyao

Resolution:

Fixed

Votes:

Labels:

None

Environment:

Branch: b1_8 (Revision: c5c2986be490b2fbceb4b38d6c983d279f4bbcf8)
Distro/Arch: RHEL6/x86_64 (patchless client), RHEL5/x86_64 (server)
Network: tcp

Severity:

Bugzilla ID:

20,569

Rank (Obsolete):

5023

Description

sanity test 170 failed as follows:

== test 170: test lctl df to handle corrupted log ===================== == 03:15:15
 sanity test_170: @@@@@@ FAIL: expected 248 bad lines, but got 249 
Dumping lctl log to /home/yujian/test_logs/2011-04-21/021725/sanity.test_170.*.1303380918.log
tar: Removing leading `/' from member names
/home/yujian/test_logs/2011-04-21/021725/sanity-1303380918.tar.bz2
 sanity test_170: @@@@@@ FAIL: test_170 failed with 1 
Dumping lctl log to /home/yujian/test_logs/2011-04-21/021725/sanity.test_170.*.1303380945.log
tar: Removing leading `/' from member names
/home/yujian/test_logs/2011-04-21/021725/sanity-1303380945.tar.bz2
Resetting fail_loc on all nodes...done.

Maloo report: https://maloo.whamcloud.com/test_sets/d06b18ee-6c23-11e0-b32b-52540025f9af

This is an known issue on b1_8 as bug 20569.

Comments

Comment by Prakash Surya (Inactive) [ 04/May/11 ]

This might not be isolated to the b1_8 branch. I seem to be running into this same issue while single node testing on master/2.x

Maloo report: https://maloo.whamcloud.com/test_sets/95559c2a-7678-11e0-9b1b-52540025f9af

Comment by Sarah Liu [ 05/May/11 ]

got the same error on RHEL6
https://maloo.whamcloud.com/test_sets/0ebe02d6-77ab-11e0-9b1b-52540025f9af

Comment by Peter Jones [ 06/May/11 ]

This keeps coming up in testing. Lai, could you please dig in to find out what is happening here? Thanks

Comment by Lai Siyao [ 10/May/11 ]

This is a bit interesting: when the length of malformed line is less than sizeof(struct ptldebug_header), it's not counted into bad lines if this line is at the end of the log file.

I will fix the test script to reflect this.

Comment by Lai Siyao [ 10/May/11 ]

static int parse_buffer(int fdin, int fdout)
{
        ...
        readhdr:
                rc = read(fdin, ptr, count);
                if (rc <= 0)
                         goto print;
        ...
                if (hdr->ph_len > 4094 ||       /* is this header bogus? */
                    hdr->ph_cpu_id > 65536 ||
                    hdr->ph_stack > 65536 ||
                    hdr->ph_sec < (1 << 30) ||
                    hdr->ph_usec > 1000000000 ||
                    hdr->ph_line_num > 65536) {
                        ...
                        if (HDR_SIZE - count > 0) {
                                int left = HDR_SIZE - count;

                                memmove(buf, buf + count, left);
                                ptr = buf + left;

                                goto readhdr;
                        }
        ...
}

return value of retried readhdr may be 0 for the case described above, and this malformed line is silently ignored.

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » i686,client,el5,ofa #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » i686,client,el5,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,ubuntu1004,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,ofa #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,ofa #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el6,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » i686,server,el5,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,client,el5,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » i686,server,el5,ofa #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » x86_64,server,el5,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Build Master (Inactive) [ 11/May/11 ]

Integrated in lustre-b1_8 » i686,client,el6,inkernel #47
~~LU-231~~ sanity_test_170: FAIL: expected 248 bad lines, but got 249

Johann Lombardi : 2ecce620a67478365c2d0c57ca580c32914c09fe
Files :

lustre/tests/sanity.sh

Comment by Peter Jones [ 11/May/11 ]

Is this same change needed on master?

Comment by Prakash Surya (Inactive) [ 11/May/11 ]

Is this same change needed on master?

I have been running into this issue on master, so I would say yes.

Comment by Lai Siyao [ 11/May/11 ]

Peter, I will port it to master also.