https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3373 Lustre <p>This ticket intended to track ldiskfs patches work on FC18 3.7 kernel.</p> LU-3373

ldiskfs patches for FC19

Improvement Minor Resolved Fixed Yang Sheng Yang Sheng Tue, 21 May 2013 14:12:08 +0000 Fri, 15 Nov 2019 20:29:00 +0000 Tue, 1 Jul 2014 01:13:48 +0000 Lustre 2.7.0 0 6 <p>FC18 have 3.9.2 kernel already. Why 3.7 kernel is going to be supported?</p> <p>Can you post a link to download the kernel source. I will gladly update the kernel patch in <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a>. This is of interest since one of my goals for 2.5+ is to create a patchless ldiskfs.</p> <p><a href="http://mirrors.med.harvard.edu/fedora/updates/18/SRPMS/kernel-3.9.2-200.fc18.src.rpm" class="external-link" target="_blank" rel="nofollow noopener">http://mirrors.med.harvard.edu/fedora/updates/18/SRPMS/kernel-3.9.2-200.fc18.src.rpm</a></p> <p>I think work start from 3.7 can keep max compatible for further support.</p> <p>First patchset: <a href="http://review.whamcloud.com/#/c/7263/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/7263/</a></p> <p>I'll continue update it after 3.9.5 kernel work done.</p> <p>I think this a little behind the curve. current kernel in fc18 is 3.9.11. In fc19 it's 3.10.4</p> <p>As far as I know everything needed for 3.9 has already landed. Not so true of 3.10.</p> <p>I don't see any mods to ldiskfs autoconf here to automatically choose the right (new) patch series.</p> <p><a href="http://review.whamcloud.com/#/c/7263/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/7263/</a></p> <p>sanity test fail list:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sanity: FAIL: test_17m e2fsck should not report error upon short/long symlink MDT: rc=4 FAIL 17m (43s) sanity: FAIL: test_24A Expected 5000 files, got 5005 (5005 unique) FAIL 24A (5s) sanity: FAIL: test_27k &gt; 4194304 FAIL 27k (1s) sanity: FAIL: test_33a create FAIL 33a (1s) sanity: FAIL: test_36c FAIL 36c (1s) sanity: FAIL: test_36d FAIL 36d (1s) sanity: FAIL: test_102c setstripe failed FAIL 102c (2s) sanity: FAIL: test_103 permissions failed FAIL 103 (17s) sanity: FAIL: test_104b lfs check servers test failed FAIL 104b (1s) sanity: FAIL: test_129 exceeded dir size limit 4096 x 1 4096 : 12288 bytes FAIL 129 (9s) sanity: FAIL: test_133c The counter for destroy on ost was not incremented FAIL 133c (16s) sanity: FAIL: test_180c failed to load module obdecho FAIL 180c (2s) sanity: FAIL: test_183 test_183 failed with 1 FAIL 183 (3s) sanity: FAIL: test_184c concurrent write on /mnt/lustre/d0.sanity/d184/184c/file1 failed FAIL 184c (448s) sanity: FAIL: test_218 multiop failed while creating a file FAIL 218 (3s) sanity: FAIL: test_219 test_219 failed with 1 FAIL 219 (2s) FAIL 228a (8s) </pre> </div></div> <p>With the patches for <a href="https://jira.whamcloud.com/browse/LU-3319" title="Adapt to 3.10 upstream kernel proc_dir_entry change" class="issue-link" data-issue-key="LU-3319"><del>LU-3319</del></a>, <a href="https://jira.whamcloud.com/browse/LU-3974" title="Support for linux 3.11 kernel" class="issue-link" data-issue-key="LU-3974"><del>LU-3974</del></a>, and this ticket we can almost get ldiskfs/osd-ldiskfs to build. As you can see from the below errors most of the errors are proc related which will be addressed for <a href="https://jira.whamcloud.com/browse/LU-3319" title="Adapt to 3.10 upstream kernel proc_dir_entry change" class="issue-link" data-issue-key="LU-3319"><del>LU-3319</del></a>. The other bug deals with the reader -&gt; iterate migrate discussed on <a href="https://jira.whamcloud.com/browse/LU-3974" title="Support for linux 3.11 kernel" class="issue-link" data-issue-key="LU-3974"><del>LU-3974</del></a>. We are almost there <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:66:<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_internal.h:625: error: 'struct lprocfs_static_vars' declared inside parameter list<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_internal.h:625: error: its scope is only this definition or declaration, which is probably not what you want<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c: In function 'osd_ldiskfs_it_fill':<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:4676: error: 'const struct file_operations' has no member named 'readdir'<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c: In function 'osd_mod_init':<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:5806: error: storage size of 'lvars' isn't known<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:5817: error: passing argument 4 of 'class_register_type' from incompatible pointer type<br/> lustre-2.5.50/lustre/include/obd_class.h:89: note: expected 'struct lu_device_type *' but argument is of type 'char *'<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:5817: error: too many arguments to function 'class_register_type'<br/> lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.c:5806: error: unused variable 'lvars'<br/> make<span class="error">&#91;6&#93;</span>: *** <span class="error">&#91;lustre-2.5.50/lustre/osd-ldiskfs/osd_handler.o&#93;</span> Error 1<br/> make<span class="error">&#91;5&#93;</span>: *** <span class="error">&#91;lustre-2.5.50/lustre/osd-ldiskfs&#93;</span> Error 2</p> <p><a href="http://review.whamcloud.com/7948" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7948</a><br/> <a href="http://review.whamcloud.com/7794" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7794</a><br/> <a href="http://review.whamcloud.com/7727" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7727</a></p> <p><a href="http://review.whamcloud.com/8110" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8110</a><br/> <a href="http://review.whamcloud.com/8116" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8116</a></p> <p>Latest sanity status: </p> <p>sanity: FAIL: test_17m e2fsck should not report error upon short/long symlink MDT: rc=4<br/> FAIL 17m (44s)<br/> sanity: FAIL: test_103 permissions failed<br/> FAIL 103 (6s)<br/> sanity: FAIL: test_133c The counter for destroy on ost was not incremented<br/> FAIL 133c (14s)<br/> sanity: FAIL: test_133e Bad write_bytes sum, expected 1376256, got 0<br/> FAIL 133e (3s)</p> <p>I also had a patch for osd-ldiskfs iterate change but mine was unstable. Now I see it was due to me not updating the position field in struct dir_context. Your patch doesn't currently build for me on normal rhel6.4 but I working on fixing that. I should have something ready in the next few hours. Thanks Yang for finding that issue.</p> <p>I think there is still one more call to ldiskfs_journal_start_sb() in osd_handler.c that hasn't been switched to osd_journal_start_sb() by the existing patches. Maybe could be taken care of by a refresh of #7794?</p> <p>Besides that all that's missing is the readdir -&gt; iterate conversion that James has already mentioned. And of course finding all the bugs. Much of the previous hole has been filled in by recent server patch #8232 from James.</p> <p>We're getting very close to having a full ldiskfs server build.</p> <p><a href="http://review.whamcloud.com/8231" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8231</a></p> <p>It lives!!!! Behold ldiskfs on a 3.11.1 kernel. I had to fix up some of the ldiskfs patches but it works.</p> <p>jsimmons@spoon46:~$ uname -r;df<br/> 3.11.1<br/> Filesystem 1K-blocks Used Available Use% Mounted on<br/> unionfs 88763392 62004224 22251520 74% /<br/> none 100 0 100 0% /dev/shm<br/> none 2938844 18352 2920492 1% /tmp<br/> /dev/loop0 133560 1576 118624 2% /tmp/lustre/mds1<br/> /dev/loop1 171080 9380 147700 6% /tmp/lustre/ost1<br/> /dev/loop2 171080 9380 147700 6% /tmp/lustre/ost2<br/> 10.37.248.48@o2ib1:/lustre<br/> 342160 18760 295400 6% /lustre/barry</p> <p>with the refresh of #7794 and the addition of #8231 I can now complete a full server build with ldiskfs on the 3.11.7 kernel that is current in fc19 too.</p> <p>major milestone!!</p> <p>I'm almost afraid to try some functional tests.<br/> James, can you describe the changes you needed in ldiskfs patches? They worked for me as is.</p> <p>Yes with the refresh now all I have to do is set the ldiskfs series manually. Also you need patch <a href="http://review.whamcloud.com/#/c/8237" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8237</a> which I just pushed. I also updated <a href="http://review.whamcloud.com/#/c/8116" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8116</a> locally to test if ext4_map_blocks() is available.</p> <p>I was using Peng's original upstream patch. I see #8237 now.</p> <p>By the way I do have a prototype autoconf patch that selects the ldiskfs series. I haven't pushed it because I'm not sure it's production worthy.</p> <p>Send it my way. I will try it out. Tomorrow I will setup a real file system using a 3.11 kernel to see how it holds up. If it holds up I plan to mount it on our cray test bed and run a bunch of jobs to stress test it.</p> <p>it lives, take II<br/> I'm seeing first signs of life now with ldiskfs in fc19:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@fedora19 tests]# llmount.sh Stopping clients: fedora19 /mnt/lustre (opts:) Stopping clients: fedora19 /mnt/lustre2 (opts:) Loading modules from /usr/lib64/lustre/tests/.. detected 1 online CPUs by sysfs libcfs will create CPU partition based on online CPUs debug=vfstrace rpctrace dlmtrace neterror ha config ioctl super subsystem_debug=all -lnet -lnd -pinger gss/krb5 is not supported quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /tmp/lustre-mdt1 Format ost1: /tmp/lustre-ost1 Format ost2: /tmp/lustre-ost2 Checking servers environments Checking clients fedora19 environments Loading modules from /usr/lib64/lustre/tests/.. detected 1 online CPUs by sysfs libcfs will create CPU partition based on online CPUs debug=vfstrace rpctrace dlmtrace neterror ha config ioctl super subsystem_debug=all -lnet -lnd -pinger gss/krb5 is not supported Setup mgs, mdt, osts Starting mds1: -o loop /tmp/lustre-mdt1 /mnt/mds1 Started lustre-MDT0000 Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1 Started lustre-OST0000 Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2 Started lustre-OST0001 Starting client: fedora19: -o user_xattr,flock fedora19@tcp:/lustre /mnt/lustre Using TIMEOUT=20 seting jobstats to procname_uid Setting lustre.sys.jobid_var from disable to procname_uid Changed after 0s: from '' to 'disable' Waiting 90 secs for update Updated after 9s: wanted 'procname_uid' got 'procname_uid' disable quota as required [root@fedora19 tests]# lfs df UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 133560 1576 118624 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 171080 9380 147700 6% /mnt/lustre[OST:0] lustre-OST0001_UUID 171080 9380 147700 6% /mnt/lustre[OST:1] filesystem summary: 342160 18760 295400 6% /mnt/lustre </pre> </div></div> <p>One odd quirk. lustre-ldiskfs.spec is setup with a Requires: modutils. However there isn't any rpm in fedora that provides modutils. I could only install lustre with a --force option on the command line.</p> <p>Looks like in earlier distro modutils was one of the Provides: in the module-init-tools rpm. The fedora rpm that contains similar files as the old module-init-tools rpm is named kmod, but it has no Provides: modutils in it.</p> <p>Ran several jobs (various IOR, mdtest configurations and a science app) against the 3.11 base lustre file system using ldiskfs. It was successful so I would say we are in pretty good shape.</p> <p>O_LOV_DELAY_CREATE conflict with __O_TMPFILE. Need change it. lod and osp procfs patch need fix some bug. I have commented in gerrit.</p> <p>I think <a href="https://jira.whamcloud.com/browse/LU-4209" title="O_LOV_DELAY_CREATE conflict with __O_TMPFILE" class="issue-link" data-issue-key="LU-4209"><del>LU-4209</del></a> already exists to address the problem with O_LOV_DELAY_CREATE.</p> <p><a href="http://review.whamcloud.com/#/c/8116" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8116</a> looks ready for inspection and possible merger.</p> <p>I see patch 8116 has been abandon. I believe the point of the patch was to limit the port of patches for fc19 support. Do this mean the ldiskfs patches will be updated or will the rhel6.5 patch be enough?</p> <p><a href="http://review.whamcloud.com/#/c/8231" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8231</a> is ready for inspection and possible merger.</p> <p>Patches landed to Master. Final patch <a href="http://review.whamcloud.com/#/c/8116" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8116</a> is being tracked under <a href="https://jira.whamcloud.com/browse/LU-5276" title="ldiskfs patches for FC19" class="issue-link" data-issue-key="LU-5276"><del>LU-5276</del></a></p> Duplicate LU-3228 Related LU-4209 LU-3675 LU-6446 LU-3974 LU-5276 Development Rank 1|hzvrh3: Rank (Obsolete) 8342