Whamcloud Community JIRA

Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us

9.4.14

940014

05-12-2023

[LU-1812] 3.6/FC18 Server Patches https://jira.whamcloud.com/browse/LU-1812 Lustre LU-1812

3.6/FC18 Server Patches

New Feature Major Resolved Fixed Lai Siyao Jodi Levi Fri, 31 Aug 2012 16:14:54 +0000 Tue, 11 Jun 2013 20:58:38 +0000 Fri, 7 Jun 2013 12:11:12 +0000 Lustre 2.4.0 Lustre 2.5.0 Lustre 2.4.0 Lustre 2.5.0 0 14 This work can be divided into several parts: <ul> <li>finish patchless server (<a href="https://jira.whamcloud.com/browse/LU-20" title="patchless server kernel" class="issue-link" data-issue-key="LU-20"><del>LU-20</del></a>) or port remaining kernel patches (8 patches) to FC17.</li> <li>port ldiskfs patches (40 patches) to support FC17 ext4.</li> <li>port server code to FC17, and this implies client has support FC17 already.</li> </ul> There are patches for SLES11 SP2 (3.0.38) at <a href="http://gauss.credativ.com/~mne/lustre/patchsets/" class="external-link" target="_blank" rel="nofollow noopener">http://gauss.credativ.com/~mne/lustre/patchsets/</a>, which may be a good starting point for the 3.5.2 ldiskfs support. Well, they will need to sign off on their usage for us to be able to land them so we should not look at that code unless they push it into gerrit. I have been in touch with them about this but have not heard back. I'm willing to pick this up. Now I don't have a test bed with SP2 images but I can use a SP2 build box with extra disks. Can you validate ldisk without a lustre file system? It should be possible to mount and use ldiskfs as a local filesystem, though we never test it that way so there may be some unforeseen bugs. It would also be possible to build the lustre code against the newer kernels before ldiskfs is available by disabling the ldiskfs code and instead building against ZFS. First draft patch at <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> i have a set of patches for ldiskfs for SLES11SP2 that i produced independent of the above (since i didn't know about it). any interest? i haven't tested yet since i am still working toward getting a server built. My patch is for basic kernel support not ldiskfs. Yes I'm interested and I will test it. I like to base it against Chris work for <a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a>. Besides the current out standing patches we have some work left for the osd-ldiskfs layer. The first problem seems to be a bit easier to tackle. The api for fsync has changed between kernel versions with it excepting 2, 3 and 4 args. The 2 and 3 argument version seems to be easy to handle but the 4 arugment requires the start and end loff_t for the file. So we have static int osd_object_sync(const struct lu_env *env, struct dt_object *dt) { ... mutex_lock(&inode->i_mutex); #ifdef HAVE_FILE_FSYNC_4ARGS rc = file->f_op->fsync(file, start, end, 0); #elif defined(HAVE_FILE_FSYNC_2ARGS) rc = file->f_op->fsync(file, 0); #else rc = file->f_op->fsync(file, dentry, 0); #endif mutex_unlock(&inode->i_mutex); RETURN(rc); } The question is where do we get the start and end using the osd api? Now the second issue is the quota api is very very different in newer kernels. For example all the vfs_dq_* functions have been removed. Reintroducing them does not seem to be the right approach. with regard to ->fsync() i think the right thing is to simply call vfs_fsync(). inside it does vfs_fsync_range(file, 0, LLONG_MAX, datasync). with regard to ldiskfs, i got an ldiskfs from sles11sp2 to pass small acceptance. the hardest part is that dentry has grown a new member d_sb which is used instead inode->i_sb in some of the calls. so it isnt enough to just set dentry->d_inode to inode before entering ldiskfs from the osd-ldiskfs layer. James, the Lustre network protocol already contains the ability to pass the start/end offset as part of the OST_SYNC RPC. This is passed down to ofd_sync(), but is lost in the call to dt_object_sync(). I think it makes sense to change the OSD API to include the start and end parameters down the stack, and just drop them for the older kernels that only support sync on the whole inode. It seems the client-sync ll_fsync() code didn't wire this up correctly either, and are dropping the start and end parameters and passing 0 and OBD_OBJECT_EOF to cl_file_sync_range(). That should be fixed at the same time. It makes sense to create a separate sub-task for issues like this and the quota changes, so that they can get an appropriate audience (e.g. Niu and Johann for quota). James, seems that there may be a new patch for the quota renames as well: <a href="http://review.whamcloud.com/5119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5119</a> this is one more issue with fsync(). kernel commit 02c24a82187d5a628c68edfe71ae60dc135cd178 moved the i_mutex into the fsync() handlers. Ugh!!! It appears duplicate work was done in <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> I've uploaded my ldiskfs changes that bring it up to (at least 3.0) as <a href="http://review.whamcloud.com/5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5133</a> I see two competing sets of ldiskfs patches for sles11sp2. One from Jeff <a href="http://review.whamcloud.com/5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5133</a> and one from Chas <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> With a bit of hand waving and careful selection of what other patches I use, I can get either one to compile. Do we have a decision yet about which one is better? Seems to need an ext4/ldiskfs expert to decide. Jeff's set is a bit bigger if that means anything. To work on fc18 these changes need to have updates in at least autodetect_target in funcs.h check_options in build When I run on fc18 I get this <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">++ sed -ne '/^BUILDID =/s/.*= *//p' lustre/META + BUILDID=g932548b + load_target + EXTRA_VERSION_save= + for patchesdir in '"$EXTERNAL_PATCHES"' '"$TOPDIR/lustre/lustre/kernel_patches"' + TARGET_FILE=/targets/.target + '[' -r /targets/.target ']' + for patchesdir in '"$EXTERNAL_PATCHES"' '"$TOPDIR/lustre/lustre/kernel_patches"' + TARGET_FILE=/var/lib/jenkins/workspace/lustre-reviews-fc18/arch/x86_64/build_type/client/distro/fc18/ib_stack/inkernel/BUILD/lustre/lustre/kernel_patches/targets/.target + '[' -r /var/lib/jenkins/workspace/lustre-reviews-fc18/arch/x86_64/build_type/client/distro/fc18/ib_stack/inkernel/BUILD/lustre/lustre/kernel_patches/targets/.target ']' + '[' -r /var/lib/jenkins/workspace/lustre-reviews-fc18/arch/x86_64/build_type/client/distro/fc18/ib_stack/inkernel/BUILD/lustre/lustre/kernel_patches/targets/.target ']' + fatal 1 'Target was not found.' + cleanup + true + error 'Target was not found.' + local 'msg=Target was not found.' + '[' -n 'Target was not found.' ']' + echo -e '\nlbuild: Target was not found.' lbuild: Target was not found. + exit 1 + run_exit_traps + local i num_items=1 ++ seq 0 -1 0 + for i in '$(seq $((num_items-1)) -1 0)' + '[' -z 'kill -INT -22239 || true' ']' + eval kill -INT -22239 '||' true ++ kill -INT -22239 ../build/exit_traps.sh: line 59: kill: (-22239) - No such process ++ true + rc=1 + '[' 1 '!=' 0 ']' + echo 'Build failed' Build failed + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE </pre> </div></div> I will try and get a builder running so you can see the issues, this may or may be be easy because for logistical reasons of not stopping other builds. I also have ldiskfs patches that work up until the 3.7 kernel (openSUSE 12.3 – had some free time this weekend). At some point, dropping most of the inode_map_pages infrastructure in favor of the in-ext4 implementation should be discussed. The open coded stuff is falling behind (WRT bigalloc in particular, which isn't explicitly disallowed on ldiskfs) and the only reason not to use it is that the map of created blocks isn't passed back from the in-ext4 version. It turns out that nothing actually uses the map of created blocks or the optional_mutex argument to fsfilt_operations->fs_map_inode_pages anyway, so the in-ext4 version should be sufficient. Unless there's another user of that API out there that's out of the tree. 3.8 drops the kernel_thread export entirely. kthread_run replaces it. That API has been around forever, so it should be a fairly straightforward drop-in, with the exception that the naming of the thread in cfs_daemonize moves to cfs_create_thread. That's easy enough but took a little bit of wiggling to make it work on Darwin (untested of course, but looks sane). I'd like someone who knows the Lustre code ins-and-outs a bit more than I do to review the usage of cfs_daemonize_ctxt vs cfs_block_allsigs. In some cases the signal blocking happens only on linux and, in others, universally. The big difference between kernel_thread and kthread_run is that kthread_run already starts from a clean context, so daemonize isn't needed – unshare_fs_struct may be in some cases. Signals are ignored instead of blocked, so the signal handling logic needs to be looked at as well. Chris, can you pull in your jenkins changes into my ldiskfs work? I've only done local builds against SLE11 SP2 so I don't have any of that stuff set up. Bob I hope to have Jeff and Chris merge their work. It looks like Jeff has more fixes so it will be more favored <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> Great work Jeff on support up to 3.7. Once this is sorted away I was hoping to shrink the number of patches. If you look at ldiskfs support for SLES11 SP1 it reuses all the rhel6 patches plug one more patch specifically for SuSE. The reason for this approach was to take advantage of any fixes that go into ldiskfs for rhel6 with no extra work. I hope we can do the same for the new ldiskfs work. Also keep a eye on patch <a href="http://review.whamcloud.com/#change,5004" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5004</a> since it will impact this ldiskfs. One possibly significant difference I see between the two is Chas' set fixes autoconf to #define HAVE_LDISKFS_PDO but Jeffs' set doesn't (unless I missed something). Both needed some refresh on the pdirops patch before they would build right with current sp2 update kernel version 3.0.51-0.7.9 While working on the kernel patches for SLES11 SP2 I noticed two stray patches, bio_add_page.patch and quota-support-64-bit-quota-format.patch. They don't seem to be used by anyone. Any ideas on those? yes, just saw the pdir op problem today after updating my system. i already have a fix for that. about about ext4's fsync() now taking inode->i_mutex? what is the right behavior? not taking the i_mutex in the lustre code or reverting the i_mutex lock from fsync() when you build the ldiskfs module? Chas, one of Jeff's patches outside the ldiskfs set seems to deal with the i_mutex/fsync. see <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> Don't know if more changes are needed or not. Just FYI my recipe to get things building on sles11sp2 pulled in a lot of Jeff's other patches, including <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> The expectation is that fs implementation of ->fsync takes i_mutex. When things are nested like they are in Lustre, it's less clear, but the code shows that it's always just a passthrough to fsync anyway. In any case, it's dependent on HAVE_FILE_FSYNC_4ARGS. The commit that added the ranges to ->fsync also pushed the locking down into the file system implementation. There are only two explicit calls to ->fsync left: lustre_fsync, which is a pure passthrough and never had the lock anyway. osd_object_sync call only braces the call to ->fsync, so the proper locking is in ldiskfs already. So the fsync handling I have in my patches is correct. Bob, in addition to the ldiskfs patches, the following were required for SLES11SP2: <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> ldiskfs: ->dirty_inode takes two args <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> osd-ldiskfs: handle 4-arg fsync properly <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> ldiskfs: handle ext4_ext_cache without ec_type member <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> ldiskfs: ext4_free_blocks takes 6 arguments <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> osd-ldiskfs: use struct fs_disk_quota for quotas <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> llite: update quota APIs <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> osd-ldiskfs: set dentry->d_sb for xattr calls <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> osd-ldiskfs: Use __REQ_WRITE instead of BIO_RW. <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> mgs: fix cleanup of nonexistant MGSSELF_NAME proc file <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> mgs: fix missing error code return in mgs_fs_setup <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> osd-ldiskfs: Unlock new inodes before iput <blockquote>One possibly significant difference I see between the two is Chas' set fixes autoconf to #define HAVE_LDISKFS_PDO but Jeffs' set doesn't</blockquote> FYI, HAVE_LDISKFS_PDO will disappear when the landing backlog is flushed. <a href="http://review.whamcloud.com/5004" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5004</a> The recipe I used was: <a href="http://review.whamcloud.com/#change,4859" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4859</a> <a href="http://review.whamcloud.com/#change,4860" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4860</a> <a href="http://review.whamcloud.com/#change,4861" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4861</a> <a href="http://review.whamcloud.com/#change,4863" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4863</a> <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> <a href="http://review.whamcloud.com/#change,4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4970</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> <a href="http://review.whamcloud.com/#change,5118" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5118</a> <a href="http://review.whamcloud.com/#change,5119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5119</a> <a href="http://review.whamcloud.com/#change,5121" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5121</a> Some of those are Jeff's, some aren't. The following didn't seem to be absolutely necessary, but I put them in anyway: <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5122" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5122</a> <a href="http://review.whamcloud.com/#change,5124" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5124</a> On top of those I add the these for Jeff's ldiskfs: <a href="http://review.whamcloud.com/#change,5113" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5113</a> <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> or this one for Chas's ldiskfs: <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> Chris, thanks for the pointer to <a href="http://review.whamcloud.com/5004" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5004</a>. I missed that one. I probably would have grabbed it too. Doesn't play nicely with <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> in it's current form, but I could have worked around that. That is about the same combo of patches, we me currently testing with Chas ldiskfs patch. One other patch is missing is the lustre kernel patches in <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> I just updated it after I got my build system to properly patch a SuSE kernel and tested it out. Please give it a try as well. Thanks Thanks, James. I didn't forget about <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a>. I left it out of my recipe for two reasons: 1) It only involved the kernel patch series and didn't seem critical in getting the ldiskfs/lvfs portion of the world to build. 2) Until your refresh just now it didn't play well with the latest sp2 kernel. I had to massage nearly every patch a bit for context in the available commit to get it to work. I will retry with the latest. Having and using the kernel patch series and doing a kernel build is an important part of the building a sles11sp2 server process, I just didn't think that patch was quite ready yet. The refresh of <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> seems good for sles11sp2. Applied smoothly, kernel build in progress. I'm not in a position to try out the fc18 stuff right now. Kernel build finished up fine. My complete recipe for sles11sp2 now looks like: <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> <a href="http://review.whamcloud.com/#change,4859" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4859</a> <a href="http://review.whamcloud.com/#change,4860" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4860</a> <a href="http://review.whamcloud.com/#change,4861" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4861</a> <a href="http://review.whamcloud.com/#change,4863" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4863</a> <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> <a href="http://review.whamcloud.com/#change,4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4970</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> <a href="http://review.whamcloud.com/#change,5118" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5118</a> <a href="http://review.whamcloud.com/#change,5119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5119</a> <a href="http://review.whamcloud.com/#change,5121" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5121</a> <a href="http://review.whamcloud.com/#change,5004" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5004</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5122" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5122</a> <a href="http://review.whamcloud.com/#change,5124" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5124</a> <a href="http://review.whamcloud.com/#change,5113" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5113</a> <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> The ldiskfs patches in <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> still need a bit of hand waving. Probably due for a refresh. Seems to be something wrong with rpm building in sles11sp2. 'make' goes to completion and does build ldiskfs. 'make rpms' appears to finish, but no ldiskfs-* rpms get built. Don't see any ldiskfs modules in lustre-modules rpm. suspect this is due to includes being in include/generated. There is a patch that fixed this in lustre.spec.in, but nothing was ever done in ldiskfs/lustre-ldiskfs.spec.in My theory is looking good. If I take the '%{!?kversion: %global kversion...' line from lustre.spec.in and put it in ldiskfs/lustre-ldiskfs.spec.in, then I see lustre-ldiskfs-* rpms being built OK. Since fixes up are needed for ldiskfs rpms to build and the ldiskfs patches both need refreshing it would be a excellent time to merge all the work together. yes bob, that is what i did to get the rpm's to build for ldiskfs. this should be a separate patch/fix from ldiskfs support i would think. this .spec should have been fixed at the same time as the lustre.spec <a href="http://review.whamcloud.com/#change,5188" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5188</a> I see a recent change went into ldiskfs patches in master: <a href="http://review.whamcloud.com/#change,5179" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5179</a> Might a change like this also be needed in <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> or <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> ? it seems likely. i updated <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> patchset 9 broke some stuff that i copied into patchset 10. patchset11 of <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> does build now. Yes, I got caught trying to use patchset 10 for a few minutes. patchset 11 works much better. What code base are you successfully building 4972 against? I fetched patchset 11 and the ldiskfs patch set doesn't apply to my SP2 tree's copy of ext4. Based on some of the context for changes to fs/ext4/super.c, which still has the "Fourth Extended Filesystem with extents" module description that was changed to remove the "with extents" in 2.6.29, it looks like this is against SLE11 GA's 2.6.27 kernel. Am I missing something? I'm building OK against the 3.0.51 kernel I see in most recent update of sles11sp2. Not 2.6.<anything>. Ah, I know what it is. My quilt configuration explicitly disallows fuzz and this patch set has fuzz when applied to the SP2 tree. It should be refreshed. i think you need to allow fuzz. if another kernel comes out, there might be slight differences. i am not sure it makes sense to support older kernels than 'the latest release' from sle11sp2 (unless whamcloud is going to release a specific lustre kernel for suse). Unfortunately cray tends to use one kernel for the life of their products so we need to make sure the patches work over a range of kernels of version 3.0. Later kernels like 3.1 are a different story. that might be difficult since ext4_isize() in the p_dirop patch changes signature and in one set of 3.0 kernels you need to patch two instances and in another you only need to patch one. Which version did this occur at? Currently the kernel version in use is 3.0.34-0.7 I'm using a snapshot of our git repo from December, 3.0.51-15. The thing is the context hasn't changed for the bits I'm talking about – it changed in the 2.6.29 days. I get Chas's point about fuzz – it was just unexpected since my quilt setup is for our kernel repo where everything must apply with 0 fuzz. BTW, what's the standard procedure for updating other people's patch sets here? I noticed James updated a few of my patches responding to review, and I added some review notes to Chas's patchset 11 in 4972. Should I update it and submit? I ask because in the cases of the patches I suggested be switched over to using backports of the mainline commits, I already have them exported and backported. Okay so ext4_isize is fine at my version. Makes sense since I haven't seen any issues with building the current patch set. As for updating patches well that depends on the people you are working with. For me I have no problem with someone updating any of the patches I have worked on <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> Can't speak for Chas tho I did update his ldiskfs patch but only to rebase on top of <a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a>. The goal here is to merge all the ldiskfs work everyone has done. Chas should we label the patch with <a href="https://jira.whamcloud.com/browse/LU-2641" title="ldiskfs: Updated support for SLES 11 SP2 (v3.0 kernel)" class="issue-link" data-issue-key="LU-2641"><del>LU-2641</del></a> instead of <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> since it is ldiskfs specific for SLES11 SP2? I don't really care one way or the other about owner/authorship of these kinds of patches. There isn't any massive intellectual property associated with them. If you have updates to a patch, resubmit it. I will address Jeff's comments this weekend if he doesn't get to it before then. Somewhere between 3.0.34 and 3.0.51 ext4 got an extra ext4_isize(). My sources tell me there was an update or two between those. Never upgrading the kernel is a strange idea. What happens when there is an exploit? Our usual practice is to supported the latest released kernel update for the distros we support. Cray create their own releases so can make any adaptations that they need to make for older versions. 3.0.46 added the second ext4_isize as a backport of commit b71fc079 (ext4: fix fdatasync() for files with only i_size changes). I've updated the SP2 ldiskfs patchset, though it has a new dependency on not using LDISKFS_FS(sb)->s_qf_inums since that part of ext4-quota-first-class.patch wasn't incorporated into mainline. Since the fs fails the mount if the inodes aren't there, we can just grab them out of the superblock like ext4 does itself. After I get some testing in, I'll push it to gerrit. I've updated and pushed the patchset. My current recipe looks like: <a href="http://review.whamcloud.com/5264" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5264</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> ldiskfs: don't use sb->s_qf_inums <a href="http://review.whamcloud.com/5121" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5121</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> osd-ldiskfs: Use __REQ_WRITE instead of BIO_RW. <a href="http://review.whamcloud.com/5122" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5122</a> – <a href="https://jira.whamcloud.com/browse/LU-2729" title="MGS issues warning when trying to remove nonexistent proc file" class="issue-link" data-issue-key="LU-2729"><del>LU-2729</del></a> mgs: fix cleanup of nonexistant MGSSELF_NAME proc file <a href="http://review.whamcloud.com/5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5115</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> osd-ldiskfs: handle 4-arg fsync properly <a href="http://review.whamcloud.com/5188" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5188</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> ldiskfs: detect generated utsrelease.h <a href="http://review.whamcloud.com/4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4966</a> – <a href="https://jira.whamcloud.com/browse/LU-812" title="Support for Linux 3.0 kernels" class="issue-link" data-issue-key="LU-812"><del>LU-812</del></a> ldiskfs: super_operations->dirty_inode now takes a flag <a href="http://review.whamcloud.com/4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4970</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> ext4: ec_type has been dropped from ext4_ext_cache <a href="http://review.whamcloud.com/5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5001</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> fsfilt: ext_pblock renamed to ext4_ext_pblock <a href="http://review.whamcloud.com/5118" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5118</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> osd-ldiskfs: use struct fs_disk_quota for quotas <a href="http://review.whamcloud.com/5119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5119</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> llite: update quota APIs <a href="http://review.whamcloud.com/5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5120</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> osd-ldiskfs: oti_obj_dentry needs d_sb set <a href="http://review.whamcloud.com/4803" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4803</a> – <a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a> ldiskfs: Reorganize ldiskfs kernel patches <a href="http://review.whamcloud.com/4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4972</a> – <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> ldiskfs: Add kernel patch series for SLES11SP2 Jeff, Thanks for the latest. Trying out the new recipe now. Just to help keep things sane could you please mark <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> Abandoned since all the new work is now happening in <a href="http://review.whamcloud.com/4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4972</a> ? Would hate for anybody to waste time looking at or picking up obsolete stuff. Bob, since our patch series differ, I took a minute to reconcile the differences between them. You can drop <a href="http://review.whamcloud.com/5124" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5124</a>. That was a result of a bug introduced in my ldiskfs patches. It resulted in a working SLES but applying that patch on a RHEL system would make it unhappy. <a href="http://review.whamcloud.com/5113" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5113</a> and <a href="http://review.whamcloud.com/#change,5133" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5133</a> can be dropped as well. I've compared and integrated the differences between those and 4972 and pushed the result to 4972. These have been pulled into master in the past few weeks: <a href="http://review.whamcloud.com/5004" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5004</a> <a href="http://review.whamcloud.com/4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4649</a> <a href="http://review.whamcloud.com/4859" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4859</a> <a href="http://review.whamcloud.com/4860" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4860</a> <a href="http://review.whamcloud.com/4861" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4861</a> <a href="http://review.whamcloud.com/4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4991</a> looks like <a href="http://review.whamcloud.com/5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5001</a> is superseding it, which is why I took it in my series. So that just leaves <a href="http://review.whamcloud.com/4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4649</a>, which didn't conflict with any of my patches at all. Ok, all of the patches I submitted initially that aren't in that list have been marked as abandoned now. The new recipe isn't working for me. Builds go to completion and look fine, but mounts of ldiskfs aren't working. In particular the mount done in mkfs.lustre to write out config inodes fails. client cmd: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># mkfs.lustre --mgs --mdt --index=0 --fsname=lustre /dev/sdb Permanent disk data: Target: lustre:MDT0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x65 (MDT MGS first_time update ) Persistent mount opts: user_xattr,errors=remount-ro Parameters: checking for existing Lustre data: not found device size = 1024MB formatting backing filesystem ldiskfs on /dev/sdb target name lustre:MDT0000 4k blocks 262144 options -I 512 -i 2048 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000 -I 512 -i 2048 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F /dev/sdb 262144 mkfs.lustre: Unable to mount /dev/sdb: No such process mkfs.lustre FATAL: failed to write local files mkfs.lustre: exiting with 3 (No such process) </pre> </div></div> strace of the cmd shows that it's the mount() syscall of ldiskfs that fails: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> . . . mkdir("/tmp/mntMBwvlZ", 0700) = 0 mount("/dev/sdb", "/tmp/mntMBwvlZ", "ldiskfs", 0, "user_xattr,errors=remount-ro") = -1 ESRCH (No such process) write(2, "mkfs.lustre: Unable to mount /de"..., 55mkfs.lustre: Unable to mount /dev/sdb: No such process ) = 55 . . . </pre> </div></div> /var/log/messages says: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Feb 4 15:29:26 suse2 kernel: [19073.692340] LDISKFS-fs warning (device sdb): ldiskfs_enable_quotas:4939: Failed to enable quota (type=0) tracking. Please run e2fsck to fix. Feb 4 15:29:26 suse2 kernel: [19073.692350] LDISKFS-fs (sdb): mount failed Feb 4 15:34:44 suse2 kernel: [19390.782444] LDISKFS-fs warning (device sdb): ldiskfs_enable_quotas:4939: Failed to enable quota (type=0) tracking. Please run e2fsck to fix. Feb 4 15:34:44 suse2 kernel: [19390.782456] LDISKFS-fs (sdb): mount failed </pre> </div></div> I ran into that too but I hadn't used a working 2.4 system yet. modprobe quota_v2 fixes it. Since I now know it's unexpected, I'll look into fixing it. modprobe quota_v2 works for me, too. Seems like there's some kind of dependency failure there somewhere. Thanks, Jeff. It looks like it's because QFMT_VFS_V1 doesn't have an entry in the module list that the dquot uses to request a module be loaded. The SLES kernel has CONFIG_QFMT_V1=m. The RHEL kernel has CONFIG_QFMT_V1=y, so it's not hitting the problem. I've written up a patch and sent it to linux-fsdevel. It'll land in a SLES kernel update as soon as it's accepted there. In the interim, we'll need to manually load quota_v2.ko for testing. yeah, i dropped the ball on that. i mentioned the problem on <a href="http://jira.whamcloud.com/browse/LU-1842" class="external-link" rel="nofollow">http://jira.whamcloud.com/browse/LU-1842</a> it would be nice if this got fixed. Jeff can you point me to the fix for the quota. I can include that in the 4649 patch. This is the fix: <a href="http://www.spinics.net/lists/linux-fsdevel/msg62124.html" class="external-link" target="_blank" rel="nofollow noopener">http://www.spinics.net/lists/linux-fsdevel/msg62124.html</a> but it's not something that can be included except in lustre/kernel_patches. I expect the patch itself to land in -stable updates soon and distro release kernels from other vendors soon thereafter. Of course, it could be worked around by doing an explicit request_module("quota_v2") but that's just ugly. I note that many of the recommended configs in lustre/kernel_patches/kernel_configs have CONFIG_QFMT_V2=y. If in the future we have published configs for sles11 and pre-built kernel rpms with this setting the issue may become less important. It looks like Ted beat me to the punch by a few days. <a href="http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=commitdiff;h=c3ad83d9efdfe6a86efd44945a781f00c879b7b4" class="external-link" target="_blank" rel="nofollow noopener">http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=commitdiff;h=c3ad83d9efdfe6a86efd44945a781f00c879b7b4</a> Funny how a bug that's been there since 2.6.33 drew attention from both of us within days of each other. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> I've checked it into the SLE11 SP2 tree, the openSUSE 12.3 tree, and our master branch so it should roll out in the next SLE11 SP2 update. BTW, we publish our kernel git repos publicly, so you can get it more quickly here: <a href="http://kernel.suse.com/" class="external-link" target="_blank" rel="nofollow noopener">http://kernel.suse.com/</a> Since you are pushing patches to your distro once I'm done testing your latest rounds of patches I will update my kernel side patch and then we could the patch upstream in time for the next release of SP2. This is in reference to change 4649. Neil Brown, the MD RAID maintainer, is in my team at SUSE. I've passed on a copy of that patch to have him take a look at the raid5 fix for mop. A version of my dev_set_rdonly patch can be found here: <a href="http://kernel.suse.com/cgit/kernel-source/tree/patches.suse/block-add-dev_check_rdonly-and-friends-for-lustre-testing?h=SLE11-SP3" class="external-link" target="_blank" rel="nofollow noopener">http://kernel.suse.com/cgit/kernel-source/tree/patches.suse/block-add-dev_check_rdonly-and-friends-for-lustre-testing?h=SLE11-SP3</a> There are some small cosmetic differences between that one and the one I'm planning to use. I'm going to pr_warn instead of printk, add a message about clearing the read-only state, and remove an extra prototype in there that we don't need. By the time you read this, that might actually be fixed. It is already in my repository but our internal git server (which mirrors out to that one) isn't being cooperative right now. MUch nicer look Jeff. I will integrate that into my next patch set. Will this be integrated into SP2? Okay I had to do some rebasing with the latest code so I pushed the needed changes. Patch 5001 is needed before the patch for <a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a> since it alters one of the core rhel5 ldiskfs patches. Give it a try It looks like the major shack up of the fsfilt layer is mostly done. The working combo; must apply in order; I'm testing is <a href="https://jira.whamcloud.com/browse/LU-2158" title="remove lvfs and fsfilt code" class="issue-link" data-issue-key="LU-2158"><del>LU-2158</del></a> <a href="http://review.whamcloud.com/#change,5246" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5246</a> <a href="http://review.whamcloud.com/#change,5250" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5250</a> <a href="http://review.whamcloud.com/#change,5331" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5331</a> <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> <a href="http://review.whamcloud.com/#change,4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4970</a> <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5188" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5188</a> <a href="https://jira.whamcloud.com/browse/LU-2059" title="mgc to backup configuration on osd-based llogs" class="issue-link" data-issue-key="LU-2059"><del>LU-2059</del></a> <a href="http://review.whamcloud.com/#change,5049" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5049</a> <a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a> <a href="http://review.whamcloud.com/#change,4803" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4803</a> <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> <a href="http://review.whamcloud.com/#change,5121" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5121</a> <a href="https://jira.whamcloud.com/browse/LU-2729" title="MGS issues warning when trying to remove nonexistent proc file" class="issue-link" data-issue-key="LU-2729"><del>LU-2729</del></a> <a href="http://review.whamcloud.com/#change,5122" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5122</a> Also we have patches from <a href="https://jira.whamcloud.com/browse/LU-2760" title="Add error handling to ext4_mkdir on RHEL6" class="issue-link" data-issue-key="LU-2760"><del>LU-2760</del></a>, <a href="https://jira.whamcloud.com/browse/LU-2761" title="rhel: use the upstream version of the journal callback patch" class="issue-link" data-issue-key="LU-2761"><del>LU-2761</del></a>, <a href="https://jira.whamcloud.com/browse/LU-2762" title="rhel: use upstream ext4_kvzalloc" class="issue-link" data-issue-key="LU-2762"><del>LU-2762</del></a> but they conflict with my SLES11 SP1 platform. Will need to work on that later. James, Think you are missing this one. I use it in my recipe: <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> Oops. I'm using that as well. With the fsfilt cleanup hopefully this will not be changing to much going forward. Bob have you been getting any results yet? Yes, I have been getting good results with my current recipe on sles11sp2. I haven't started using changes 5246, 5250, 5336 yet. I will try adding them & see what happens. Have been trying to work with 5279,5280,5281,5282,5283. Haven't included them for real in my current recipe yet because 5282, the one relocating dynlocks, breaks things in its current form. Have been waiting for that to refresh and settle out. Didn't realize some of them had problems in sles11sp1. I have an updated version of the dynlocks patch I'll upload in a minute. I see the refresh of the dynlocks patch. Seems smoother than before. However it does generate an alarming looking build warning: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>WARNING: /home/bogl/lustre-release/lustre/obdclass/obdclass.o(.text+0x30d7): Section mismatch in reference from the function class_dynlock_init() to the function .init.text:dynlock_cache_init() The function class_dynlock_init() references the function __init dynlock_cache_init(). This is often because class_dynlock_init lacks a __init annotation or the annotation of dynlock_cache_init is wrong. </pre> </div></div> Probably should have put that last comment on <a href="https://jira.whamcloud.com/browse/LU-2757" title="Move dynlocks out of ldiskfs" class="issue-link" data-issue-key="LU-2757"><del>LU-2757</del></a> Ah, it's because class_dynlock_init wasn't marked __init and dynlock_cache_init was. Non-init functions aren't allowed to call __init functions. In this case it wouldn't have had any runtime impact other than a bit of memory lost since class_dynlock_init is also only called from an __init function. I've fixed it so class_dynlock_init is also __init in Patchset 5. Just a comment here - I've been completely ignoring all of these patches until there is some indication that they are not being refreshed continuously, and are in a shape that they are ready for landing. In particular that the patches are working on both SLES11 SP2 and RHEL6. If I'm incorrect in waiting to start my inspections, or if there are patches early in the series that are stable and can be landed while later patches are being refreshed frequently, please let me know. I would say the patches ready for inspection and landing are: <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> - you asked about passing in the flag so if it needs to be refresh that is okay. <a href="http://review.whamcloud.com/#change,4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4970</a> <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5121" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5121</a> <a href="http://review.whamcloud.com/#change,5188" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5188</a> The reset depend on other patches landing (<a href="https://jira.whamcloud.com/browse/LU-2473" title="ldiskfs RHEL6.4 support" class="issue-link" data-issue-key="LU-2473"><del>LU-2473</del></a>, <a href="https://jira.whamcloud.com/browse/LU-2059" title="mgc to backup configuration on osd-based llogs" class="issue-link" data-issue-key="LU-2059"><del>LU-2059</del></a>, and <a href="https://jira.whamcloud.com/browse/LU-2158" title="remove lvfs and fsfilt code" class="issue-link" data-issue-key="LU-2158"><del>LU-2158</del></a>) or need more work. <a href="http://review.whamcloud.com/#change,5122" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5122</a> should also be mentioned. Looks like it's already inspected and just needs landing. Bob, except that patch is at the end of a series of patches, and cannot typically be landed until all of the previous ones are. If patches are independent of each other, they should be submitted from separate branches, otherwise they are subject to much longer delays for landing. Patch 5122 is only dependent on one patch (<a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a>) which is unrelated. The question is which is faster, refreshing the patch with no dependency and going through new review or waiting until the previous patch is merged? Patchset 5 fixes the build warning for me. Have added the following changes into my recipe with good success on sles11sp2: 5330, 5279, 5280, 5281, 5282. Haven't been able to make <a href="http://review.whamcloud.com/#change,5283" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5283</a> apply nicely lately. Think it may need refresh due to overlapping changes in one of the rhel6 ldiskfs patches from recent <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a>. That last report may have been a tad optimistic. Builds complete OK, but functionality is not so good. Seeing failed mounts and crashes. May have more details later. Bob without patches 5330, 5279, 5280, 5281, 5282 it works fine correct? yes, worked fine without. Now after a few reboots it's working better with those added patches too. I think I may have had a mix of old and new bits impacting me before. Will try to be more careful with updates and installs. Andreas can you review <a href="http://review.whamcloud.com/#change,4970" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4970</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> Both are ready for merger. James, Jeff, Is it time to turn on Andreas for review of the sles11sp2 ldiskfs patches? The only official dependency left is <a href="http://review.whamcloud.com/#change,4803" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4803</a>, which has been reviewed & tested and looks ready to land. <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> has been stable for a week now. I've had good results using it on sles11sp2 so far. Is much more change to come expected? I would say so, also a few other patches are ready as well. Andreas can you review. <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> <a href="http://review.whamcloud.com/#change,4972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4972</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> Sorry had to finish untangling patch <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> from all the dependencies it had. Now it can be inspected and merged on its own. The patches still need to get SLES11 SP2 working are: <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> # 2 arg ->dirty_inode; fixes build failure <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> # 6 arg ext4_free_blocks; fixes build failure <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> # ext4_ext_pblock rename; fixes build failure <a href="http://review.whamcloud.com/#change,5115" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5115</a> # locking around ->fsync; fixes deadlock <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> # set ->d_sb; fixes crash in xattr operations <a href="http://review.whamcloud.com/#change,5503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5503</a> # fix typo in REQ_WRITE check Of those 4966 needs more work. Patches 5115 and 5120 are in the queue to be merged. Patches 4991,5001,5503 need to be inspected. Andreas can you take a look at 4991, 5001, and 5503. Note I slimmed down patch 5001 since fsfilt will be cleaned up in patches for <a href="https://jira.whamcloud.com/browse/LU-2158" title="remove lvfs and fsfilt code" class="issue-link" data-issue-key="LU-2158"><del>LU-2158</del></a> from me. Thanks. <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> still needs to land too. Has yet to pass tests. Can almost build. It fails to build because of mdt_fid2path in mdt_handler.c is not setting rc to some default value. Other than that it builds. Will need to create a patch for that. Is rc = o the right choice or is the bug larger than that? As for 4649 yes it also needs to be updated and worked on. I was thinking about Jeff's patch for SP3 and dev_readonly. Instead of dev_readonly why not use the FAIL_MAKE_REQUEST functionality. Jeff also are you fine with my approach for tunable handling? Can we push that upstream for SP2? For a update where we are at. The following set of patches is needed to build and run lustre on SLES11 SP2 <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> - <a href="https://jira.whamcloud.com/browse/LU-812" title="Support for Linux 3.0 kernels" class="issue-link" data-issue-key="LU-812"><del>LU-812</del></a> <a href="http://review.whamcloud.com/#change,5675" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5675</a> - <a href="https://jira.whamcloud.com/browse/LU-1199" title="lustre build system overhaul" class="issue-link" data-issue-key="LU-1199"><del>LU-1199</del></a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> - <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5656" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5656</a> - <a href="https://jira.whamcloud.com/browse/LU-2929" title="build error with gcc 4.6.3" class="issue-link" data-issue-key="LU-2929"><del>LU-2929</del></a> <a href="http://review.whamcloud.com/#change,5503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5503</a> - <a href="https://jira.whamcloud.com/browse/LU-2642" title="lustre: Updated support for SLES11 SP2" class="issue-link" data-issue-key="LU-2642"><del>LU-2642</del></a> <a href="http://review.whamcloud.com/#change,5708" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5708</a> - <a href="https://jira.whamcloud.com/browse/LU-2748" title="OSD uses kmalloc with high order to allocate a keys" class="issue-link" data-issue-key="LU-2748"><del>LU-2748</del></a> <a href="http://review.whamcloud.com/#change,5709" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5709</a> - is a bug fix for ldiskfs on SP2 <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> - optional and needs to be updated. James, Thanks for the update on the recipe. This more or less matches what I've been using the last few days. <a href="http://review.whamcloud.com/#change,5708" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5708</a> was new to me. <a href="http://review.whamcloud.com/#change,5503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5503</a> is ready to land now, I think. The patches currently in gatekeeper waiting to land are <a href="http://review.whamcloud.com/#change,4966" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4966</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,5656" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5656</a> <a href="http://review.whamcloud.com/#change,5503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5503</a> While they aren't absolutely required for sles11sp2 to work, I added the following to James' latest recipe without any problems: <a href="http://review.whamcloud.com/#change,5279" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5279</a> <a href="http://review.whamcloud.com/#change,5280" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5280</a> <a href="http://review.whamcloud.com/#change,5281" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5281</a> <a href="http://review.whamcloud.com/#change,5282" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5282</a> <a href="http://review.whamcloud.com/5786" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5786</a> will be needed after the next SP2 update since some ext4 bug fixes were pulled in via my submission to -stable. Jeff, Those changes in #5786 look fine to me but we will have to be careful not to land them until the next SP2 update is really available and shows up when anybody does 'zipper up'. Can you please let us know when that happens? Jeff have you looked at <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> for how I handle the optimizations as module parameters as you suggested. It would be nice to have those upstream for SLES11. Time for a update for where we are at. Patches needed now are: <a href="http://review.whamcloud.com/#change,5675" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5675</a> <a href="https://jira.whamcloud.com/browse/LU-1199" title="lustre build system overhaul" class="issue-link" data-issue-key="LU-1199"><del>LU-1199</del></a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="https://jira.whamcloud.com/browse/LU-1812" title="3.6/FC18 Server Patches" class="issue-link" data-issue-key="LU-1812"><del>LU-1812</del></a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> - finally passed <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> <a href="http://review.whamcloud.com/#change,5709" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5709</a> - <a href="https://jira.whamcloud.com/browse/LU-2926" title="crash in is_bad_inode()" class="issue-link" data-issue-key="LU-2926"><del>LU-2926</del></a> bug fix for ldiskfs Using James' most recent recipe, I'm seeing a build failure in mdt_handler.c. I can fix it on the fly with the following patch: <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>diff --git a/lustre/mdt/mdt_handler.c b/lustre/mdt/mdt_handler.c index ba3f24d..97c56a5 100644 --- a/lustre/mdt/mdt_handler.c +++ b/lustre/mdt/mdt_handler.c @@ -5547,7 +5547,7 @@ static int mdt_path_current(struct mdt_thread_info *info, char *ptr; int reclen; struct linkea_data ldata = { 0 }; - int rc; + int rc = 0; ENTRY; /* temp buffer for path element, the buffer will be finally freed </pre> </div></div> Think this was introduced by a recent commit that builds fine in Centos 6.x Haven't yet filed a bug on it. I should mention that I add <a href="http://review.whamcloud.com/#change,4804" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4804</a> to the recipe. While it's optional in sles11sp2 it includes changes to the ldiskfs patch series selection rules for all distros. Dosen't hurt to make sure those changes work right in sles11sp2 too. Ug the rc not set issue again. We really need to set the compiler flags on rhel so these issues can be avoid. Please post the JIRA ticket once you open it. New bug is <a href="https://jira.whamcloud.com/browse/LU-3043" title="server build for sles11sp2 is broken" class="issue-link" data-issue-key="LU-3043"><del>LU-3043</del></a>. Have already linked it and pushed a 1-line fix. Looks like all the patches are good to go now. Andreas can you please inspect patches: <a href="http://review.whamcloud.com/#change,5675" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5675</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> <a href="http://review.whamcloud.com/#change,5709" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5709</a> Thanks. Also <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> has to reviewed again since it had to rebased to master due to changes in the code from other commits. As Andreas is on vacation we may want to round up somebody else to do these reviews. <a href="http://review.whamcloud.com/#change,5854" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5854</a> is also good to go & just needs another +review. Don't worry - I will be assigning alternate reviewers to cover for Andreas's vacation <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> Time to do some reviews of the patches. Due to false negatives with Maloo and patch conflicts a few of the patches need to be updated and are ready for reviewed again. <a href="http://review.whamcloud.com/#change,5675" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5675</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> The following are now ready for landing: <a href="http://review.whamcloud.com/#change,5675" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5675</a> <a href="http://review.whamcloud.com/#change,4991" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4991</a> <a href="http://review.whamcloud.com/#change,5001" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5001</a> <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> is still awaiting some review. With any luck <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a> should be ready soon too if maloo stays healthy. <a href="http://review.whamcloud.com/#change,4804" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4804</a> also looks good to go now. Down to one major patch - <a href="http://review.whamcloud.com/#change,5120" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,5120</a>. Looking at the Maloo results I see it failed due to <a href="https://jira.whamcloud.com/browse/LU-3057" title="Hang at sanity test_56x and test_56w" class="issue-link" data-issue-key="LU-3057"><del>LU-3057</del></a> and memory exhaustion for sanity65ic. Can some one with the power to restart the test please due for this patch. We also have <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> for server kernel optimization and support to simulate disk failure. The progress is certainly encouraging! I have retriggered the testing for the 5120 change set. Yahoo!!! Last patch needed for SLES11 SP2 to work server side landed. Only patch <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a> is left for this ticket which is kernel tuning and the dev_read_only patch to simulate a failed disk. Only one patch left, <a href="http://review.whamcloud.com/#change,4649" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4649</a>, which is for FC18 only. Peter can you link this ticket to <a href="https://jira.whamcloud.com/browse/LU-2686" title="Kernel update for 3.7.2-201.fc18" class="issue-link" data-issue-key="LU-2686"><del>LU-2686</del></a> and label this as a Lustre 2.5 item. ok This patch has landed to master. The work for kernel side support is now done so we can close this ticket. ok. Thanks James Duplicate LU-2641 Related LU-2642 LU-2686 LU-3043 LU-3337 LUDOC-82 Development Rank 1|hzv4gn: Rank (Obsolete) 4256