[#LUDOC-394] Use proper names for sections in the manual

[LUDOC-394] Use proper names for sections in the manual Created: 14/Jan/18 Updated: 21/May/21 Resolved: 21/May/21
Status:	Resolved
Project:	Lustre Documentation
Component/s:	None
Affects Version/s:	None
Fix Version/s:	Lustre 2.14.0 Manual

Type:

Improvement

Priority:

Minor

Reporter:

Andreas Dilger

Assignee:

Lustre Manual Triage

Resolution:

Fixed

Votes:

Labels:

easy

Attachments:

image-2020-04-30-14-51-54-002.png

Issue Links:

Related
is related to	LUDOC-456	Update the xml ids throughout the manual	Open

Rank (Obsolete):

9223372036854775807

Description

Currently, a large number of sections are named with auto-generated section references like dbdoclet.50438199_14978 that have no meaning. Unfortunately, these section references appear in the HTML manual as the relative URL, like http://doc.lustre.org/lustre_manual.xhtml#dbdoclet.50438199_14978

It would be better to give these section references a proper name, like dbdoclet.deactivating_mdt_ost (as is done in patch https://review.whamcloud.com/30864 "~~LUDOC-305~~ maintenance: handling full/failed OSTs" in this specific case), but across the whole manual.

Changing these references is relatively straight forward:

search for dbdoclet.* (or in a few cases section_*)
for any reference that does not have a descriptive name
- make up a new name
- search the rest of the files for the old name
- replace the old name with the new name

This could be done in a number of smaller patches, so that the whole thing doesn't break if any other patch lands to the manual. While the manual isn't a hotbed of activity, it will probably take too long to do this manually than can be accomplished in a single sitting, and it cannot be done automatically, so better to break it up into smaller chunks.

Comments

Comment by Joey White-Swift [ 30/Apr/20 ]

All,

I have written a Shell Script to automate the correction by creating a sed file that looks something like this:

With the intent of running all of those sed replaces against each xml file of the docs.

Unfortunately, some of the docs have the </title> tag (which is what I referenced in the script) in weird places so that is why on the ZFS they have all _____. I would guess that about 75% of the files were able to be processed properly by the script. If if all of the XML files adhere to this standard (with the </title> adjacent to the correct title and the xml:id coming first in the document):
......
<section xml:id="dbdoclet.50438191_20721">
<title><indexterm>
<primary>LNet</primary>
<secondary>supported networks</secondary>
</indexterm>Supported Network Types*</title>*
<para>The LNet code module includes LNDs to support many network types including:</para>
<itemizedlist>
.....
then the script would be able to have a standard format for each of the internal links, every time, and could be run at compile time for new additions to keep consistency. This format currently for the example above would be to have Supported_Network_Types as the xml:id but the formatting could be changed easily enough.

If this sort of "revamp" is not something that you are interested in doing then I will sanitize the sed commands and offer a large number of corrections.

~Joey

Comment by Andreas Dilger [ 03/May/20 ]

Joey, I think it is perfectly reasonable to submit a large patch that changes the "easy-to-fix" problems in the manual, then reviewing repetitive changes is not so much effort.

Even if this doesn't solve all problems, it still gets us further ahead.

Comment by Andreas Dilger [ 02/Feb/21 ]

Joey, would you be able to submit a patch with the updates from your script? I've been updating some of the section references manually, but would welcome some assistance in this area.

Comment by Joey White-Swift [ 02/Feb/21 ]

Andreas,

Covid sort of scrambled everything and I totally forgot about this. I can go back to see if I still have the script. One though I did have in going through this (and I think there might me another ticket as well) is about using a "meta-document" that could be compiled into different versions of docs such as HTML, PDF and even man pages.

I am old school so my preference would be something like (gt)roff (I know, I know) but I have used sphinx which would support static html pages, pdf and man pages comfortably with internal linking and could be added to the build script to get built with versions of LUSTRE. (The only advantage for some form of roff is that is will be on the system anyways if internal links aren't paramount). If you appreciate an approach of moving to a meta-document, then maybe spending time to switch to that instead of fixing the existing code base could prove more "profitable". I might also be able to script up something to aid in the bulk of the transition.

I will look for the script in the mean time, but please let me know. I would rather do a bit more work up front to convert and make things easy for those that come later.

~Joey

Comment by Andreas Dilger [ 03/Feb/21 ]

One though I did have in going through this (and I think there might me another ticket as well) is about using a "meta-document" that could be compiled into different versions of docs such as HTML, PDF and even man pages.

I'm not sure what you mean? The current Docbook XML format is already being built in HTML, PDF, and ePUB formats:
https://build.whamcloud.com/job/lustre-manual/lastSuccessfulBuild/artifact/

I'm not sure whether nroff would be the right format for the Operations Manual. Definitely we do use nroff for the actual man pages in lustre/doc, and we can always use more contributors there if you have experience in that area (e.g. LU-4315)

Comment by Joey White-Swift [ 03/Feb/21 ]

Andreas,

I have found the script and made a couple of changes to it to account for the lack of unique title links. I have created a branch and hope to get all of the seds run across the files today. Once I have done that, how would you like me to proceed to get the changes back?

I will look into the roff issue in the future, once this is squared.

~Joey

Comment by Andreas Dilger [ 04/Feb/21 ]

Joey, getting a series of relatively smaller patches that update the references to/from one file at a time would be easiest for review, and shouldn't cause (m)any conflicts between files.

Comment by Joey White-Swift [ 10/Feb/21 ]

Andreas,

I have a set of patches, per file, that are ready for you. I also have a list of all of the substitutions that were conducted on each file for your reference. How would you like me to provide them?

~Joey

Comment by Andreas Dilger [ 11/Feb/21 ]

Hi Joey, please push patches to Gerrit (see https://wiki.lustre.org/Using_Gerrit fir details). All of the patches can use this LUDOC ticket. Ideally, if the patches are non-conflicting they would each be pushed from a separate branch, so that if there is a reason one of them can't land it doesn't block the others. However, given the relatively simple manner of the patches I don't expect a huge problem.

I'd recommend to start with 1-2 patches first so we can address any issues with commit messages, etc. first, and then submit them in a few batches to avoid pounding the builders all at once.

Comment by Gerrit Updater [ 18/Mar/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/42070
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under ConfiguringLustre.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 9635dda3b53004260f8a5cedffb74f0cbcb68e19

Comment by Arshad Hussain [ 18/Mar/21 ]

Hi Joey, I saw the yours/Andreas comments in this ticket after I pushed the patch. Sorry about this...Your patchs are ready please feel free to push it. I would not be disturbing this ticket since you are already working on this.

Comment by Andreas Dilger [ 18/Mar/21 ]

As yet, Joey doesn't have an account in Gerrit, so there is no way to add them as a reviewer on Arshad's patch.

Comment by Gerrit Updater [ 18/Mar/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/42070/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under ConfiguringLustre.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 4354e542d77e8ee5e08fc43106f0bd09eb7f4464

Comment by Joey White-Swift [ 18/Mar/21 ]

I do not have a provider of OpenID, so I can put a zip of the patches as attachment so you can move forward.

~Joey

Comment by Arshad Hussain [ 19/Mar/21 ]

>I do not have a provider of OpenID

No problem. Create an account in UbuntuOne/launchpad. Then log into gerrit with your Launchpad ID.

Comment by Joey White-Swift [ 19/Mar/21 ]

I have logged into Gerrit, added an ssh key, and sent an email to info@whamcloud.com.

With any luck they will be up soon.

~Joey

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43353
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under UnderstandingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: a3f9cf3351e03b30b25ca757cce0e6f0983c02fa

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43354
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under ManagingFailover.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 16c30375fcb2b15851e9f083bd36943cf0f11b25

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43355
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under TroubleShootingRecovery.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: b59de4056351b4a5c864c7ff7f0ab43a7d5711d8

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43356
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreTuning.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: c2a0253d70c9c0e93e45e2a47255eae35d709c6c

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43357
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under InstallingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 8763734be730f70e1b43ba397ffe7c4abd0e22a6

Comment by Gerrit Updater [ 17/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43358
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under UpgradingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: c48a5f8ac4514e6b6b457d0fb8a835a25978b10d

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43358/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under UpgradingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 921417b64f0c185323c17390edfc128eba980259

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43357/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under InstallingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 4779f3173afc069f7a1ecd4406910db513bb8247

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43356/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreTuning.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: e418d09f026c64de4132edb613890f8067c71497

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43355/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under TroubleShootingRecovery.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: d9e6c337eeca2e88d1db07edb101cd72442b098c

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43354/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under ManagingFailover.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: a21827d3c777f2cfa640ee6d67dc87ab5a9df39d

Comment by Gerrit Updater [ 17/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43353/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under UnderstandingLustre.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: e096f06b81ed50da7c6f21867162f272299465ef

Comment by Gerrit Updater [ 18/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43359
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreOperations.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 1b785092e59af24c00bf46cad920a215dd567ffd

Comment by Gerrit Updater [ 18/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43360
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreDebugging.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 1f9bca1c797422a812a1be9635f18b8e8137b2b3

Comment by Gerrit Updater [ 18/Apr/21 ]

Arshad Hussain (arshad.hussain@aeoncomputing.com) uploaded a new patch: https://review.whamcloud.com/43361
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under SystemConfigurationUtilities.xml
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 053a4e9ff91215c1a0a0ae1ac128e0f97fc54aea

Comment by Gerrit Updater [ 21/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43360/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreDebugging.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: f7edc1052d1752e09e6a1e39843cecb23ac42880

Comment by Gerrit Updater [ 21/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43361/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under SystemConfigurationUtilities.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: ede18e4d53e2ab87c1ea01e6ced25534a60d9f99

Comment by Gerrit Updater [ 21/Apr/21 ]

Andreas Dilger (adilger@whamcloud.com) merged in patch https://review.whamcloud.com/43359/
Subject: ~~LUDOC-394~~ manual: Add meaningful ref names under LustreOperations.xml
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 871cc5fda762843433db14d0ecf6e7086e395752