[LUDOC-23] Update the "Determining MDS/MDT Space Requirements" to reflect the recent changes in 1.8.7 and 2.1 release

• Status: Closed
• Project: Lustre Documentation
• Type: Technical task
• Priority: Minor
• Reporter: Zhiqi Tao (Inactive)
• Assignee: Zhiqi Tao (Inactive)
• Resolution: Fixed

Description

The default inode size in 1.8.7 and 2.1 has been changed to 2KB.

The section should be updated:

"
When calculating the MDT size, the important factor to consider is the number of files to be stored in the file system. This determines the number of inodes needed, which drives the MDT sizing. To be on the safe side, plan for 4 KB per inode on the MDT, which is the default value. Attached storage required for Lustre metadata is typically 1-2 percent of the file system capacity depending upon file size.

For example, if the average file size is 5 MB and you have 100 TB of usable OST space, then you can calculate the minimum number of inodes as follows:

(100 TB * 1024 GB/TB * 1024 MB/GB) / 5 MB/inode = 20 million inodes

We recommend that you use at least twice the minimum number of inodes to allow for future expansion and allow for an average file size smaller than expected. Thus, the required space is:

4 KB/inode * 40 million inodes = 160 GB

If the average file size is small, 4 KB for example, Lustre is not very efficient as the MDT uses as much space as the OSTs. However, this is not a common configuration for Lustre.
"

To

"
When calculating the MDT size, the important factor to consider is the number of files to be stored in the file system. This determines the number of inodes needed, which drives the MDT sizing. To be on the safe side, plan for 2 KB per inode on the MDT, which is the default value. Attached storage required for Lustre metadata is typically 1-2 percent of the file system capacity depending upon file size.

For example, if the average file size is 5 MB and you have 100 TB of usable OST space, then you can calculate the minimum number of inodes as follows:

(100 TB * 1024 GB/TB * 1024 MB/GB) / 5 MB/inode = 20 million inodes

We recommend that you use at least twice the minimum number of inodes to allow for future expansion and allow for an average file size smaller than expected. Thus, the required space is:

2 KB/inode * 40 million inodes = 80 GB

If the average file size is small, 4 KB for example, Lustre is not very efficient as the MDT uses as much space as the OSTs. However, this is not a common configuration for Lustre.
"

Comments

[Richard Henwood (Inactive)]

Hi Zhiqi - thanks for this.

One quick question: Should the last line also be updated to say '2 KB for example'?

If the average file size is small, 4 KB for example, Lustre is not very efficient as the MDT uses as much space as the OSTs. However, this is not a common configuration for Lustre.

[Zhiqi Tao (Inactive)]

Hi Richard,

The last line is referred to the average file size in the file system. We don't need to change the 4KB to 2KB.

Thanks,
Zhiqi

[Andreas Dilger]

This change is in http://review.whamcloud.com/1828.

This is actually only a subset of the changes that were done to mkfs.lustre in LU-255. I filed LUDOC-3 to track those changes, but as yet have not worked on that bug.

[Andreas Dilger]

The change 1828 was merged into the manual.