[LU-1187] DNE Phase 1: Remote Directories

• Type: Epic
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Epic Name: DNE Phase 1
• Assignee: Lai Siyao
• Reporter: Richard Henwood (Inactive)

Description

DNE Phase 1: Remote Direcories

This issue is for tracking progress of DNE Phase 1: Remote Directories.

Introduction

Today's HPC systems routinely configure Lustre with tens of thousands of clients. Each client draws resources from a single metadata server (MDS). The efforts of the Lustre community continue to improve the performance of the MDS, however, as client numbers continue to increase the single MDS represents a fundamental limit to filesystem scalability.

The goal of the Distributed Namespace (DNE) project is to a deliver a documented and tested implementation of Lustre that addresses this scaling limit by distributing the filesystem metadata over multiple metadata servers. This is an ambitious engineering project that will take place over a period of two years. It requires considerable engineering and testing resource and will therefore be performed in the two phases described below.

Phase 1: Remote Directories

This phase introduces a useful minimum of distributed metadata functionality. The purpose primarily to ensure that efforts concentrate on clean code restructuring for DNE. The phase focuses on extensive testing to shake out bugs and oversights not only in the implementation but also in administrative procedures. DNE brings new usage patterns that must necessarily adapt to manage multiple metadata servers.

The Lustre namespace will be distributed by allowing directory entries to reference sub-directories on different metadata targets (MDTs).  Individual directories will remain bound to a single MDT, therefore metadata throughput on single directories will stay limited by single MDS performance, but metadata throughput aggregated over distributed directories will scale.

The creation of non-local subdirectories will initially be restricted to administrators with a Lustre-specific mkdir command.  This will ensure that administrators retain control over namespace distribution to guarantee performance isolation.

Issue Links

• is duplicated by: LU-2359 fldb_seq_start() needs to check for halting condition
• is duplicated by: LU-1183 Cleanup old CMD code
• is related to: LU-1445 fid on OST landing
• is related to: LU-1186 Object update handler
• is related to: LU-1184 Delete hash methods from LMV
• is related to: LU-1185 FID on OST
• is related to: LU-991 cleanup LMV for DNE
• is related to: LUDOC-86 DNE Phase I Doc Changes
• is related to: LU-2240 implement index range lookup for osd-zfs.
• Trackbacks: Lustre Community Development in Progress Features are being developed for future Lustre releases both at Whamcloud and by other organizations in the Lustre community. These will be eligible for inclusion in future Lustre releases as per our processes

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31700/
Subject: LU-1187 idl: remove obsolete directory split flags
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5c53c353fd823df93fbee07f4c62786899584ab2

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31700
Subject: LU-1187 idl: remove obsolete directory split flags
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7b6e57a4dd1cffd91726a5b4f0b4841f034e640e

[Alexey Lyashkov]

I like to see intel land a patches without good compatibility code.
commit 75ae281dac43534f65df0113a4bf5ccaf5aedca9
Author: Wang Di <[e-mail]>
Date: Thu Jul 26 04:02:58 2012 -0700

LU-1187 mdt: Add MDS_INODELOCK_PERM lock for remote dir

+               /* Note: We have to match both LOOKUP and PERM lock
+                * here to make sure the dentry is valid and no one
+                * changing the permission.
+                * But if the client connects < 2.4 server, which will
+                * only grant LOOKUP lock, so we can only Match LOOKUP
+                * lock for old server */
+               if (exp_connect_flags(ll_i2mdexp(de->d_inode)) &&
+                                                       OBD_CONNECT_LVB_TYPE)
+                       matched_bits =
+                               MDS_INODELOCK_LOOKUP | MDS_INODELOCK_PERM;
+               else
+                       matched_bits = MDS_INODELOCK_LOOKUP;            

why you check a connect_LVB it's don't related and very different feature? if you want you must check an osd_ibits_know or introduce a new connect flag.

@@ -3565,10 +3569,11 @@ static int mdt_intent_getattr(enum mdt_it_code opcode,
 
         switch (opcode) {
         case MDT_IT_LOOKUP:
-                child_bits = MDS_INODELOCK_LOOKUP;
+               child_bits = MDS_INODELOCK_LOOKUP | MDS_INODELOCK_PERM;
                 break;
         case MDT_IT_GETATTR:
-                child_bits = MDS_INODELOCK_LOOKUP | MDS_INODELOCK_UPDATE;
+               child_bits = MDS_INODELOCK_LOOKUP | MDS_INODELOCK_UPDATE |
+                            MDS_INODELOCK_PERM;
                 break;
         default:

you should be don't send an ibits don't supported by client to him, because it's may confuse client code in some case.

patch also have many spaces at end of line, but it's don't block a landing the patch.

[Di Wang]

Test plan for DNE phaseI