[LU-143] hash function of ldlm_namespace and lu_site is not good enough Created: 17/Mar/11  Updated: 04/Jun/16  Resolved: 12/Dec/11

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.1.0
Fix Version/s: Lustre 2.2.0, Lustre 2.1.2

Type: Improvement Priority: Major
Reporter: Liang Zhen (Inactive) Assignee: Liang Zhen (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Rank (Obsolete): 4810

 Description   

After turning on CFS_HASH_DEBUG_2, we can see hundreds or more traveling on list of these hash-tables if one single client create/stat/unlink millions of files under a single directory, which is bad for performance.
I've worked out a patch to improve the hash function, will post for review very soon.

 Comments   
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in lustre-reviews » client,el5 #22
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in reviews-centos5 #597
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in lustre-reviews » server,el6 #22
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in lustre-reviews » server,el5 #22
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in lustre-reviews » client,el6 #22
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in lustre-reviews » client,ubuntu #22
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 29/Mar/11 ]

Integrated in reviews-centos5 #606
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • libcfs/libcfs/hash.c
  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 31/Mar/11 ]

Integrated in lustre-reviews » client,el5 #34
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 31/Mar/11 ]

Integrated in lustre-reviews » client,ubuntu #34
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 31/Mar/11 ]

Integrated in lustre-reviews » client,el6 #34
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 31/Mar/11 ]

Integrated in lustre-reviews » server,el5 #34
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 31/Mar/11 ]

Integrated in lustre-reviews » server,el6 #34
LU-143 fid hash improvements

Liang Zhen : 28609bf7d475e0fae951ddca60693114c7d353d5
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by James A Simmons [ 02/Nov/11 ]

What ever happened to this patch?

Comment by Liang Zhen (Inactive) [ 02/Nov/11 ]

James, I will try to push this patch to 2.2, before that I need to gather some performance data.

Thanks
Liang

Comment by James A Simmons [ 18/Nov/11 ]

Just to let you know I have been using this patch for a long time with no problems.

Comment by Peter Jones [ 12/Dec/11 ]

Landed for 2.2

Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,client,sles11,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,client,el6,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,server,el6,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,client,el5,ofa #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,client,el6,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
  • lustre/obdclass/lu_object.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,client,el5,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,client,ubuntu1004,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,server,el5,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • libcfs/libcfs/hash.c
  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,client,el5,ofa #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,client,el5,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » x86_64,server,el5,ofa #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = FAILURE
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,server,el5,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/obdclass/lu_object.c
  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,server,el6,inkernel #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • lustre/ldlm/ldlm_resource.c
  • libcfs/libcfs/hash.c
  • lustre/obdclass/lu_object.c
Comment by Build Master (Inactive) [ 12/Dec/11 ]

Integrated in lustre-master » i686,server,el5,ofa #378
LU-143 fid hash improvements (Revision 23d5254c8d607a0ae1065853011ecfc82f0543a2)

Result = SUCCESS
Oleg Drokin : 23d5254c8d607a0ae1065853011ecfc82f0543a2
Files :

  • libcfs/libcfs/hash.c
  • lustre/ldlm/ldlm_resource.c
  • lustre/obdclass/lu_object.c
Generated at Sat Feb 10 01:04:10 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.