[LU-1715] flock deadlock detection does not work properly

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.5.0
• Type: Bug
• Priority: Minor
• Reporter: Andriy Skulysh
• Assignee: Keith Mannthey (Inactive)
• Resolution: Fixed
• Labels: patch
• Severity: 3

Description

flock deadlocks are checked on the first attempt to grant the flock only. if we try again to grant it after its blocking lock is cancelled, we do not check for deadlocks which also may exist.

the reproducer is attached, its output is the following:
4730: sleeping 1
4731: taking lock0 [0, 100]
4732: taking lock1 [100, 200]
4731: done
4731 sleeping 1
4732: done
4732 sleeping 2
4730: taking lock2 [200, 300]
4730: done
4730: taking lock0 [0, 100]
4731: taking lock3 [100, 300]
4732: putting lock1 [100, 200]
4732: done
4732 Exit
--> it is waiting forever here.

Issue Links

• is related to: LU-3783 MDS crash, LustreError: 34483:0:(ldlm...

Comments

[Andriy Skulysh]

http://review.whamcloud.com/3553

[Nathan Rutman]

Xyratex-bug-id: MRP-393

[Keith Mannthey (Inactive)]

Looks like this has been sitting for a bit. Please rebase the patch if it is still a desired change.

[Andriy Skulysh]

I have a problem with adding new ldlm flag. It looks like master uses autogen to generate lustre_dlm_flags.h
I've added flag description to lustre_dlm_flags.def but observe following error :

$ pwd
/local/work/lustre-release/contrib/bit-masks
$ make
+ autogen --version
+ autogen -DTESTING=1 lustre_dlm_flags.def
/bin/bash: line 46: .: script.sh: file not found
columns: illegal option – end= };
columns (GNU AutoGen) - Columnize Input Text - Ver. 1.2
USAGE: columns [ -<flag> [<val>] | --<name>[

{=| }<val>] ]...
Flg Arg Option-Name Description
-W Num width Maximum Line Width
-c Num columns Desired number of columns
-w Num col-width Set width of each column
Num spread maximum spread added to column width
no fill Fill lines with input
-I Str indent Line prefix or indentation
Str first-indent First line prefix
Num tab-width tab width
-s opt sort Sort input text
-f Str format Formatting string for each input
-S Str separation Separation string - follows all but last
Str line-separation string at end of all lines but last
no by-columns Print entries in column order
-i Str input Input file (if not stdin)
-v opt version Output version information and exit
-? no help Display usage information and exit
-! no more-help Extended usage information passed thru pager
-> opt save-opts Save the option state to a config file
-< Str load-opts Load options from a config file

Options are specified by doubled hyphens and their name
or by a single hyphen and the flag character.

please send bug reports to: [e-mail]
columns: illegal option – end= };
columns (GNU AutoGen) - Columnize Input Text - Ver. 1.2
USAGE: columns [ -<flag> [<val>] | --<name>[{=| }

<val>] ]...
Flg Arg Option-Name Description
-W Num width Maximum Line Width
-c Num columns Desired number of columns
-w Num col-width Set width of each column
Num spread maximum spread added to column width
no fill Fill lines with input
-I Str indent Line prefix or indentation
Str first-indent First line prefix
Num tab-width tab width
-s opt sort Sort input text
-f Str format Formatting string for each input
-S Str separation Separation string - follows all but last
Str line-separation string at end of all lines but last
no by-columns Print entries in column order
-i Str input Input file (if not stdin)
-v opt version Output version information and exit
-? no help Display usage information and exit
-! no more-help Extended usage information passed thru pager
-> opt save-opts Save the option state to a config file
-< Str load-opts Load options from a config file

Options are specified by doubled hyphens and their name
or by a single hyphen and the flag character.

please send bug reports to: [e-mail]
+ rm -f ../../lustre/include/lustre_dlm_flags.h
+ sed '/It has been AutoGen-ed/s/-ed.*/-ed/;s/ *$//' lustre_dlm_flags.h

I've installed autogen-5.9.4-7 using :
$ yum --enablerepo=epel-testing install autogen

How autogen is supposed to be executed ? manually or during Lustre make ?
BTW script.sh is generated in the current directory and I managed to patch lustre_dlm_flags.tpl to execute it from current directory.

[Keith Mannthey (Inactive)]

You do need to run autogen.sh Manually. When I make Lustre by hand I run ./autogen.sh in the top level directory before I do anything else.

A rough guide for building dev builds by hand can be seen here: https://wiki.hpdd.intel.com/display/PUB/Walk-thru-+Build+Lustre+MASTER+on+RHEL+6.3+from+Whamcloud+git

My basic build steps are:
./autogen.sh
./configure ....
make

I will keep an eye on your new patch to see if it build in the autotest system.

[Andriy Skulysh]

Of course I run autogen.sh, but it doesn't generate lustre_dlm_flags.h. Another executable autogen (without .sh) http://www.gnu.org/software/autogen/ is needed to do this job.

[Andreas Dilger]

Bruce added the auto build of the LDLM flags, perhaps he can comment on how this should be done correctly, if it isn't automatically generated by make.

[Bruce Korb (Inactive)]

Bruce added the auto build of the LDLM flags, perhaps he can comment on how this should be done correctly

To have it automatically done by make would require that autogen be installed by builders of Lustre. You can install it for any distro, but you have to explicitly request it in a distro-specific way.

You do need to run autogen.sh Manually

Well, no. "autogen.sh" has nothing whatever to do with autogen. I offered to help the Gnome project regenerate some of their pre-configure sources, but they declined the help, still used the name for their bootstrap process and it became a "standard" name for bootstrapping projects. I advocate renaming autogen.sh to bootstrap.sh. Anyway, the method for rebuilding the flags (once autogen is installed) is:

cd contrib/bit-masks && make

[Bruce Korb (Inactive)]

P.S. the output should be:

$ autogen --version
autogen (GNU AutoGen) 5.17.1
$ make
>sh> autogen --version
>sh> autogen -DTESTING=1 lustre_dlm_flags.def
>sh> rm -f ../../lustre/include/lustre_dlm_flags.h
>sh> sed '/It has been AutoGen-ed/s/-ed.*/-ed/;s/ *$//' lustre_dlm_flags.h
$ 

I don't think the actual version makes much difference.

[Keith Mannthey (Inactive)]

Master patch has been merged.