[LU-1532] only allow FID_SEQ_NORMAL from clients Created: 15/Jun/12  Updated: 29/Apr/14  Resolved: 12/Nov/12

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.4.0, Lustre 2.1.4
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: John Hammond Assignee: Mikhail Pershin
Resolution: Fixed Votes: 0
Labels: fid, server
Environment:
  1. uname -r
    2.6.32-220.13.1.el6.l22.x86_64
  2. cat /proc/fs/lustre/version
    lustre: 2.2.55
    kernel: patchless_client
    build: 2.2.55--CHANGED-2.6.32-220.13.1.el6.l22.x86_64

Issue Links:
Duplicate
is duplicated by LU-1549 open-unlinked files are accessible th... Resolved
Related
is related to LU-1518 Missing/bad operations in mdd_{obf,do... Resolved
Sub-Tasks:
Key
 Summary
 Type
 Status
 Assignee
LU-1552 opening lost+found: triggers (mdt_ope... Technical task Resolved WC Triage  
Story Points: 2
Severity: 3
Rank (Obsolete): 3979

 Description   

[sanity@r62-lustre lustre]$ echo 'Achievement unlocked.' > .lustre/fid/[0xc:0x0:0x0]/SANITY
[sanity@r62-lustre lustre]$ stat .lustre/fid/[0xc:0x0:0x0]/SANITY
File: `.lustre/fid/[0xc:0x0:0x0]/SANITY'
Size: 22 Blocks: 1 IO Block: 2097152 regular file
Device: 2c54f966h/743766374d Inode: 144115205255725057 Links: 1
Access: (0664/rw-rw-r-) Uid: ( 500/ sanity) Gid: ( 500/ sanity)
Access: 2012-06-15 15:06:07.000000000 -0500
Modify: 2012-06-15 15:06:05.000000000 -0500
Change: 2012-06-15 15:06:05.000000000 -0500
[sanity@r62-lustre lustre]$ cat .lustre/fid/[0xc:0x0:0x0]/SANITY
Achievement unlocked.

[root@r62-lustre lustre-release]# debugfs /dev/loop0
debugfs 1.41.90.wc4 (01-Sep-2011)
debugfs: ls CONFIGS
12 (12) . 2 (12) .. 13 (20) mountdata 15 (20) _mgs-sptlrpc
16 (24) lustre-client 17 (24) lustre-sptlrpc 18 (24) lustre-MDT0000
19 (24) lustre-params 21 (28) changelog_catalog
22 (44) changelog_users 93 (16) sptlrpc 95 (24) lustre-OST0000
96 (24) lustre-OST0001 100 (3800) SANITY

[sanity@r62-lustre lustre]$ rm .lustre/fid/[0xc:0x0:0x0]/mountdata

Message from syslogd@r62-lustre at Jun 15 15:09:26 ...
kernel:LustreError: 2858:0:(osd_internal.h:665:osd_fid2oi()) ASSERTION( !fid_is_igif(fid) ) failed:

Message from syslogd@r62-lustre at Jun 15 15:09:26 ...
kernel:LustreError: 2858:0:(osd_internal.h:665:osd_fid2oi()) LBUG

 Comments   
Comment by Li Wei (Inactive) [ 17/Jun/12 ]

The osd_fid2oi() side effect is being fixed under LU-1534.

Comment by Peter Jones [ 18/Jun/12 ]

Mike

Could you please look at this patch?

Thanks

Peter

Comment by John Hammond [ 18/Jun/12 ]

Hi Peter, Mike,

Sorry I haven't had time to upload a patch on this one. For newly created filesystems it may be as simple as changing the mode to 0755 when CONFIGS is created, but I haven't tried to see if this would have any unwanted side-effects. For existing filesystems, a manual chmod may be the best solution.

Note also that this issue applies to the OBJECTS and NIDTBL_VERSIONS directories.

John

Comment by Andreas Dilger [ 03/Sep/12 ]

The patch for LU-1518 (http://review.whamcloud.com/3726) blocks client access to FID_SEQ_DOT_LUSTRE for specific operations in order to prevent specific files in the .lustre tree from being renamed or otherwise modified.

However, there are still a number of problems that may arise from clients accessing files in other FID_SEQ ranges. A safer approach would be to allow client->MDT access to only specific FID_SEQ ranges, and only open up those ranges if/when there is a specific need to do so. That will make the MDT and OST "safe by default" instead of blocking problematic ranges of objects after the fact.

A preliminary list of acceptable SEQ/FIDs for the MDT:

  • "ROOT" (whatever specific FID it might be, as determined by MDT_GETSTATUS RPC and lookup-by-name)
  • FID_SEQ_ECHO (for mds-survey echo_client testing?)
  • FID_SEQ_IGIF..FID_SEQ_IGIF_MAX (only for filesystems that were upgraded from 1.8)
  • FID_SEQ_DOT_LUSTRE (blocked for specific operations at a lower level)
  • FID_SEQ_NORMAL

The "ROOT" FID may have a variety of values (IGIF for 1.8 filesystems, one FID_SEQ_SPECIAL FID for new filesystems, FID_SEQ_NORMAL for filesystems upgraded by the Xyratex 2.x upgrade tool), so it should just be allowd regardless of what FID is actually in use. This can be instantiated by the reply from the MDT_GETSTATUS RPC, which is doing a lookup-by-name and should be able to bypass the by-FID sanity checks. Similarly, the FID_SEQ_IGIF range should only be allowed for upgraded 1.8 filesystems, otherwise it would allow access to any objects in the filesystem.

I haven't thought of a good way to retroactively determine if the filesystem was newly formatted with 2.x vs. upgraded from 1.8 yet. One possibility is to set an OBD_COMPAT_IGIF flag in the last_rcvd file if the filesystem is upgraded from 1.8 (i.e. missing OBD_COMPAT_20) or if some lookup-by-name/OI Scrub finds an inode with an IGIF FID. Filesystems without OBD_COMPAT_IGIF set would refuse lookups by IGIF FIDs.

There would have to be some delay in enforcing this restriction, since existing upgraded filesystems would not have it set and it might accidentally block IGIF lookups for previously-upgraded filesystems. Maybe the setting of OBD_COMPAT_IGIF could be added for 2.1.4/2.3/master, and enforcement only in master?

Acceptable SEQ/FIDs for the OST:

  • FID_SEQ_OST_MDT0 (for legacy OST objects)
  • FID_SEQ_LLOG (do we still allow/need this for 2.4 OSTs?)
  • FID_SEQ_ECHO (obdfilter-survey testing)
  • FID_SEQ_IDIF..FID_SEQ_IDIF_MAX (for legacy OST objects)
  • FID_SEQ_NORMAL (for FID-on-OST)

Unlike the MDT, there is no need to block legacy FID_SEQ_OST_MDT0/FID_SEQ_IDIF objects from clients, since they cannot be used to access objects via a different FID namespace.

Comment by Andreas Dilger [ 19/Oct/12 ]

This patch should fix this problem for the MDS: http://review.whamcloud.com/4255

Comment by Mikhail Pershin [ 12/Nov/12 ]

Fixed by patch for LU-1531:
http://review.whamcloud.com/4255

Generated at Sat Feb 10 01:17:28 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.