Changes to the filesystem while find is running - comments?

[James Youngman]

Hello,

GNU find does a sanity check before and after calling chdir(). It
stats the directory it's about to move into, and stats "." after
chdir() succeeds. It then compares the device numbers and inode
numbers returned by he two stat() calls. This is done in a function
called wd_sanity_check(). If these are different, find prints an
error and exits fatally (in versions up to and including GNU findutils
4.2.5.

The intent here is to detect the situation when someone is moving
directories about while find is running. This could be done with
malicious intent. The thinking behind this is that (for example)
there is a risk that find be persuaded to search a part of the
filesystem which it hadn't planned to (e.g. find /foo -depth -delete).

If you are running an automounter, it will often mount a filesystem
when triggered by the fact that a process chdir()s into a mount point.
This will obviously trigger a failure in the check that
wd_sanity_check() performs. Hence for many versions of findutils,
find will not be able to search a directory hierarchy containing an
automountd mount point. That's obviously undesirable.

I have recently started trying to figure out a way to solve this
problem without entirely disabling wd_sanity_check().

As of findutils 4.2.6/4.2.7, find will read /etc/mtab (/etc/mnttab, or
whatever it is called on the current system) to determine if the
directory it has just moved into has recently been either mounted or
unmounted. It does this by reading the mount table at startup and
again when wd_sanity_check() detects a problem (if the filesystem
appears in one list but not the other, it has either been mounted or
unmounted). If ...

1. the relevant directory has recently been mounted or unmounted
AND
2. find is travelling down the directory hierarchy, not up

.... then wd_sanity_check() allows the program to continue but prints a
warning. Otherwise, it will exit fatally as before. The rationale
behind being sensitive to the direction of the chdir() is that
chdir("..") shouldn't cause a filesystem to become mounted and
although the automountd might time a filesystem out and unmount it,
this only happens after a period of inactivity, and find will have
just been using this particular filesystem. The check in
wd_sanity_check() doesn't pay attention to the actual type of the
filesystem (though it does report the type in the error message).

I have run into a problem as of findutils-4.2.7. This is simply that
there seem to be cases where automountd on Solaris works by exchanging
one mounted filesystem for another. I could support/allow this
behaviour by noticing that the device number for the filesystem has
changed and allow the wd_sanity_check() test to succeed, but then the
effect of this change is that almost any combination of changes to the
filesystem device number is permitted. Hence my question is :-

Is it worthwhile at all to perform this "sanity check" or is it of
insufficient benefit?

If there is some benefit in making a test of this type, what scenarios
should find watch out for? What tests are useful and practical to
help find determine if someone is trying to exploit or mislead it?

I'm not considering here cases where the find command line is
specified by an untrusted data source - for the purposes of this
discussion the source of untrusted data is the directory hierarchy
that find is traversing.

I'd be grateful for your thoughts.

Regards,
James.

[Dmitry V. Levin]

--2oS5YaxWCcQjTEyO
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi,

On Mon, Nov 22, 2004 at 10:05:04AM -0800, Martin Buchholz wrote:
[...]
> If we have fchdir, I see find as maintaining a stack of
> file descriptors to directories that have been chdir'ed into.
[...]
> In general, use fchdir and fstat whenever possible.

find utility implementation should not base entirely on stack of file
descriptors, due to open file descriptors limits.

Same issues arise with fts(3)/ftw(3) implementations and its clones.


--=20
ldv

--2oS5YaxWCcQjTEyO
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFBojDp9viEa8HiNCkRApfjAJ9zq7T66c9RO0junZS5qI DMPhnFKQCePS3M
M2cSBe6L7/JPuq8PHaxD1kI=
=ypRQ
-----END PGP SIGNATURE-----

--2oS5YaxWCcQjTEyO--

[Martin Buchholz]

All:

The guarantee `find' makes that symlinks are not followed is
an important one, for the sanity of users, especially sysadmins.

Replacing a directory with a symlink is a common operation.
rmdir bin; ln -s ~fred/bin
should be innocuous.

So something like find's current sanity check is valuable.

Here's an idea to make this more robust in the face of
symlinks and automounters.

Before a chdir to "foo", take stock:
- record stat("."); DOTFD = open("."); (get a fd to ".")
- record stat("foo"); (make sure foo is a regular directory)
Then
- chdir "foo"
- stat("."); compare dev, inode with recorded stat("foo")
- if different, we suspect either symlinks or automounter.
In this case, go back to original directory.
- if we have fchdir, then
fchdir(DOTFD); and try again. give up if we fail a second
time. This means that a very rare collision with an
asynchronous symlink creation will not be fatal; in my opinion
it doesn't even need a warning.
- If we don't have fchdir, getting back to the parent might be
tougher. In the case of the automounter, we can do chdir(".."),
then stat(".") and check that we're back in original directory.
If that doesn't work, we chdir("/absolute/real/path/to/parent"),
again stat(".") and compare dev/inode with saved stat of parent
directory.

If we have fchdir, I see find as maintaining a stack of
file descriptors to directories that have been chdir'ed into.

Another idea:

If we *always* use fchdir in place of chdir, we should
never risk chdir'ing into a symlink, since we always
check that the fd we get from open is a dir and not
a symlink.

In general, use fchdir and fstat whenever possible.

The functional programming people might suggest,
instead of a stack of open fd, forking and chdiring
whenever a subdir is explored. In this case
getting back to the parent is simply exit().
But this is very likely to be too inefficient
in the Real World.

Martin

James Youngman wrote:
> Hello,
>
> GNU find does a sanity check before and after calling chdir(). It
> stats the directory it's about to move into, and stats "." after
> chdir() succeeds. It then compares the device numbers and inode
> numbers returned by he two stat() calls. This is done in a function
> called wd_sanity_check(). If these are different, find prints an
> error and exits fatally (in versions up to and including GNU findutils
> 4.2.5.
>
> The intent here is to detect the situation when someone is moving
> directories about while find is running. This could be done with
> malicious intent. The thinking behind this is that (for example)
> there is a risk that find be persuaded to search a part of the
> filesystem which it hadn't planned to (e.g. find /foo -depth -delete).
>
> If you are running an automounter, it will often mount a filesystem
> when triggered by the fact that a process chdir()s into a mount point.
> This will obviously trigger a failure in the check that
> wd_sanity_check() performs. Hence for many versions of findutils,
> find will not be able to search a directory hierarchy containing an
> automountd mount point. That's obviously undesirable.
>
> I have recently started trying to figure out a way to solve this
> problem without entirely disabling wd_sanity_check().
>
> As of findutils 4.2.6/4.2.7, find will read /etc/mtab (/etc/mnttab, or
> whatever it is called on the current system) to determine if the
> directory it has just moved into has recently been either mounted or
> unmounted. It does this by reading the mount table at startup and
> again when wd_sanity_check() detects a problem (if the filesystem
> appears in one list but not the other, it has either been mounted or
> unmounted). If ...
>
> 1. the relevant directory has recently been mounted or unmounted
> AND
> 2. find is travelling down the directory hierarchy, not up
>
> ... then wd_sanity_check() allows the program to continue but prints a
> warning. Otherwise, it will exit fatally as before. The rationale
> behind being sensitive to the direction of the chdir() is that
> chdir("..") shouldn't cause a filesystem to become mounted and
> although the automountd might time a filesystem out and unmount it,
> this only happens after a period of inactivity, and find will have
> just been using this particular filesystem. The check in
> wd_sanity_check() doesn't pay attention to the actual type of the
> filesystem (though it does report the type in the error message).
>
> I have run into a problem as of findutils-4.2.7. This is simply that
> there seem to be cases where automountd on Solaris works by exchanging
> one mounted filesystem for another. I could support/allow this
> behaviour by noticing that the device number for the filesystem has
> changed and allow the wd_sanity_check() test to succeed, but then the
> effect of this change is that almost any combination of changes to the
> filesystem device number is permitted. Hence my question is :-
>
> Is it worthwhile at all to perform this "sanity check" or is it of
> insufficient benefit?
>
> If there is some benefit in making a test of this type, what scenarios
> should find watch out for? What tests are useful and practical to
> help find determine if someone is trying to exploit or mislead it?
>
> I'm not considering here cases where the find command line is
> specified by an untrusted data source - for the purposes of this
> discussion the source of untrusted data is the directory hierarchy
> that find is traversing.
>
> I'd be grateful for your thoughts.
>
> Regards,
> James.

[Paul Szabo]

James,

You asked on bugtraq:

> GNU find does a sanity check before and after calling chdir(). It
> stats the directory it's about to move into, and stats "." after
> chdir() succeeds. It then compares the device numbers and inode
> numbers returned by he two stat() calls. This is done in a function
> called wd_sanity_check(). If these are different, find prints an
> error and exits fatally (in versions up to and including GNU findutils
> 4.2.5.
> ...
> [ Shenanigans with automounter upset wd_sanity_check() ]
> As of findutils 4.2.6/4.2.7, find will read /etc/mtab ...
> [ Shenanigans in Solaris automounter upset mtab check ]
> ... Hence my question is :-
> Is it worthwhile at all to perform this "sanity check" or is it of
> insufficient benefit?
> I'd be grateful for your thoughts.

What I would like to see implemented (in some messy pseudo-code, starting
in parent directory):

PARENT=stat(".");
SUBDIR=stat("subdir");
chdir("subdir");
DOT=stat(".");
if (SUBDIR != DOT) {
Print warning message
}
else {
Go on with find (recurse)
}
chdir("..");
DOT=stat(".");
if (PARENT != DOT) {
Print message
Exit with fatal error
}

Do not descend into "dodgy" directories, but back out of them; exit fatally
if you cannot get back to solid ground.

Is this doable?

Cheers,

Paul Szabo - psz@maths.usyd.edu.au http://www.maths.usyd.edu.au:8000/u/psz/
School of Mathematics and Statistics University of Sydney 2006 Australia

[James Youngman]

On Mon, Nov 22, 2004 at 06:05:43PM -0800, Martin Buchholz wrote:

> I mean,
> - first go back to the parent directory
> - then lstat("foo"); check if it's a symlink or a real directory
> - if a symlink, then this is fishy, but it could happen non-maliciously.
> I would issue a warning, then continue, without chdir'ing into foo.
> - if a directory, then probably we've hit the automounter problem.
> Don't issue a warning; chdir("foo") again; this time it should work.

It sounds like this strategy, which is the same as we currently do,
but with a retry, is fairly sensible.

The only problem that occurs to me is that we did not recheck "foo" to
see if it still matches the predictes specified on the find command
line (think about commands like
find /z -user fred -o -name baz -print

.... here, if the original "foo" was owned by root but the "new" foo is
owned by fred, our retrying technique has ensured that we have
wandered into a place which we are not supposed to. It's possible
that we could simply reissue process_path() to retry instead of
directly trying the chdir("foo") again. I can't remember offhand if
that is likely to be a viable strategy.

> Unlike replacing directories with symlinks, where the malicious
> possibilities are evident, I don't see any malicious possibilities
> arising out of mounted filesystems replaced by other filesystems.

Is there a consensus agreeing with this point of view? If so, that
would make the implementsation much simpler...


> Hmmm... You're right. I guess you'd have to:
> FD=open("foo");
> ST1=lstat("foo");

I think find would have to use xstat() there because "-L" ("-follow")
might be in effect, problematic though it is security-wise. xstat in
GNU find is a function pointer that points to lstat() if the -P flag
is in effect (this is the default; the explicit -P is a BSD
invention), points to stat() if the -L flag is in effect, and points
to a more complex function optionh_stat() if -H is in effect
(optionh_stat() eventually calls either stat() or lstat()).

> ST2=fstat(FD);
> compare(ST1,ST2);
> fchdir(FD);

I wonder what happens there if we're at an automount mount point.
Does the fchdir() provoke automount() into mounting the filesystem?
Would you be able to check this, Martin?

> On a related note,
> Solaris has some interesting non-standard functions:
>
> int openat(int fildes, const char *path, int oflag, /*
> mode_t mode */...);
>
> The openat() function is identical to the open() function
> except that the path argument is interpreted relative to the
> starting point implied by the fd argument. If the fd argu-
> ment has the special value AT_FDCWD, a relative path argu-
> ment will be resolved relative to the current working direc-
> tory. If the path argument is absolute, the fd argument is
> ignored.

So, with an absolute path or with AT_FDCWD, it's equivalent to
open(2)? If'a a shame that openat() doesn't have a flag to prevent it
following symbolic links. Or does it?

[James Youngman]

On Tue, Nov 23, 2004 at 12:38:52PM +1100, Paul Szabo wrote:

> [ James wrote ]

> > This algorithm is certainly robust, but it will never descend into an
> > automount directory hierarchy. Do you think we can do better than
> > that without opening the door to more exploits?
>
> Hmm... It would not descend into just-now-changed automounts (and it may
> not be able to get back out of them), but it should be able to traverse
> reasonably long-lived mounts.

The problem is though that when you chdir() into an automount mount
point, automount aill automatically mount it for you. Hence if an
automount filesystem wasn't already mounted, if you chdir() into it it
immediately becomes a "just-now-changed" mount point. That's the
essensce of the problem I am trying to solve.

Regards,
James

[Martin Buchholz]

James Youngman wrote:
> On Mon, Nov 22, 2004 at 10:05:04AM -0800, Martin Buchholz wrote:
>
>
>>Here's an idea to make this more robust in the face of
>>symlinks and automounters.
>>
>>Before a chdir to "foo", take stock:
>>- record stat("."); DOTFD = open("."); (get a fd to ".")
>>- record stat("foo"); (make sure foo is a regular directory)
>>Then
>>- chdir "foo"
>>- stat("."); compare dev, inode with recorded stat("foo")
>>- if different, we suspect either symlinks or automounter.
>
>
> With the exception of having DOTFD, this is what GNU find currently
> does, and has done for some time.

DOTFD allows us to fchdir back to where we were more reliably.
Except that we use up an extra file descriptor.

>>In this case, go back to original directory.
>>- if we have fchdir, then
>> fchdir(DOTFD); and try again.
>
>
> Do you mean, just reissue chdir("foo"), or begin again with the stat()
> call? If the latter, haven't we failed to spot an attempt to
> decieve find?
>
> I'd like to complete this line of enquiry, because it's an answer to
> this that I'm really seeking.

I mean,
- first go back to the parent directory
- then lstat("foo"); check if it's a symlink or a real directory
- if a symlink, then this is fishy, but it could happen non-maliciously.
I would issue a warning, then continue, without chdir'ing into foo.
- if a directory, then probably we've hit the automounter problem.
Don't issue a warning; chdir("foo") again; this time it should work.
If not, hmmmm... Perhaps we got a SIGSTOP at the wrong time and
got restarted 10 minutes later....Try a third time; if that doesn't
work, issue a warning, and continue without chdir'ing into foo.

Unlike replacing directories with symlinks, where the malicious
possibilities are evident, I don't see any malicious possibilities
arising out of mounted filesystems replaced by other filesystems.
If bad guys can mount filesystems in arbitrary locations, you're in
trouble anyways.

>>If we have fchdir, I see find as maintaining a stack of
>>file descriptors to directories that have been chdir'ed into.
>
>
> I can see that that would be useful but it would fail to comply with
> the POSIX standard, which specifies:
>
> The find utility shall be able to descend to arbitrary
> depths in a file hierarchy and shall not fail due to path
> length limitations (unless a path operand specified by the
> application exceeds {PATH_MAX} requirements)

The above does not make it completely clear that find must be completely
free of non-path-length resource constraints. Nevertheless, your point
that filedescriptors are depressingly, still a scarce resource, is
well taken. I suppose you could play games with the file descriptor
limits, and on a system with either infinite or large limits,
use the stack of fd idea. Or on a system where file descriptor limits
are per-process, you could use a stack of fds until you hit the resource
limit, then fall back to doing things the other way (i.e. chdir ("..")).

But that would be a lot of work to get right.

>
>>Another idea:
>>
>>If we *always* use fchdir in place of chdir, we should
>>never risk chdir'ing into a symlink, since we always
>>check that the fd we get from open is a dir and not
>>a symlink.
>
>
> Of course, open(2) will follow a symlink, if the directory we
> originally stat()ed is replaced by a symlink just before we issue the
> open() call. We of course can guard against that by issuing an
> lstat() on the fd once we have opened it.

Hmmm... You're right. I guess you'd have to:
FD=open("foo");
ST1=lstat("foo");
ST2=fstat(FD);
compare(ST1,ST2);
fchdir(FD);

This might have the same performance characteristics as the
current implementation, since we save the stat(".") after
we chdir.

> Thanks for your thoughts,
> James.

On a related note,
Solaris has some interesting non-standard functions:

int openat(int fildes, const char *path, int oflag, /*
mode_t mode */...);

The openat() function is identical to the open() function
except that the path argument is interpreted relative to the
starting point implied by the fd argument. If the fd argu-
ment has the special value AT_FDCWD, a relative path argu-
ment will be resolved relative to the current working direc-
tory. If the path argument is absolute, the fd argument is
ignored.


int fstatat(int fildes, const char *path, struct stat *buf,
int flag);

The fstatat() function obtains file attributes similar to
the stat(), lstat(), and fstat() functions. If the path
argument is a relative path, it is resolved relative to the
fildes argument rather than the current working directory.
If path is absolute, the fildes argument is unused. If the
fildes argument has the special value AT_FDCWD, defined in
<fcntl.h>, relative paths are resolved from the current
working directory. If the flag argument is
AT_SYMLNK_NOFOLLOW, defined in <fcntl.h>, the function
behaves like lstat() and does not automatically follow sym-
bolic links. See fsattr(5).


Thanks for your hard work maintaining this very important tool.

Martin

[Paul Szabo]

James,

>> Hmm... It would not descend into just-now-changed automounts (and it may
>> not be able to get back out of them), but it should be able to traverse
>> reasonably long-lived mounts.
>
> The problem is though that when you chdir() into an automount mount
> point, automount aill automatically mount it for you. Hence if an
> automount filesystem wasn't already mounted, if you chdir() into it it
> immediately becomes a "just-now-changed" mount point. That's the
> essensce of the problem I am trying to solve.

I think find should never cause an automount to "trigger" and cause it to
be mounted. It is OK to traverse if it was mounted to start with; is surely
not OK to traverse if it wasn't already mounted. Maybe your problem is
sidestepped by this principle?

[Right now cannot think of examples where find causing automounts to
trigger would be an obvious security or performance issue.]

To prevent find from causing an automount to trigger, maybe you could
somehow detect the presence of the mount point, check its status, and
(after a warning) not descend if it wasn't mounted. [I use the Debian
autofs package; this uses a normally empty directory, which is populated
with mounted directories when in use. Are we talking about the short time
between the mkdir of the mountpoint and the mount?]

Cheers,

Paul Szabo - psz@maths.usyd.edu.au http://www.maths.usyd.edu.au:8000/u/psz/
School of Mathematics and Statistics University of Sydney 2006 Australia

[Martin Buchholz]

James Youngman wrote:
> On Mon, Nov 22, 2004 at 06:05:43PM -0800, Martin Buchholz wrote:
>
>
>>I mean,
>>- first go back to the parent directory
>>- then lstat("foo"); check if it's a symlink or a real directory
>>- if a symlink, then this is fishy, but it could happen non-maliciously.
>> I would issue a warning, then continue, without chdir'ing into foo.
>>- if a directory, then probably we've hit the automounter problem.
>> Don't issue a warning; chdir("foo") again; this time it should work.
>
>
> It sounds like this strategy, which is the same as we currently do,
> but with a retry, is fairly sensible.
>
> The only problem that occurs to me is that we did not recheck "foo" to
> see if it still matches the predictes specified on the find command
> line (think about commands like
> find /z -user fred -o -name baz -print
>
> ... here, if the original "foo" was owned by root but the "new" foo is
> owned by fred, our retrying technique has ensured that we have
> wandered into a place which we are not supposed to. It's possible
> that we could simply reissue process_path() to retry instead of
> directly trying the chdir("foo") again. I can't remember offhand if
> that is likely to be a viable strategy.

I don't know the details of find's implementation.
Conceptually, I would want to forget everything I knew about "foo",
except that we've already tried it, So re-process this path again
in its entirety. This way, when find reports a file, it was
at least reporting the true state of the file AT SOME POINT.

E.g. if a file is changing from a regular file owned by Bob,
to a directory owned by Alice, then
find -user Bob -type d
should never report that file.

>
>>Unlike replacing directories with symlinks, where the malicious
>>possibilities are evident, I don't see any malicious possibilities
>>arising out of mounted filesystems replaced by other filesystems.
>
>
> Is there a consensus agreeing with this point of view? If so, that
> would make the implementsation much simpler...
>
>
>
>>Hmmm... You're right. I guess you'd have to:
>>FD=open("foo");
>>ST1=lstat("foo");
>
>
> I think find would have to use xstat() there because "-L" ("-follow")
> might be in effect, problematic though it is security-wise. xstat in
> GNU find is a function pointer that points to lstat() if the -P flag
> is in effect (this is the default; the explicit -P is a BSD
> invention), points to stat() if the -L flag is in effect, and points
> to a more complex function optionh_stat() if -H is in effect
> (optionh_stat() eventually calls either stat() or lstat()).

My suggestion is very high-level pseudocode. There are a lot of details
to get right, as you point out.

>
>>ST2=fstat(FD);
>>compare(ST1,ST2);
>>fchdir(FD);
>
>
> I wonder what happens there if we're at an automount mount point.
> Does the fchdir() provoke automount() into mounting the filesystem?
> Would you be able to check this, Martin?

It does. Here's a test program, and a sample run against an
automounter mount point:

-----------------------------------------------------------------
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>

void printID(char *file)
{
struct stat s;
if (lstat(file,&s) < 0) exit(1);
printf("file=\"%s\" dev=%ld ino=%ld\n",
file, (long) s.st_dev, (long) s.st_ino);
}

int main (int argc, char *argv[])
{
char *d = argv[1];
printID(d);
int fdParent = open(".", O_RDONLY);
int fdChild = open(d, O_RDONLY);
if (fchdir(fdChild) < 0) exit(1);
printID(".");
printID("..");
if (fchdir(fdParent) < 0) exit(1);
printID(d);
}
-----------------------------------------------------------------

$ ~/src/a.out make
file="make" dev=81033283 ino=186105
file="." dev=80744820 ino=1353076
file=".." dev=81033281 ino=186097
file="make" dev=80744820 ino=1353076
$ ~/src/a.out make
file="make" dev=80744820 ino=1353076
file="." dev=80744820 ino=1353076
file=".." dev=81033281 ino=186097
file="make" dev=80744820 ino=1353076



>
>>On a related note,
>>Solaris has some interesting non-standard functions:
>>
>> int openat(int fildes, const char *path, int oflag, /*
>> mode_t mode */...);
>>
>> The openat() function is identical to the open() function
>> except that the path argument is interpreted relative to the
>> starting point implied by the fd argument. If the fd argu-
>> ment has the special value AT_FDCWD, a relative path argu-
>> ment will be resolved relative to the current working direc-
>> tory. If the path argument is absolute, the fd argument is
>> ignored.
>
>
> So, with an absolute path or with AT_FDCWD, it's equivalent to
> open(2)? If'a a shame that openat() doesn't have a flag to prevent it
> following symbolic links. Or does it?
>

fstatat has such a flag, bug openat does not.

Another interesting function is


DIR *fdopendir(int fildes);

The fdopendir() function opens a directory stream for the
directory file descriptor fildes. The directory file
discriptor should not be used or closed following a success-
ful function call, as this might cause undefined results
from future operations on the directory stream obtained from
the call. Use closedir(3C) to close a directory stream.

BTW, all the Sun man pages are easily googlable with a query like

openat site:docs.sun.com

Martin

[Martin Buchholz]

Paul Szabo wrote:
> James,
>
>
>>>Hmm... It would not descend into just-now-changed automounts (and it may
>>>not be able to get back out of them), but it should be able to traverse
>>>reasonably long-lived mounts.
>>
>>The problem is though that when you chdir() into an automount mount
>>point, automount aill automatically mount it for you. Hence if an
>>automount filesystem wasn't already mounted, if you chdir() into it it
>>immediately becomes a "just-now-changed" mount point. That's the
>>essensce of the problem I am trying to solve.
>
>
> I think find should never cause an automount to "trigger" and cause it to
> be mounted. It is OK to traverse if it was mounted to start with; is surely
> not OK to traverse if it wasn't already mounted. Maybe your problem is
> sidestepped by this principle?

I strongly disagree with this principle. Automount points (at least on
Solaris) are supposed to be transparent to the user; just an
implementation optimization over having them permanently mounted.

> [Right now cannot think of examples where find causing automounts to
> trigger would be an obvious security or performance issue.]
>
> To prevent find from causing an automount to trigger, maybe you could
> somehow detect the presence of the mount point, check its status, and
> (after a warning) not descend if it wasn't mounted. [I use the Debian
> autofs package; this uses a normally empty directory, which is populated
> with mounted directories when in use. Are we talking about the short time
> between the mkdir of the mountpoint and the mount?]

Linux automounters work differently. I've never quite understood the
various implementations, but the ones that use symlinks in the
implementation are quite horrible, since the symlinks leak out and
affect user programs. Solaris' automounter is really quite
unobtrusive, except for programs like `find' that check the dev and
inode for constancy.

Martin

[Paul Szabo]

James,

>> PARENT=stat(".");
>> SUBDIR=stat("subdir");
>> chdir("subdir");
>> DOT=stat(".");
>> if (SUBDIR != DOT) {
>> Print warning message /*[1]*/
>> }
>> else {
>> Go on with find (recurse)
>> }
>> chdir("..");
>> DOT=stat(".");
>> if (PARENT != DOT) {
>> Print message
>> Exit with fatal error
>> }
>
> Certainly. In fact it's how GNU findutils is implemented, except for
> the fact that the "warning message" is a fatal error in GNU findutils
> releases up to and including 4.2.5. In later versions (we're
> currently at 4.2.7 on ftp://alpha.gnu.org/gnu/findutils) a hack was
> introduced to try to support a special case which works when we've
> traversed an automount mount point on Solaris).

So, you already do the "where did I get back to after chdir(..)" check?

> This algorithm is certainly robust, but it will never descend into an
> automount directory hierarchy. Do you think we can do better than
> that without opening the door to more exploits?

Hmm... It would not descend into just-now-changed automounts (and it may
not be able to get back out of them), but it should be able to traverse
reasonably long-lived mounts.

Cheers,

Paul Szabo - psz@maths.usyd.edu.au http://www.maths.usyd.edu.au:8000/u/psz/
School of Mathematics and Statistics University of Sydney 2006 Australia

[James Youngman]

On Mon, Nov 22, 2004 at 10:05:04AM -0800, Martin Buchholz wrote:

> Here's an idea to make this more robust in the face of
> symlinks and automounters.
>
> Before a chdir to "foo", take stock:
> - record stat("."); DOTFD = open("."); (get a fd to ".")
> - record stat("foo"); (make sure foo is a regular directory)
> Then
> - chdir "foo"
> - stat("."); compare dev, inode with recorded stat("foo")
> - if different, we suspect either symlinks or automounter.

With the exception of having DOTFD, this is what GNU find currently
does, and has done for some time.

> In this case, go back to original directory.
> - if we have fchdir, then
> fchdir(DOTFD); and try again.

Do you mean, just reissue chdir("foo"), or begin again with the stat()
call? If the latter, haven't we failed to spot an attempt to
decieve find?

I'd like to complete this line of enquiry, because it's an answer to
this that I'm really seeking.

> - If we don't have fchdir, getting back to the parent might be
> tougher. In the case of the automounter, we can do chdir(".."),
> then stat(".") and check that we're back in original directory.
> If that doesn't work, we chdir("/absolute/real/path/to/parent"),
> again stat(".") and compare dev/inode with saved stat of parent
> directory.

In the worst possible case we can fchdir() back to the directory from
which find was invoked, which we need to keep in order to support
-exec.

> If we have fchdir, I see find as maintaining a stack of
> file descriptors to directories that have been chdir'ed into.

I can see that that would be useful but it would fail to comply with
the POSIX standard, which specifies:

The find utility shall be able to descend to arbitrary
depths in a file hierarchy and shall not fail due to path
length limitations (unless a path operand specified by the
application exceeds {PATH_MAX} requirements)

> Another idea:
>
> If we *always* use fchdir in place of chdir, we should
> never risk chdir'ing into a symlink, since we always
> check that the fd we get from open is a dir and not
> a symlink.

Of course, open(2) will follow a symlink, if the directory we
originally stat()ed is replaced by a symlink just before we issue the
open() call. We of course can guard against that by issuing an
lstat() on the fd once we have opened it.

However, we should then compare the results of that fstat() against
the result we expected when we did stat("foo"). We therefore STILL
have to decide what to do if the st_dev value has changed. In other
words, fchdir() might be useful but doesn't help us resolve our
problem, I think.

> The functional programming people might suggest,
> instead of a stack of open fd, forking and chdiring
> whenever a subdir is explored.

That would also fail to comply with POSIX, I think.

Thanks for your thoughts,
James.

[James Youngman]

On Tue, Nov 23, 2004 at 07:39:56AM +1100, Paul Szabo wrote:

> What I would like to see implemented (in some messy pseudo-code, starting
> in parent directory):
>
> PARENT=stat(".");
> SUBDIR=stat("subdir");
> chdir("subdir");
> DOT=stat(".");
> if (SUBDIR != DOT) {
> Print warning message /*[1]*/
> }
> else {
> Go on with find (recurse)
> }
> chdir("..");
> DOT=stat(".");
> if (PARENT != DOT) {
> Print message
> Exit with fatal error
> }
>
> Do not descend into "dodgy" directories, but back out of them; exit fatally
> if you cannot get back to solid ground.
>
> Is this doable?

Certainly. In fact it's how GNU findutils is implemented, except for
the fact that the "warning message" is a fatal error in GNU findutils
releases up to and including 4.2.5. In later versions (we're
currently at 4.2.7 on ftp://alpha.gnu.org/gnu/findutils) a hack was
introduced to try to support a special case which works when we've
traversed an automount mount point on Solaris).

This algorithm is certainly robust, but it will never descend into an
automount directory hierarchy. Do you think we can do better than
that without opening the door to more exploits?

Regards,
James.

[Martin Buchholz]

Casper.Dik@Sun.COM wrote:
>>I can see that that would be useful but it would fail to comply with
>>the POSIX standard, which specifies:
>>
>> The find utility shall be able to descend to arbitrary
>> depths in a file hierarchy and shall not fail due to path
>> length limitations (unless a path operand specified by the
>> application exceeds {PATH_MAX} requirements)
>

> But PATH_MAX is limited and the number of file descriptors is perhaps
> not.
>
> (On Solaris, PATH_MAX is 1024 so you require at most 512 file
> descriptors to keep the stack of directories: 512 is less than the
> default hard limit of 65536 file descriptors per process [S9, S8
> and before used 1024, still >> 512)

My reading of the above paragraph from the POSIX standard is
that find is required to be able to traverse arbitrary
depths, even when the resulting path length exceeds PATH_MAX.

On my Solaris 9 system, the default file descriptor limit
appears to be 256.

I am genuinely surprised that Solaris still has such a
relatively small PATH_MAX. Linux has 4096.

Like other arbitrary system limits of its ilk, PATH_MAX
is evil, and is one of the more persuasive arguments for
getting rid of the C language and its fixed-size
stack-allocated buffers.

char path[PATH_MAX]; /* considered harmful */

Martin

> Casper

[Casper.Dik@Sun.COM]

>I am genuinely surprised that Solaris still has such a
>relatively small PATH_MAX. Linux has 4096.

Really, there are things you cannot change because of
binary compatibility. PATH_MAX is one.

Having a 4K path seems rather pointless; the longest path on my
system is 225 bytes; a factor of 4 over that borders on the ridiculous.

>Like other arbitrary system limits of its ilk, PATH_MAX
>is evil, and is one of the more persuasive arguments for
>getting rid of the C language and its fixed-size
>stack-allocated buffers.
>

>char path[PATH_MAX]; /* considered harmful */

Evil, yes, but old source code never dies.

Casper

[devnull@Rodents.Montreal.QC.CA]

(Because I'm so sick of the broken autoresponders bugtraq is full of,
I've set the From: on this mail to a black-hole address. Please use
the address in the signature if you actually want to reach me.)

>> Before a chdir to "foo", take stock:
>> - record stat("."); DOTFD = open("."); (get a fd to ".")

This is not possible if . is a search-only directory. (While find will
not work very well in such a directory, care should be taken that it
not fall over just because someone happened to chmod the read bits away
from a directory just when find happens to be in it.)

More generally, I've wanted to do this - use file descriptors as
handles onto directories - and often wished for an O_NOACCESS mode to
open() in consequence.

> Of course, open(2) will follow a symlink, if the directory we
> originally stat()ed is replaced by a symlink just before we issue the
> open() call. We of course can guard against that by issuing an
> lstat() on the fd once we have opened it.

ITYPM fstat(), not lstat(), in the last line.

/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse@rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

[Casper.Dik@Sun.COM]

>I can see that that would be useful but it would fail to comply with
>the POSIX standard, which specifies:
>
> The find utility shall be able to descend to arbitrary
> depths in a file hierarchy and shall not fail due to path
> length limitations (unless a path operand specified by the
> application exceeds {PATH_MAX} requirements)

But PATH_MAX is limited and the number of file descriptors is perhaps
not.

(On Solaris, PATH_MAX is 1024 so you require at most 512 file
descriptors to keep the stack of directories: 512 is less than the
default hard limit of 65536 file descriptors per process [S9, S8
and before used 1024, still >> 512)

Casper

[James Youngman]

On Wed, Nov 24, 2004 at 08:51:38AM +0100, Casper.Dik@Sun.COM wrote:
>
> >I can see that that would be useful but it would fail to comply with
> >the POSIX standard, which specifies:
> >
> > The find utility shall be able to descend to arbitrary
> > depths in a file hierarchy and shall not fail due to path
> > length limitations (unless a path operand specified by the
> > application exceeds {PATH_MAX} requirements)
>
> But PATH_MAX is limited and the number of file descriptors is perhaps
> not.

Systems differ. Some have no limits on the depth of a directory
hierarchy. Certainly I've created directory hierarchies of over
800,000 levels on HP-UX 9 and on Linux. GNU Hurd has no limits at all
(and therefore used not to #define PATH_MAX at all).


> (On Solaris, PATH_MAX is 1024 so you require at most 512 file
> descriptors to keep the stack of directories: 512 is less than the
> default hard limit of 65536 file descriptors per process [S9, S8
> and before used 1024, still >> 512)

A worthwhile point, but since the POSIX standard doesn't mention fd
limits here I suspect that there may be an expectation in some
quarters that find will still be able to work even if the soft limit
is much lower.

That being said, consistently using fchdir() is a robust and sensible
approach. However, I'm trying to correctly solve the problem of
secure downward movement in the directory hierarchy, which as far as I
can see is not fully solved simply by using fchdir().

Regards,
James.