linux-kernel - Re: [PATCH] fs: Preserve error code in get_empty

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20130221163605.GA13379@thunk.org>
Date:	Thu, 21 Feb 2013 11:36:05 -0500
From:	Theodore Ts'o <tytso@....edu>
To:	Al Viro <viro@...IV.linux.org.uk>
Cc:	Anatol Pomozov <anatol.pomozov@...il.com>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	Louis Huemiller <lhuemill@...gle.com>
Subject: Re: [PATCH] fs: Preserve error code in get_empty_filp()

> > A little bit of context for this change. We at Google work on a test
> > framework that shows how kernel behaves under memory pressure. In the
> > codepath that I am fixing the syscalls return ENFILE error, but in
> > fact the correct error would be ENOMEM. get_empty_filp() should
> > preserve the original error and not to replace all errors with ENFILE.
> 
> The trouble is, you are introducing previously impossible return values
> for pipe(2).  The rest of it is probably OK (even though I'd prefer to
> split get_empty_filp() part into a separate commit), but this one has
> potential for breaking previously correct userland code.  OTOH, FreeBSD has
> done that a while ago and they apparently had been able to cope with the
> fallout.

Sure, but Posix/SUSv3 has always said that system calls can return
error values that aren't listed in the standard (or the man page).
Given that most applications check for an error, and then use the
errno to log an error which a human can interpret, it would seem to me
to be better to return ENOMEM rather than to return the clearly wrong
ENFILE; after all, we could potentially have only a handful of file
descriptors open at the time when pipe(2) fails due to lack of memory,
and the error code:

       ENFILE The system limit on the total number of open files has
              been reached.

is clearly wrong.

Are you aware of any applications that would blow up if pipe(2)
returned any possible error other than ENFILE?

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/