linux-kernel - [PATCH 1/2] fs/pipe.c: preserve alloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1445117169-9665-1-git-send-email-ebiggers3@gmail.com>
Date:	Sat, 17 Oct 2015 16:26:08 -0500
From:	Eric Biggers <ebiggers3@...il.com>
To:	linux-fsdevel@...r.kernel.org
Cc:	linux-kernel@...r.kernel.org, viro@...iv.linux.org.uk,
	Eric Biggers <ebiggers3@...il.com>
Subject: [PATCH 1/2] fs/pipe.c: preserve alloc_file() error code

If sys_pipe() was unable to allocate a 'struct file', it always failed
with ENFILE, which means "The number of simultaneously open files in the
system would exceed a system-imposed limit." However, alloc_file()
actually returns an ERR_PTR value and might fail with other error codes.
Currently, in addition to ENFILE, it can fail with ENOMEM, potentially
when there are few open files in the system.  Update sys_pipe() to
preserve this error code.

In a prior submission of a similar patch (1) some concern was raised
about introducing a new error code for sys_pipe().  However, for most
system calls, programs cannot assume that new error codes will never be
introduced.  In addition, ENOMEM was, in fact, already a possible error
code for sys_pipe(), in the case where the file descriptor table could
not be expanded due to insufficient memory.

	(1) http://comments.gmane.org/gmane.linux.kernel/1357942

Signed-off-by: Eric Biggers <ebiggers3@...il.com>
---
 fs/pipe.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/fs/pipe.c b/fs/pipe.c
index 8865f79..997de34 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -693,17 +693,20 @@ int create_pipe_files(struct file **res, int flags)

 	d_instantiate(path.dentry, inode);

-	err = -ENFILE;
 	f = alloc_file(&path, FMODE_WRITE, &pipefifo_fops);
-	if (IS_ERR(f))
+	if (IS_ERR(f)) {
+		err = PTR_ERR(f);
 		goto err_dentry;
+	}

 	f->f_flags = O_WRONLY | (flags & (O_NONBLOCK | O_DIRECT));
 	f->private_data = inode->i_pipe;

 	res[0] = alloc_file(&path, FMODE_READ, &pipefifo_fops);
-	if (IS_ERR(res[0]))
+	if (IS_ERR(res[0])) {
+		err = PTR_ERR(res[0]);
 		goto err_file;
+	}

 	path_get(&path);
 	res[0]->private_data = inode->i_pipe;
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/