linux-kernel - Re: 4.9.30 NULL pointer dereference in __remove_shared_vm

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <d61fec36-ab07-26e6-6572-5c8a58cbe393@I-love.SAKURA.ne.jp>
Date:   Thu, 8 Jun 2017 19:21:02 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:     Tommi Rantala <tommi.t.rantala@...ia.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andrea Arcangeli <aarcange@...hat.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Linux-MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: 4.9.30 NULL pointer dereference in __remove_shared_vm_struct

Tommi Rantala wrote:
> I have hit this kernel bug twice with 4.9.30 while running trinity, any 
> ideas? It's not easily reproducible.

No idea. But if you can reproduce this problem, I think you can retry with
the OOM reaper disabled (like shown below), for the latter report is 10 seconds
after the OOM reaper reclaimed memory.

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index ec9f11d..7e17242 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -560,8 +560,8 @@ static void oom_reap_task(struct task_struct *tsk)
 	struct mm_struct *mm = tsk->signal->oom_mm;

 	/* Retry the down_read_trylock(mmap_sem) a few times */
-	while (attempts++ < MAX_OOM_REAP_RETRIES && !__oom_reap_task_mm(tsk, mm))
-		schedule_timeout_idle(HZ/10);
+	//while (attempts++ < MAX_OOM_REAP_RETRIES && !__oom_reap_task_mm(tsk, mm))
+	//	schedule_timeout_idle(HZ/10);

 	if (attempts <= MAX_OOM_REAP_RETRIES)
 		goto done;

Since line 137 is atomic_inc(), file->f_inode was for some reason NULL, wasn't it?

	if (vma->vm_flags & VM_DENYWRITE)
		atomic_inc(&file_inode(file)->i_writecount);

And mmput() from exit_mm() from do_exit() is called before exit_files() is
called from do_exit(). Thus, something by error made file->f_inode == NULL,
despite quite few locations set f_inode to NULL.

# grep -nFr -- '->f_inode ' *
fs/file_table.c:168:    file->f_inode = path->dentry->d_inode;
fs/file_table.c:224:    file->f_inode = NULL;
fs/open.c:711:  f->f_inode = inode;
fs/open.c:782:  f->f_inode = NULL;
fs/overlayfs/copy_up.c:36:      if (f->f_inode == d_inode(dentry))

Maybe the OOM reaper by error reclaimed and somebody zeroed the reclaimed
page containing file->f_inode.

JFYI, 4.9.30 does not have commit 235190738aba7c5c ("oom-reaper: use
madvise_dontneed() logic to decide if unmap the VMA") backported.