linux-kernel - Re: Kmemleak infrastructure improvement for task_struct leaks and call

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200506174019.GA2869@paulmck-ThinkPad-P72>
Date:   Wed, 6 May 2020 10:40:19 -0700
From:   "Paul E. McKenney" <paulmck@...nel.org>
To:     Qian Cai <cai@....pw>
Cc:     Catalin Marinas <catalin.marinas@....com>,
        Linux-MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: Kmemleak infrastructure improvement for task_struct leaks and
 call_rcu()

On Wed, May 06, 2020 at 12:22:37PM -0400, Qian Cai wrote:
> == task struck leaks ==
> There are leaks from task struct from time to time where someone forgot to call put_task_struct() somewhere leading to leaks. For example,
> 
> https://lore.kernel.org/lkml/C1CCBDAC-A453-4FF2-908F-0B6E356223D1@lca.pw/
> 
> It was such a pain to debug this kind of leaks at the moment, as all we could do was to audit the code by checking all new put_task_struct()  and get_task_struct() call sites which is error-prone because there could be other new call sites like get_pid_task() which would also need to be balanced with put_task_struct() as well.
> 
> What do you think about adding some aux call traces for kmemleak in general? For example, if the tracking object is a task struct, it would save call traces for the first and last call of both get_task_struct() and put_task_struct(). Then, it could be expanded to track other refcount-based leaks in the future.
> 
> == call_rcu() leaks ==
> Another issue that might be relevant is that it seems sometimes, kmemleak will give a lot of false positives (hundreds) because the memory was supposed to be freed by call_rcu()  (for example, in dst_release()) but for some reasons, it takes a long time probably waiting for grace periods or some kind of RCU self-stall, but the memory had already became an orphan. I am not sure how we are going to resolve this properly until we have to figure out why call_rcu() is taking so long to finish?

I know nothing about kmemleak, but I won't let that stop me from making
random suggestions...

One approach is to do an rcu_barrier() inside kmemleak just before
printing leaked blocks, and check to see if any are still leaked after
the rcu_barrier().

If kmemleak works on crash dumps, another approach is to scan RCU's
callback lists.  This will miss those callbacks that rcu_do_batch()
was in the middle of invoking, though.  It also misses cases where
someone passes a linked structure to call_rcu(), and then frees the
structure piece by piece within the callback function.

> Another solution is to add aux call traces for both skb_dst_drop() and skb_dst_set() for this case, but that there are many places to free memory via call_rcu() like inode free etc.

And call_rcu() has no idea where the memory starts.  And again, sometimes
there is memory linked from that passed to call_rcu() that will be freed
by the callback function.

In theory, these linked-structure cases could be handled by checking
the callback function and then traversing the links.  I wouldn't be
that ambitious, but don't let me discourage you.  ;-)

							Thanx, Paul