linux-kernel - Re: [PATCHv2] mm/slub: fix lockups on PREEMPT && !SMP kernels

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAAmzW4MD_bj1pec-HeYEZa8SZsnp2u24mweCHn4qfLB9AwpXxg@mail.gmail.com>
Date:	Wed, 18 Mar 2015 14:59:25 +0900
From:	Joonsoo Kim <js1304@...il.com>
To:	Mark Rutland <mark.rutland@....com>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Catalin Marinas <catalin.marinas@....com>,
	Christoph Lameter <cl@...ux.com>,
	David Rientjes <rientjes@...gle.com>,
	Jesper Dangaard Brouer <brouer@...hat.com>,
	Joonsoo Kim <iamjoonsoo.kim@....com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Pekka Enberg <penberg@...nel.org>,
	Steve Capper <steve.capper@...aro.org>
Subject: Re: [PATCHv2] mm/slub: fix lockups on PREEMPT && !SMP kernels

2015-03-17 21:15 GMT+09:00 Mark Rutland <mark.rutland@....com>:
> Commit 9aabf810a67cd97e ("mm/slub: optimize alloc/free fastpath by
> removing preemption on/off") introduced an occasional hang for kernels
> built with CONFIG_PREEMPT && !CONFIG_SMP.
>
> The problem is the following loop the patch introduced to
> slab_alloc_node and slab_free:
>
> do {
>         tid = this_cpu_read(s->cpu_slab->tid);
>         c = raw_cpu_ptr(s->cpu_slab);
> } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
>
> GCC 4.9 has been observed to hoist the load of c and c->tid above the
> loop for !SMP kernels (as in this case raw_cpu_ptr(x) is compile-time
> constant and does not force a reload). On arm64 the generated assembly
> looks like:
>
> ffffffc00016d3c4:       f9400404        ldr     x4, [x0,#8]
> ffffffc00016d3c8:       f9400401        ldr     x1, [x0,#8]
> ffffffc00016d3cc:       eb04003f        cmp     x1, x4
> ffffffc00016d3d0:       54ffffc1        b.ne    ffffffc00016d3c8 <slab_alloc_node.constprop.82+0x30>
>
> If the thread is preempted between the load of c->tid (into x1) and tid
> (into x4), and an allocation or free occurs in another thread (bumping
> the cpu_slab's tid), the thread will be stuck in the loop until
> s->cpu_slab->tid wraps, which may be forever in the absence of
> allocations/frees on the same CPU.
>
> This patch changes the loop condition to access c->tid with READ_ONCE.
> This ensures that the value is reloaded even when the compiler would
> otherwise assume it could cache the value, and also ensures that the
> load will not be torn.
>
> Signed-off-by: Mark Rutland <mark.rutland@....com>
> Cc: Andrew Morton <akpm@...ux-foundation.org>
> Cc: Catalin Marinas <catalin.marinas@....com>
> Cc: Christoph Lameter <cl@...ux.com>
> Cc: David Rientjes <rientjes@...gle.com>
> Cc: Jesper Dangaard Brouer <brouer@...hat.com>
> Cc: Joonsoo Kim <iamjoonsoo.kim@....com>
> Cc: Linus Torvalds <torvalds@...ux-foundation.org>
> Cc: Pekka Enberg <penberg@...nel.org>
> Cc: Steve Capper <steve.capper@...aro.org>
> ---
>  mm/slub.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> Since v1 [1]:
> * Do not erroneously remove the loop
>
> [1] lkml.kernel.org/r/1426261632-8911-1-git-send-email-mark.rutland@....com
>
> diff --git a/mm/slub.c b/mm/slub.c
> index 6832c4e..82c4737 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -2449,7 +2449,8 @@ redo:
>         do {
>                 tid = this_cpu_read(s->cpu_slab->tid);
>                 c = raw_cpu_ptr(s->cpu_slab);
> -       } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
> +       } while (IS_ENABLED(CONFIG_PREEMPT) &&
> +                unlikely(tid != READ_ONCE(c->tid)));
>
>         /*
>          * Irqless object alloc/free algorithm used here depends on sequence
> @@ -2718,7 +2719,8 @@ redo:
>         do {
>                 tid = this_cpu_read(s->cpu_slab->tid);
>                 c = raw_cpu_ptr(s->cpu_slab);
> -       } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
> +       } while (IS_ENABLED(CONFIG_PREEMPT) &&
> +                unlikely(tid != READ_ONCE(c->tid)));
>
>         /* Same with comment on barrier() in slab_alloc_node() */
>         barrier();
> --

Hello,

Could you show me generated code again?

What we need to check is redoing whole things in the loop.
Previous attached code seems to me that it already did
refetching c->tid in the loop and this patch looks only handle
refetching c->tid.
READ_ONCE(c->tid) will trigger redoing 'tid = this_cpu_read(s->cpu_slab->tid)'?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/