linux-kernel - Re: [BUG] Slab corruption during XFS writeback under memory pressure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <24d2f83f-5281-ab3c-9e91-985a4b8e2f8b@fb.com>
Date:	Mon, 18 Jul 2016 19:05:44 -0700
From:	Calvin Owens <calvinowens@...com>
To:	Dave Chinner <david@...morbit.com>
CC:	<linux-block@...r.kernel.org>, <kernel-team@...com>,
	<linux-kernel@...r.kernel.org>, <linux-scsi@...r.kernel.org>,
	<xfs@....sgi.com>
Subject: Re: [BUG] Slab corruption during XFS writeback under memory pressure

On 07/17/2016 11:02 PM, Dave Chinner wrote:
> On Sun, Jul 17, 2016 at 10:00:03AM +1000, Dave Chinner wrote:
>> On Fri, Jul 15, 2016 at 05:18:02PM -0700, Calvin Owens wrote:
>>> Hello all,
>>>
>>> I've found a nasty source of slab corruption. Based on seeing similar symptoms
>>> on boxes at Facebook, I suspect it's been around since at least 3.10.
>>>
>>> It only reproduces under memory pressure so far as I can tell: the issue seems
>>> to be that XFS reclaims pages from buffers that are still in use by
>>> scsi/block. I'm not sure which side the bug lies on, but I've only observed it
>>> with XFS.
> [....]
>> But this indicates that the page is under writeback at this point,
>> so that tends to indicate that the above freeing was incorrect.
>>
>> Hmmm - it's clear we've got direct reclaim involved here, and the
>> suspicion of a dirty page that has had it's bufferheads cleared.
>> Are there any other warnings in the log from XFS prior to kasan
>> throwing the error?
>
> Can you try the patch below?

Thanks for getting this out so quickly :)

So far so good: I booted Linus' tree as of this morning and reproduced the ASAN
splat. After applying your patch I haven't triggered it.

I'm a bit wary since it was hard to trigger reliably in the first place... so I
lined up a few dozen boxes to run the test case overnight. I'll confirm in the
morning (-0700) they look good.

Thanks,
Calvin

> -Dave.