linux-kernel - Re: Zram writeback feature unstable with heavy swap utilization

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180724025330.GB558@jagdpanzerIV>
Date:   Tue, 24 Jul 2018 11:53:30 +0900
From:   Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>
To:     Minchan Kim <minchan@...nel.org>
Cc:     Tino Lehnig <tino.lehnig@...tabo.de>, ngupta@...are.org,
        linux-kernel@...r.kernel.org,
        Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: Zram writeback feature unstable with heavy swap utilization -
 BUG: Bad page state in process...

On (07/24/18 10:03), Minchan Kim wrote:
> On Mon, Jul 23, 2018 at 02:29:32PM +0200, Tino Lehnig wrote:
> > Hello,
> > 
> > after enabling the writeback feature in zram, I encountered the kernel bug
> > below with heavy swap utilization. There is one specific workload that
> > triggers the bug reliably and that is running Windows in KVM while
> > overcommitting memory. The Windows VMs would fill all allocated memory with
> > zero pages while booting. A few seconds after the host hits zram swap, the
> > console on the host is flooded with the bug message. A few more seconds
> > later I also encountered filesystem errors on the host causing the root
> > filesystem to be mounted read-only. The filesystem errors do not occur when
> > leaving RAM available for the host OS by limiting physical memory of the
> > QEMU processes via cgroups.
> > 
> > I started three KVM instances with the following commands in my tests. Any
> > Windows ISO or disk image can be used. Less instances and smaller allocated
> > memory will also trigger the bug as long as swapping occurs. The type of
> > writeback device does not seem to matter. I have tried a SATA SSD and an
> > NVMe Optane drive so far. My test machine has 256 GB of RAM and one CPU. I
> > saw the same behavior on another machine with two CPUs and 128 GB of RAM.
> > 
> > The bug does not occur when using zram as swap without "backing_dev" being
> > set, but I had even more severe problems when running the same test on
> > Ubuntu Kernels 4.15 and 4.17. Regardless of the writeback feature being used
> > or not, the host would eventually lock up entirely when swap is in use on
> > zram. The lockups may not be related directly to zram though and were
> > apparently fixed in 4.18. I had absolutely no problems on Ubuntu Kernel 4.13
> > either, before the writeback feature was introduced.
> 
> We didn't release v4.18 yet. Could you say what kernel tree/what version
> you used?
> 
> Now I don't have enough time to dig in.
> 
> Sergey, I really appreciate if you could have availabe time to look into.
> Anyway, I could try to see it asap if Sergey is not available.
> No worry.

Interesting case.

Will take me several days to get to it.
Sorry, quite busy at the moment.

	-ss