linux-kernel - Re: [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAKgT0UeN=o0NratKSmhBhBBmwQ06qo9tiU9sqvpq_u8uMZPM1w@mail.gmail.com>
Date:   Wed, 6 Mar 2019 15:12:33 -0800
From:   Alexander Duyck <alexander.duyck@...il.com>
To:     "Michael S. Tsirkin" <mst@...hat.com>
Cc:     David Hildenbrand <david@...hat.com>,
        Nitesh Narayan Lal <nitesh@...hat.com>,
        kvm list <kvm@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Paolo Bonzini <pbonzini@...hat.com>, lcapitulino@...hat.com,
        pagupta@...hat.com, wei.w.wang@...el.com,
        Yang Zhang <yang.zhang.wz@...il.com>,
        Rik van Riel <riel@...riel.com>, dodgen@...gle.com,
        Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
        dhildenb@...hat.com, Andrea Arcangeli <aarcange@...hat.com>
Subject: Re: [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting

On Wed, Mar 6, 2019 at 2:18 PM Michael S. Tsirkin <mst@...hat.com> wrote:
>
> On Wed, Mar 06, 2019 at 10:40:57PM +0100, David Hildenbrand wrote:
> > On 06.03.19 21:32, Michael S. Tsirkin wrote:
> > > On Wed, Mar 06, 2019 at 07:59:57PM +0100, David Hildenbrand wrote:
> > >> On 06.03.19 19:43, Michael S. Tsirkin wrote:
> > >>> On Wed, Mar 06, 2019 at 01:30:14PM -0500, Nitesh Narayan Lal wrote:
> > >>>>>> Here are the results:
> > >>>>>>
> > >>>>>> Procedure: 3 Guests of size 5GB is launched on a single NUMA node with
> > >>>>>> total memory of 15GB and no swap. In each of the guest, memhog is run
> > >>>>>> with 5GB. Post-execution of memhog, Host memory usage is monitored by
> > >>>>>> using Free command.
> > >>>>>>
> > >>>>>> Without Hinting:
> > >>>>>>                  Time of execution    Host used memory
> > >>>>>> Guest 1:        45 seconds            5.4 GB
> > >>>>>> Guest 2:        45 seconds            10 GB
> > >>>>>> Guest 3:        1  minute               15 GB
> > >>>>>>
> > >>>>>> With Hinting:
> > >>>>>>                 Time of execution     Host used memory
> > >>>>>> Guest 1:        49 seconds            2.4 GB
> > >>>>>> Guest 2:        40 seconds            4.3 GB
> > >>>>>> Guest 3:        50 seconds            6.3 GB
> > >>>>> OK so no improvement.
> > >>>> If we are looking in terms of memory we are getting back from the guest,
> > >>>> then there is an improvement. However, if we are looking at the
> > >>>> improvement in terms of time of execution of memhog then yes there is none.
> > >>>
> > >>> Yes but the way I see it you can't overcommit this unused memory
> > >>> since guests can start using it at any time.  You timed it carefully
> > >>> such that this does not happen, but what will cause this timing on real
> > >>> guests?
> > >>
> > >> Whenever you overcommit you will need backup swap.
> > >
> > > Right and the point of hinting is that pages can just be
> > > discarded and not end up in swap.
> > >
> > >
> > > Point is you should be able to see the gain.
> > >
> > > Hinting patches cost some CPU so we need to know whether
> > > they cost too much. How much is too much? When the cost
> > > is bigger than benefit. But we can't compare CPU cycles
> > > to bytes. So we need to benchmark everything in terms of
> > > cycles.
> > >
> > >> There is no way
> > >> around it. It just makes the probability of you having to go to disk
> > >> less likely.
> > >
> > >
> > > Right and let's quantify this. Does this result in net gain or loss?
> >
> > Yes, I am totally with you. But if it is a net benefit heavily depends
> > on the setup. E.g. what kind of storage used for the swap, how fast, is
> > the same disk also used for other I/O ...
> >
> > Also, CPU is a totally different resource than I/O. While you might have
> > plenty of CPU cycles to spare, your I/O throughput might already be
> > limited. Same goes into the other direction.
> >
> > So it might not be as easy as comparing two numbers. It really depends
> > on the setup. Well, not completely true, with 0% CPU overhead we would
> > have a clear winner with hinting ;)
>
> I mean users need to know about this too.
>
> Are these hinting patches a gain:
> - on zram
> - on ssd
> - on a rotating disk
> - none of the above
> ?
>
> If users don't know when would they enable hinting?
>
> Close to one is going to try all possible configurations, test
> exhaustively and find an optimal default for their workload.
> So it's our job to figure it out and provide guidance.

Right. I think for now I will stick to testing on what I have which is
a SSD for swap, and no-overcommit for the "non of the above" case.

BTW it looks like this patch set introduced a pretty heavy penalty for
the no-overcommit case. For a 32G VM with no overcommit a 32G memhog
test is now taking over 50 seconds whereas without the patch set I can
complete the test in around 20 seconds.

> >
> > >
> > >
> > >> If you assume that all of your guests will be using all of their memory
> > >> all the time, you don't have to think about overcommiting memory in the
> > >> first place. But this is not what we usually have.
> > >
> > > Right and swap is there to support overcommit. However it
> > > was felt that hinting can be faster since it avoids IO
> > > involved in swap.
> >
> > Feels like it, I/O is prone to be slow.
> >
> >
> > --
> >
> > Thanks,
> >
> > David / dhildenb
>
> OK so should be measureable.
>
> --
> MST