linux-kernel - Re: [PATCH -mm] mm, swap: Fix race between swapoff and some swap operations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87indbnzga.fsf@yhuang-dev.intel.com>
Date:   Wed, 13 Dec 2017 10:17:41 +0800
From:   "Huang\, Ying" <ying.huang@...el.com>
To:     "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Minchan Kim <minchan@...nel.org>, <linux-mm@...ck.org>,
        <linux-kernel@...r.kernel.org>, Hugh Dickins <hughd@...gle.com>,
        Johannes Weiner <hannes@...xchg.org>,
        "Tim Chen" <tim.c.chen@...ux.intel.com>, Shaohua Li <shli@...com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        J�r�me Glisse <jglisse@...hat.com>,
        Michal Hocko <mhocko@...e.com>,
        Andrea Arcangeli <aarcange@...hat.com>,
        David Rientjes <rientjes@...gle.com>,
        Rik van Riel <riel@...hat.com>, Jan Kara <jack@...e.cz>,
        Dave Jiang <dave.jiang@...el.com>,
        Aaron Lu <aaron.lu@...el.com>
Subject: Re: [PATCH -mm] mm, swap: Fix race between swapoff and some swap operations

"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> writes:

> On Tue, Dec 12, 2017 at 09:12:20AM +0800, Huang, Ying wrote:
>> Hi, Pual,
>> 
>> "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> writes:
>> 
>> > On Mon, Dec 11, 2017 at 01:30:03PM +0800, Huang, Ying wrote:
>> >> Andrew Morton <akpm@...ux-foundation.org> writes:
>> >> 
>> >> > On Fri, 08 Dec 2017 16:41:38 +0800 "Huang\, Ying" <ying.huang@...el.com> wrote:
>> >> >
>> >> >> > Why do we need srcu here? Is it enough with rcu like below?
>> >> >> >
>> >> >> > It might have a bug/room to be optimized about performance/naming.
>> >> >> > I just wanted to show my intention.
>> >> >> 
>> >> >> Yes.  rcu should work too.  But if we use rcu, it may need to be called
>> >> >> several times to make sure the swap device under us doesn't go away, for
>> >> >> example, when checking si->max in __swp_swapcount() and
>> >> >> add_swap_count_continuation().  And I found we need rcu to protect swap
>> >> >> cache radix tree array too.  So I think it may be better to use one
>> >> >> calling to srcu_read_lock/unlock() instead of multiple callings to
>> >> >> rcu_read_lock/unlock().
>> >> >
>> >> > Or use stop_machine() ;)  It's very crude but it sure is simple.  Does
>> >> > anyone have a swapoff-intensive workload?
>> >> 
>> >> Sorry, I don't know how to solve the problem with stop_machine().
>> >> 
>> >> The problem we try to resolved is that, we have a swap entry, but that
>> >> swap entry can become invalid because of swappoff between we check it
>> >> and we use it.  So we need to prevent swapoff to be run between checking
>> >> and using.
>> >> 
>> >> I don't know how to use stop_machine() in swapoff to wait for all users
>> >> of swap entry to finish.  Anyone can help me on this?
>> >
>> > You can think of stop_machine() as being sort of like a reader-writer
>> > lock.  The readers can be any section of code with preemption disabled,
>> > and the writer is the function passed to stop_machine().
>> >
>> > Users running real-time applications on Linux don't tend to like
>> > stop_machine() much, but perhaps it is nevertheless the right tool
>> > for this particular job.
>> 
>> Thanks a lot for explanation!  Now I understand this.
>> 
>> Another question, for this specific problem, I think both stop_machine()
>> based solution and rcu_read_lock/unlock() + synchronize_rcu() based
>> solution work.  If so, what is the difference between them?  I guess rcu
>> based solution will be a little better for real-time applications?  So
>> what is the advantage of stop_machine() based solution?
>
> The stop_machine() solution places similar restrictions on readers as
> does rcu_read_lock/unlock() + synchronize_rcu(), if that is what you
> are asking.
>
> More precisely, the stop_machine() solution places exactly the
> same restrictions on readers as does preempt_disable/enable() and
> synchronize_sched().
>
> I would expect stop_machine() to be faster than either synchronize_rcu()
> synchronize_sched(), or synchronize_srcu(), but stop_machine() operates
> by making each CPU spin with interrupts until all the other CPUs arrive.
> This normally does not make real-time people happy.
>
> An compromise position is available in the form of
> synchronize_rcu_expedited() and synchronize_sched_expedited().  These
> are faster than their non-expedited counterparts, and only momentarily
> disturb each CPU, rather than spinning with interrupts disabled.  However,
> stop_machine() is probably a bit faster.
>
> Finally, syncrhonize_srcu_expedited() is reasonably fast, but
> avoids disturbing other CPUs.  Last I checked, not quite as fast as
> synchronize_rcu_expedited() and synchronize_sched_expedited(), though.
>
> You asked!  ;-)

Thanks a lot Paul!  That exceeds my expectation!

The performance of swapoff() isn't very important, probably it's not
necessary to accelerate it at the cost of realtime.  I think it is
better to use a rcu or srcu based solution.  I think the cost at reader
side should be almost same between rcu and srcu?  To use srcu, we need
to select CONFIG_SRCU when CONFIG_SWAP is enabled in Kconfig.  I think
that should be OK?

Best Regards,
Huang, Ying