lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <1f02756f-a753-459b-8e8b-85fbbe5c5126@arm.com>
Date: Wed, 19 Jun 2024 10:31:59 +0100
From: Ryan Roberts <ryan.roberts@....com>
To: Chris Li <chrisl@...nel.org>, Andrew Morton <akpm@...ux-foundation.org>
Cc: Kairui Song <kasong@...cent.com>, "Huang, Ying" <ying.huang@...el.com>,
 Kalesh Singh <kaleshsingh@...gle.com>, linux-kernel@...r.kernel.org,
 linux-mm@...ck.org, Barry Song <baohua@...nel.org>
Subject: Re: [PATCH v3 0/2] mm: swap: mTHP swap allocator base on swap cluster
 order

On 19/06/2024 10:20, Chris Li wrote:
> This is the short term solutiolns "swap cluster order" listed
> in my "Swap Abstraction" discussion slice 8 in the recent
> LSF/MM conference.
> 
> When commit 845982eb264bc "mm: swap: allow storage of all mTHP
> orders" is introduced, it only allocates the mTHP swap entries
> from new empty cluster list.  It has a fragmentation issue
> reported by Barry.
> 
> https://lore.kernel.org/all/CAGsJ_4zAcJkuW016Cfi6wicRr8N9X+GJJhgMQdSMp+Ah+NSgNQ@mail.gmail.com/
> 
> The reason is that all the empty cluster has been exhausted while
> there are planty of free swap entries to in the cluster that is
> not 100% free.
> 
> Remember the swap allocation order in the cluster.
> Keep track of the per order non full cluster list for later allocation.
> 
> User impact: For users that allocate and free mix order mTHP swapping,
> It greatly improves the success rate of the mTHP swap allocation after the
> initial phase.
> 
> Barry provides a test program to show the effect:
> https://lore.kernel.org/linux-mm/20240615084714.37499-1-21cnbao@gmail.com/
> 
> Without:
> $ mthp-swapout                                                          
> Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%   
> Iteration 2: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%                                                    
> Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%   
> Iteration 4: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%            
> Iteration 5: swpout inc: 110, swpout fallback inc: 117, Fallback percentage: 51.54%                                                 
> Iteration 6: swpout inc: 0, swpout fallback inc: 230, Fallback percentage: 100.00%          
> Iteration 7: swpout inc: 0, swpout fallback inc: 229, Fallback percentage: 100.00% 
> Iteration 8: swpout inc: 0, swpout fallback inc: 223, Fallback percentage: 100.00%                                                  
> Iteration 9: swpout inc: 0, swpout fallback inc: 224, Fallback percentage: 100.00%                                                  
> Iteration 10: swpout inc: 0, swpout fallback inc: 216, Fallback percentage: 100.00%                                                 
> Iteration 11: swpout inc: 0, swpout fallback inc: 212, Fallback percentage: 100.00%                                                 
> Iteration 12: swpout inc: 0, swpout fallback inc: 224, Fallback percentage: 100.00%                                                 
> Iteration 13: swpout inc: 0, swpout fallback inc: 214, Fallback percentage: 100.00%        
> 
> $ mthp-swapout -s
> Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 2: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 4: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 5: swpout inc: 33, swpout fallback inc: 197, Fallback percentage: 85.65%
> Iteration 6: swpout inc: 0, swpout fallback inc: 229, Fallback percentage: 100.00%
> Iteration 7: swpout inc: 0, swpout fallback inc: 223, Fallback percentage: 100.00%
> Iteration 8: swpout inc: 0, swpout fallback inc: 219, Fallback percentage: 100.00%
> Iteration 9: swpout inc: 0, swpout fallback inc: 212, Fallback percentage: 100.00%
> 
> With:
> $ mthp-swapout       
> Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%                                                    
> Iteration 2: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%                                                    
> Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%                                                    
> Iteration 4: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%       
> Iteration 5: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00% 
> Iteration 6: swpout inc: 230, swpout fallback inc: 0, Fallback percentage: 0.00%            
> ...
> Iteration 94: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 95: swpout inc: 221, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 96: swpout inc: 229, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 97: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 98: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 99: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 100: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00%
> 
> $ mthp-swapout -s
> Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 2: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 4: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 5: swpout inc: 230, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 6: swpout inc: 229, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 7: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 8: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00%
> ...
> Iteration 94: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 95: swpout inc: 212, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 96: swpout inc: 220, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 97: swpout inc: 220, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 98: swpout inc: 216, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 99: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00%
> Iteration 100: swpout inc: 225, swpout fallback inc: 0, Fallback percentage: 0.00%

Excellent!

> 
> Reported-by: Barry Song <21cnbao@...il.com>
> Signed-off-by: Chris Li <chrisl@...nel.org>
> ---
> Changes in v3:
> - Using V1 as base.
> - Rename "next" to "list" for the list field, suggested by Ying.
> - Update comment for the locking rules for cluster fields and list,
>   suggested by Ying.
> - Allocate from the nonfull list before attempting free list, suggested
>   by Kairui.

Sorry I didn't follow this original conversation. But the original intent of
having a per-cpu current cluster was to prevent interleving pages from multiple
processes and therefore optimize IO. See commit ebc2a1a69111 ("swap: make
cluster allocation per-cpu"). I wonder if this change could lead to a swap
performance regression in the common order-0 case?

Thanks,
Ryan

> - Link to v2: https://lore.kernel.org/r/20240614-swap-allocator-v2-0-2a513b4a7f2f@kernel.org
> 
> Changes in v2:
> - Abandoned.
> - Link to v1: https://lore.kernel.org/r/20240524-swap-allocator-v1-0-47861b423b26@kernel.org
> 
> ---
> Chris Li (2):
>       mm: swap: swap cluster switch to double link list
>       mm: swap: mTHP allocate swap entries from nonfull list
> 
>  include/linux/swap.h |  30 +++----
>  mm/swapfile.c        | 248 +++++++++++++++++----------------------------------
>  2 files changed, 95 insertions(+), 183 deletions(-)
> ---
> base-commit: 19b8422c5bd56fb5e7085995801c6543a98bda1f
> change-id: 20240523-swap-allocator-1534c480ece4
> 
> Best regards,


Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ