lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <wnwfgk32wyvx7tzd522ajwk5uixls7iayksrtho6c3dkvgdpek@25yqv3ohljzc>
Date: Wed, 20 Nov 2024 16:04:59 -0800
From: Shakeel Butt <shakeel.butt@...ux.dev>
To: Suren Baghdasaryan <surenb@...gle.com>
Cc: akpm@...ux-foundation.org, willy@...radead.org, 
	liam.howlett@...cle.com, lorenzo.stoakes@...cle.com, mhocko@...e.com, vbabka@...e.cz, 
	hannes@...xchg.org, mjguzik@...il.com, oliver.sang@...el.com, 
	mgorman@...hsingularity.net, david@...hat.com, peterx@...hat.com, oleg@...hat.com, 
	dave@...olabs.net, paulmck@...nel.org, brauner@...nel.org, dhowells@...hat.com, 
	hdanton@...a.com, hughd@...gle.com, minchan@...gle.com, jannh@...gle.com, 
	souravpanda@...gle.com, pasha.tatashin@...een.com, corbet@....net, 
	linux-doc@...r.kernel.org, linux-mm@...ck.org, linux-kernel@...r.kernel.org, 
	kernel-team@...roid.com
Subject: Re: [PATCH v4 2/5] mm: move per-vma lock into vm_area_struct

On Wed, Nov 20, 2024 at 03:44:29PM -0800, Suren Baghdasaryan wrote:
> On Wed, Nov 20, 2024 at 3:33 PM Shakeel Butt <shakeel.butt@...ux.dev> wrote:
> >
> > On Tue, Nov 19, 2024 at 04:08:23PM -0800, Suren Baghdasaryan wrote:
> > > Back when per-vma locks were introduces, vm_lock was moved out of
> > > vm_area_struct in [1] because of the performance regression caused by
> > > false cacheline sharing. Recent investigation [2] revealed that the
> > > regressions is limited to a rather old Broadwell microarchitecture and
> > > even there it can be mitigated by disabling adjacent cacheline
> > > prefetching, see [3].
> > > Splitting single logical structure into multiple ones leads to more
> > > complicated management, extra pointer dereferences and overall less
> > > maintainable code. When that split-away part is a lock, it complicates
> > > things even further. With no performance benefits, there are no reasons
> > > for this split. Merging the vm_lock back into vm_area_struct also allows
> > > vm_area_struct to use SLAB_TYPESAFE_BY_RCU later in this patchset.
> > > Move vm_lock back into vm_area_struct, aligning it at the cacheline
> > > boundary and changing the cache to be cacheline-aligned as well.
> > > With kernel compiled using defconfig, this causes VMA memory consumption
> > > to grow from 160 (vm_area_struct) + 40 (vm_lock) bytes to 256 bytes:
> > >
> > >     slabinfo before:
> > >      <name>           ... <objsize> <objperslab> <pagesperslab> : ...
> > >      vma_lock         ...     40  102    1 : ...
> > >      vm_area_struct   ...    160   51    2 : ...
> > >
> > >     slabinfo after moving vm_lock:
> > >      <name>           ... <objsize> <objperslab> <pagesperslab> : ...
> > >      vm_area_struct   ...    256   32    2 : ...
> > >
> > > Aggregate VMA memory consumption per 1000 VMAs grows from 50 to 64 pages,
> > > which is 5.5MB per 100000 VMAs. Note that the size of this structure is
> > > dependent on the kernel configuration and typically the original size is
> > > higher than 160 bytes. Therefore these calculations are close to the
> > > worst case scenario. A more realistic vm_area_struct usage before this
> > > change is:
> > >
> > >      <name>           ... <objsize> <objperslab> <pagesperslab> : ...
> > >      vma_lock         ...     40  102    1 : ...
> > >      vm_area_struct   ...    176   46    2 : ...
> > >
> > > Aggregate VMA memory consumption per 1000 VMAs grows from 54 to 64 pages,
> > > which is 3.9MB per 100000 VMAs.
> > > This memory consumption growth can be addressed later by optimizing the
> > > vm_lock.
> > >
> > > [1] https://lore.kernel.org/all/20230227173632.3292573-34-surenb@google.com/
> > > [2] https://lore.kernel.org/all/ZsQyI%2F087V34JoIt@xsang-OptiPlex-9020/
> > > [3] https://lore.kernel.org/all/CAJuCfpEisU8Lfe96AYJDZ+OM4NoPmnw9bP53cT_kbfP_pR+-2g@mail.gmail.com/
> > >
> > > Signed-off-by: Suren Baghdasaryan <surenb@...gle.com>
> > > Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
> >
> > Reviewed-by: Shakeel Butt <shakeel.butt@...ux.dev>
> 
> Thanks!
> 
> >
> >
> > One question below.
> >
> > > --- a/include/linux/mm_types.h
> > > +++ b/include/linux/mm_types.h
> > > @@ -716,8 +716,6 @@ struct vm_area_struct {
> > >        * slowpath.
> > >        */
> > >       unsigned int vm_lock_seq;
> > > -     /* Unstable RCU readers are allowed to read this. */
> > > -     struct vma_lock *vm_lock;
> > >  #endif
> > >
> > >       /*
> > > @@ -770,6 +768,10 @@ struct vm_area_struct {
> > >       struct vma_numab_state *numab_state;    /* NUMA Balancing state */
> > >  #endif
> > >       struct vm_userfaultfd_ctx vm_userfaultfd_ctx;
> > > +#ifdef CONFIG_PER_VMA_LOCK
> > > +     /* Unstable RCU readers are allowed to read this. */
> > > +     struct vma_lock vm_lock ____cacheline_aligned_in_smp;
> > > +#endif
> > >  } __randomize_layout;
> >
> > Do we just want 'struct vm_area_struct' to be cacheline aligned or do we
> > want 'struct vma_lock vm_lock' to be on a separate cacheline as well?
> 
> We want both to minimize cacheline sharing.
> 

For later, you will need to add a pad after vm_lock as well, so any
future addition will not share the cacheline with vm_lock. I would do
something like below. This is a nit and can be done later.

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 7654c766cbe2..5cc4fff163a0 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -751,10 +751,12 @@ struct vm_area_struct {
 #endif
 	struct vm_userfaultfd_ctx vm_userfaultfd_ctx;
 #ifdef CONFIG_PER_VMA_LOCK
+	CACHELINE_PADDING(__pad1__);
 	/* Unstable RCU readers are allowed to read this. */
-	struct vma_lock vm_lock ____cacheline_aligned_in_smp;
+	struct vma_lock vm_lock;
+	CACHELINE_PADDING(__pad2__);
 #endif
-} __randomize_layout;
+} __randomize_layout ____cacheline_aligned_in_smp;
 
 #ifdef CONFIG_NUMA
 #define vma_policy(vma) ((vma)->vm_policy)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ