linux-kernel - Re: [PATCH RESEND V5,0/2]mm: shmem: support POSIX_FADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <25765c7a-03bc-dfd3-4d21-804f966dcda9@quicinc.com>
Date:   Tue, 24 Jan 2023 18:58:09 +0530
From:   Charan Teja Kalla <quic_charante@...cinc.com>
To:     <akpm@...ux-foundation.org>, <willy@...radead.org>,
        <markhemm@...glemail.com>, <hughd@...gle.com>,
        <rientjes@...gle.com>, <surenb@...gle.com>, <shakeelb@...gle.com>,
        Michal Hocko <mhocko@...e.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Pavan Kondeti <quic_pkondeti@...cinc.com>,
        <noralf@...nnes.org>, Dan Carpenter <dan.carpenter@...cle.com>,
        <daniel.vetter@...ll.ch>
CC:     <linux-mm@...ck.org>, <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH RESEND V5,0/2]mm: shmem: support
 POSIX_FADV_[WILL|DONT]NEED for shmem files

Hi Andrew and all,

I am trying to comeback and make another attempt to upstream this patch
by providing even more details:

We have the requirement similar to of GEM objects backed by shmem but to
reclaim those pages instantaneously when the user wants it.  And for
file reclaim, users can use POSIX_FADV_DONTNEED to reclaim the pages(as
per standard, the clean pages are dropped but the dirty pages writeback
is implementation defined).  And client uses the POSIX_FADV_WILLNEED
when he needs those pages back.

This requirement is implemented on the snapdragon chips for graphics
clients where buffers of which are allocated through shmem interface(not
mapped to the userspace) and reclaimed[1][2] through a function
implemented in downstream Android kernel. We know that shmem allocation
support is already implemented in the upstream but the instantaneous
reclaim of those pages(which fadvise() can be used for the file pages)
is missed in the upstream implementation. This patch aims to implement
the support of reclaiming the shmem pages through the shmem fadvise but
__currently we are unable to find any upstream users for this usecase
and may be that's why didn't get any active reviews on this patch__.

So this attempt is to equip the linux kernel with "fadvise support for
shmem page reclaim", which might be useful in the future.

If community sees there will be some real benefits with this patch set,
will port these against linux-next. Please provide your inputs.

[1]
https://git.codelinaro.org/clo/la/platform/vendor/qcom/opensource/graphics-kernel/-/blob/gfx-kernel.lnx.1.0.r3-rel/kgsl_reclaim.c#L289
[2]
https://android.googlesource.com/kernel/common/+/refs/heads/android12-5.10/mm/shmem.c#4310

Thanks,
Charan

On 3/31/2022 12:08 PM, Charan Teja Kalla wrote:
> From: Charan Teja Reddy <quic_charante@...cinc.com>
> 
> This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED
> advices to shmem files which can be helpful for the drivers who may want
> to manage the pages of shmem files on their own, like, that are created
> through shmem_file_setup[_with_mnt]().
> 
> Changes in V5:
>  -- Moved the 'endbyte' calculations to a header function for use by shmem_fadvise().
>  -- Addressed comments from suren.
>  -- No changes in resend. Retested on the latest tip.
> 
> Changes in V4:
>   -- Changed the code to use reclaim_pages() to writeout the shmem pages to swap and then reclaim.
>   -- Addressed comments from Mark Hemment and Matthew.
>   -- fadvise() on shmem file may even unmap a page.
>   -- https://patchwork.kernel.org/project/linux-mm/patch/1644572051-24091-1-git-send-email-quic_charante@quicinc.com/
> 
> Changes in V3:
>   -- Considered THP pages while doing FADVISE_[DONT|WILL]NEED, identified by Matthew.
>   -- xarray used properly, as identified by Matthew.
>   -- Excluded mapped pages as it requires unmapping and the man pages of fadvise don't talk about them.
>   -- RESEND: Fixed the compilation issue when CONFIG_TMPFS is not defined.
>   -- https://patchwork.kernel.org/project/linux-mm/patch/1641488717-13865-1-git-send-email-quic_charante@quicinc.com/
> 
> Changes in V2:
>   -- Rearranged the code to not to sleep with rcu_lock while using xas_() functionality.
>   -- Addressed the comments from Suren.
>   -- https://patchwork.kernel.org/project/linux-mm/patch/1638442253-1591-1-git-send-email-quic_charante@quicinc.com/
> 
> changes in V1:
>   -- Created the interface for fadvise(2) to work on shmem files.
>   -- https://patchwork.kernel.org/project/linux-mm/patch/1633701982-22302-1-git-send-email-charante@codeaurora.org/
> 
> Charan Teja Reddy (2):
>   mm: fadvise: move 'endbyte' calculations to helper function
>   mm: shmem: implement POSIX_FADV_[WILL|DONT]NEED for shmem
> 
>  mm/fadvise.c  |  11 +-----
>  mm/internal.h |  21 ++++++++++
>  mm/shmem.c    | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 145 insertions(+), 10 deletions(-)
>