linux-kernel - Re: [PATCH] squashfs: Migrate from ll_rw

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191024054112.GA1998@hsiangkao-HP-ZHAN-66-Pro-G1>
Date:   Thu, 24 Oct 2019 13:41:15 +0800
From:   Gao Xiang <hsiangkao@....com>
To:     Philippe Liard <pliard@...gle.com>
Cc:     phillip@...ashfs.org.uk, linux-kernel@...r.kernel.org,
        groeck@...omium.org
Subject: Re: [PATCH] squashfs: Migrate from ll_rw_block usage to BIO

On Thu, Oct 24, 2019 at 10:23:54AM +0900, Philippe Liard wrote:
> Thanks Cristoph for taking a look. I like the idea of simplifying this if
> possible. I think I understand your suggestion in principle but I'm not
> seeing a way to apply it here. Would it be possible for you to be a little
> more specific? Let me try to explain this below.
> 
> My admittedly?limited understanding is that using BIO indirectly requires
> buffer_head or an alternative including some synchronization mechanism at
> least.
> It's true that the bio_{alloc,add_page,submit}() functions don't require
> passing a buffer_head. However because bio_submit() is asynchronous AFAICT
> the client needs to use a synchronization mechanism to wait for and notify
> the completion of the request which buffer heads provide. This is achieved
> respectively by wait_on_buffer() and {set,clear}_buffer_uptodate().
> 
> Another dependency on buffer heads is the fact that squashfs_read_data()
> calls into other squashfs functions operating on buffer heads outside this
> file. For example squashfs_decompress() operates on a buffer_head array.
> 
> Given that bio_submit() is asynchronous I'm also not seeing how the
> squashfs_bio_request allocation can be removed? There can be multiple BIO
> requests in flight each needing to carry some context used on completion
> of the request.

Personally speaking, just for Android related use cases, I'd suggest latest
EROFS if you care more about system overall performance more than compression
ratio, even https://lkml.org/lkml/2017/9/22/814 is applied (you can do
benchmark), we did much efforts 3 years ago.

And that is not only performance but noticable memory overhead (a lot of
extra memory allocations) and heavy page cache thrashing in low memory
scenarios (it's very common [1].)

[1] https://linuxplumbersconf.org/event/4/contributions/404/attachments/326/550/Handling_memory_pressure_on_Android.pdf

Thanks,
Gao Xiang