linux-kernel - Re: [PATCH v8 03/11] btrfs: add support for inserting raid stripe extents

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <477fba84-b411-46c3-9851-ef7e30732db3@wdc.com>
Date:   Thu, 14 Sep 2023 15:35:02 +0000
From:   Johannes Thumshirn <Johannes.Thumshirn@....com>
To:     Qu Wenruo <quwenruo.btrfs@....com>, Chris Mason <clm@...com>,
        Josef Bacik <josef@...icpanda.com>,
        David Sterba <dsterba@...e.com>
CC:     Christoph Hellwig <hch@....de>,
        Naohiro Aota <Naohiro.Aota@....com>, Qu Wenruo <wqu@...e.com>,
        Damien Le Moal <dlemoal@...nel.org>,
        "linux-btrfs@...r.kernel.org" <linux-btrfs@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v8 03/11] btrfs: add support for inserting raid stripe
 extents

On 14.09.23 12:07, Qu Wenruo wrote:
>>>> +	ret = btrfs_insert_item(trans, stripe_root, &stripe_key, stripe_extent,
>>>> +				item_size);
>>>
>>> Have you tested in near-real-world on how continous the RST items could
>>> be for RAID0/RAID10?
>>>
>>> My concern here is, we may want to try our best to reduce the size of
>>> RST, due to the 64K BTRFS_STRIPE_LEN.
>>>
>>
>> There are two things I can do for it. First is trying to merge contiguus
>> RST items
> 
> This is much easier, as the RST lookup code is already taking the length
> into consideration, thus only the add path need some work.
> 
> Although I'm not sure how effective it would be in real world.
> As if the merge rate is only 5%, then it barely makes a difference.
> 

I think this will be very much workload dependent. Also just having 
logically contiguous entries doesn't help much. All the physical strides 
have to be contiguous as well.

> Maybe you don't need to implement a full merge in this version, but just
> do some trace events to see the merge rate?
> 
>> and second make BTRFS_STRIPE_LEN a mkfs time constant instead
>> of a compile time constant.
> 
> Please be very careful about this, we have quite some bitmap relying on
> this. (IIRC RAID56 and scrub)
> 
> Currently unsigned long can only support up to 64 bits, thus the maximum
> stripe length would be 256K, but I'm pretty sure there would be other
> hidden traps somewhere else.
> 

Yeah, this will be (both) a longer term research project as well. RAID56 
has priority. Then Erasure Coding.