linux-kernel - Re: [6.2][regression] after commit 947a629988f191807d2d22ba63ae18259bb645c5 btrfs volume periodical forced switch to readonly after a lot of disk writes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <18b5fa1e-7d1e-4560-c98b-d7ac5fc87c3a@gmx.com>
Date:   Mon, 26 Dec 2022 11:29:42 +0800
From:   Qu Wenruo <quwenruo.btrfs@....com>
To:     Mikhail Gavrilov <mikhail.v.gavrilov@...il.com>
Cc:     wqu@...e.com, dsterba@...e.com,
        Btrfs BTRFS <linux-btrfs@...r.kernel.org>,
        Linux List Kernel Mailing <linux-kernel@...r.kernel.org>
Subject: Re: [6.2][regression] after commit
 947a629988f191807d2d22ba63ae18259bb645c5 btrfs volume periodical forced
 switch to readonly after a lot of disk writes



On 2022/12/26 10:47, Mikhail Gavrilov wrote:
> On Mon, Dec 26, 2022 at 5:15 AM Qu Wenruo <quwenruo.btrfs@....com> wrote:
>>
>> Thanks a lot for the full kernel log.
>>
>> It indeed shows something is wrong in the run_one_delayed_ref().
>> But surprisingly, if there is something wrong, I'd expect more output
>> from btrfs, as normally if one tree block failed to pass whatever the
>> checks, it should cause an error message at least.
>>
>> Since you can reproduce the bug (although I don't think this is easy to
>> reproduce), mind to apply the extra debug patch and then try to reproduce?
> 
> Of course I am still able to reproduce.
> The number of messages foreshadowing readonly has become somewhat more:
> [ 2295.155437] BTRFS error (device nvme0n1p3): level check failed on
> logical 4957418700800 mirror 1 wanted 0 found 1

OK, indeed a level mismatch.

 From the remaining lines, it shows we're failing at 
do_free_extent_accounting(), which failed at the btrfs_del_csums().

And inside btrfs_del_csums(), what we do are all regular btree 
operations, thus the tree level check should work without problem.

Thus it seems to be a corrupted csum tree.

> [ 2295.155831] BTRFS error (device nvme0n1p3: state A): Transaction
> aborted (error -5)
> [ 2295.155946] BTRFS: error (device nvme0n1p3: state A) in
> do_free_extent_accounting:2849: errno=-5 IO failure
> [ 2295.155978] BTRFS info (device nvme0n1p3: state EA): forced readonly
> [ 2295.155985] BTRFS error (device nvme0n1p3: state EA):
> run_one_delayed_ref returned -5
> [ 2295.156051] BTRFS: error (device nvme0n1p3: state EA) in
> btrfs_run_delayed_refs:2153: errno=-5 IO failure
> 
> Of course full logs are also attached.
> 
>> Another thing is, mind to run "btrfs check --readonly" on the fs?
> Result of check attached too.
> 
Could you please run "btrfs check --readonly" from a liveCD?
There are tons of possible false alerts if ran on a RW mounted fs.

Thanks,
Qu