| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 29 Feb 2024 23:05:19 +0100
From: Goffredo Baroncelli <kreijack@...ind.it>
To: Patrick Plenefisch <simonpatp@...il.com>
Cc: stable@...r.kernel.org, linux-kernel@...r.kernel.org,
Alasdair Kergon <agk@...hat.com>, Mike Snitzer <snitzer@...nel.org>,
Mikulas Patocka <mpatocka@...hat.com>, Chris Mason <clm@...com>,
Josef Bacik <josef@...icpanda.com>, David Sterba <dsterba@...e.com>,
regressions@...ts.linux.dev, dm-devel@...ts.linux.dev,
linux-btrfs@...r.kernel.org
Subject: Re: [REGRESSION] LVM-on-LVM: error while submitting device barriers
On 29/02/2024 21.22, Patrick Plenefisch wrote:
> On Thu, Feb 29, 2024 at 2:56 PM Goffredo Baroncelli <kreijack@...ind.it> wrote:
>>
>>> Your understanding is correct. The only thing that comes to my mind to
>>> cause the problem is asymmetry of the SATA devices. I have one 8TB
>>> device, plus a 1.5TB, 3TB, and 3TB drives. Doing math on the actual
>>> extents, lowerVG/single spans (3TB+3TB), and
>>> lowerVG/lvmPool/lvm/brokenDisk spans (3TB+1.5TB). Both obviously have
>>> the other leg of raid1 on the 8TB drive, but my thought was that the
>>> jump across the 1.5+3TB drive gap was at least "interesting"
>>
>>
>> what about lowerVG/works ?
>>
>
> That one is only on two disks, it doesn't span any gaps
Sorry, but re-reading the original email I found something that I missed before:
> BTRFS error (device dm-75): bdev /dev/mapper/lvm-brokenDisk errs: wr
> 0, rd 0, flush 1, corrupt 0, gen 0
> BTRFS warning (device dm-75): chunk 13631488 missing 1 devices, max
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> tolerance is 0 for writable mount
> BTRFS: error (device dm-75) in write_all_supers:4379: errno=-5 IO
> failure (errors while submitting device barriers.)
Looking at the code, it seems that if a FLUSH commands fails, btrfs
considers that the disk is missing. The it cannot mount RW the device.
I would investigate with the LVM developers, if it properly passes
the flush/barrier command through all the layers, when we have an
lvm over lvm (raid1). The fact that the lvm is a raid1, is important because
a flush command to be honored has to be honored by all the
devices involved.
>
>> However yes, I agree that the pair of disks involved may be the answer
>> of the problem.
>>
>> Could you show us the output of
>>
>> $ sudo pvdisplay -m
>>
>>
>
> I trimmed it, but kept the relevant bits (Free PE is thus not correct):
>
>
> --- Physical volume ---
> PV Name /dev/lowerVG/lvmPool
> VG Name lvm
> PV Size <3.00 TiB / not usable 3.00 MiB
> Allocatable yes
> PE Size 4.00 MiB
> Total PE 786431
> Free PE 82943
> Allocated PE 703488
> PV UUID 7p3LSU-EAHd-xUg0-r9vT-Gzkf-tYFV-mvlU1M
>
> --- Physical Segments ---
> Physical extent 0 to 159999:
> Logical volume /dev/lvm/brokenDisk
> Logical extents 0 to 159999
> Physical extent 160000 to 339199:
> Logical volume /dev/lvm/a
> Logical extents 0 to 179199
> Physical extent 339200 to 349439:
> Logical volume /dev/lvm/brokenDisk
> Logical extents 160000 to 170239
> Physical extent 349440 to 351999:
> FREE
> Physical extent 352000 to 460026:
> Logical volume /dev/lvm/brokenDisk
> Logical extents 416261 to 524287
> Physical extent 460027 to 540409:
> FREE
> Physical extent 540410 to 786430:
> Logical volume /dev/lvm/brokenDisk
> Logical extents 170240 to 416260
>
>
> --- Physical volume ---
> PV Name /dev/sda3
> VG Name lowerVG
> PV Size <2.70 TiB / not usable 3.00 MiB
> Allocatable yes
> PE Size 4.00 MiB
> Total PE 707154
> Free PE 909
> Allocated PE 706245
> PV UUID W8gJ0P-JuMs-1y3g-b5cO-4RuA-MoFs-3zgKBn
>
> --- Physical Segments ---
> Physical extent 0 to 52223:
> Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
> Logical extents 629330 to 681553
> Physical extent 52224 to 628940:
> Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
> Logical extents 0 to 576716
> Physical extent 628941 to 628941:
> Logical volume /dev/lowerVG/single_corig_rmeta_0
> Logical extents 0 to 0
> Physical extent 628942 to 628962:
> Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
> Logical extents 681554 to 681574
> Physical extent 628963 to 634431:
> Logical volume /dev/lowerVG/single_corig_rimage_0_imeta
> Logical extents 0 to 5468
> Physical extent 634432 to 654540:
> FREE
> Physical extent 654541 to 707153:
> Logical volume /dev/lowerVG/single_corig_rimage_0_iorig
> Logical extents 576717 to 629329
>
> --- Physical volume ---
> PV Name /dev/sdf2
> VG Name lowerVG
> PV Size <7.28 TiB / not usable 4.00 MiB
> Allocatable yes
> PE Size 4.00 MiB
> Total PE 1907645
> Free PE 414967
> Allocated PE 1492678
> PV UUID my0zQM-832Z-HYPD-sNfW-68ms-nddg-lMyWJM
>
> --- Physical Segments ---
> Physical extent 0 to 0:
> Logical volume /dev/lowerVG/single_corig_rmeta_1
> Logical extents 0 to 0
> Physical extent 1 to 681575:
> Logical volume /dev/lowerVG/single_corig_rimage_1_iorig
> Logical extents 0 to 681574
> Physical extent 681576 to 687044:
> Logical volume /dev/lowerVG/single_corig_rimage_1_imeta
> Logical extents 0 to 5468
> Physical extent 687045 to 687045:
> Logical volume /dev/lowerVG/lvmPool_rmeta_0
> Logical extents 0 to 0
> Physical extent 687046 to 1049242:
> Logical volume /dev/lowerVG/lvmPool_rimage_0
> Logical extents 0 to 362196
> Physical extent 1049243 to 1056551:
> FREE
> Physical extent 1056552 to 1473477:
> Logical volume /dev/lowerVG/lvmPool_rimage_0
> Logical extents 369506 to 786431
> Physical extent 1473478 to 1480786:
> Logical volume /dev/lowerVG/lvmPool_rimage_0
> Logical extents 362197 to 369505
> Physical extent 1480787 to 1907644:
> FREE
>
> --- Physical volume ---
> PV Name /dev/sdb3
> VG Name lowerVG
> PV Size 1.33 TiB / not usable 3.00 MiB
> Allocatable yes (but full)
> PE Size 4.00 MiB
> Total PE 349398
> Free PE 0
> Allocated PE 349398
> PV UUID Ncmgdw-ZOXS-qTYL-1jAz-w7zt-38V2-f53EpI
>
> --- Physical Segments ---
> Physical extent 0 to 0:
> Logical volume /dev/lowerVG/lvmPool_rmeta_1
> Logical extents 0 to 0
> Physical extent 1 to 349397:
> Logical volume /dev/lowerVG/lvmPool_rimage_1
> Logical extents 0 to 349396
>
>
> --- Physical volume ---
> PV Name /dev/sde2
> VG Name lowerVG
> PV Size 2.71 TiB / not usable 3.00 MiB
> Allocatable yes
> PE Size 4.00 MiB
> Total PE 711346
> Free PE 255111
> Allocated PE 456235
> PV UUID xUG8TG-wvp0-roBo-GPo7-sbvn-aE7I-NAHU07
>
> --- Physical Segments ---
> Physical extent 0 to 416925:
> Logical volume /dev/lowerVG/lvmPool_rimage_1
> Logical extents 369506 to 786431
> Physical extent 416926 to 437034:
> Logical volume /dev/lowerVG/lvmPool_rimage_1
> Logical extents 349397 to 369505
> Physical extent 437035 to 711345:
> FREE
>
>
> Finally, I am not sure if it's relevant, but I did struggle to expand
> the raid1 volumes across gaps when creating this setup. I did file a
> bug about that, though I am not sure if it's relevant, as I removed
> integrity and cache for brokenDisk & lvmPool:
> https://gitlab.com/lvmteam/lvm2/-/issues/6
>
> Patrick
>
--
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
Powered by blists - more mailing lists