linux-kernel - Re: [PATCH] lightnvm: pblk: recover chunk state on 1.2 devices

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <D6BDF75C-179D-424C-A8B0-950A1FD80221@cnexlabs.com>
Date:   Fri, 3 Aug 2018 12:53:37 +0000
From:   Javier Gonzalez <javier@...xlabs.com>
To:     Matias Bjørling <mb@...htnvm.io>
CC:     Hans Holmberg <hans.holmberg@...xlabs.com>,
        "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] lightnvm: pblk: recover chunk state on 1.2 devices

> On 3 Aug 2018, at 14.46, Matias Bjørling <mb@...htnvm.io> wrote:
> 
> On 08/03/2018 02:37 PM, Javier Gonzalez wrote:
>>> On 3 Aug 2018, at 14.30, Matias Bjørling <mb@...htnvm.io> wrote:
>>> 
>>> On 08/03/2018 02:02 PM, Javier Gonzalez wrote:
>>>>> On 3 Aug 2018, at 13.57, Matias Bjørling <mb@...htnvm.io> wrote:
>>>>> 
>>>>> On 07/24/2018 09:54 AM, Javier Gonzalez wrote:
>>>>>>> On 29 Jun 2018, at 13.28, Matias Bjørling <mb@...htnvm.io> wrote:
>>>>>>> 
>>>>>>> On 06/29/2018 01:22 PM, Javier Gonzalez wrote:
>>>>>>>>> On 29 Jun 2018, at 13.14, Matias Bjørling <mb@...htnvm.io> wrote:
>>>>>>>>> 
>>>>>>>>> On 06/28/2018 11:12 AM, Javier González wrote:
>>>>>>>>>> The Open-Channel 1.2 spec does not define a mechanism for the host to
>>>>>>>>>> recover the block (chunk) state. As a consequence, a newly format device
>>>>>>>>>> will need to reconstruct the state. Currently, pblk assumes that blocks
>>>>>>>>>> are not erased, which might cause double-erases in case that the device
>>>>>>>>>> does not protect itself against them (which is not specified in the spec
>>>>>>>>>> either).
>>>>>>>>> 
>>>>>>>>> It should not be specified in the spec. It is up to the device to handle
>>>>>>>>> double erases and not do it.
>>>>>>>>> 
>>>>>>>>>> This patch, reconstructs the state based on read errors. If the first
>>>>>>>>>> sector of a block returns and empty page (NVM_RSP_ERR_EMPTYPAGE), then
>>>>>>>>>> the block s marked free, i.e., erased and ready to be used
>>>>>>>>>> (NVM_CHK_ST_FREE). Otherwise, the block is marked as closed
>>>>>>>>>> (NVM_CHK_ST_CLOSED). Note that even if a block is open and not fully
>>>>>>>>>> written, it has to be erased in order to be used again.
>>>>>>>>> 
>>>>>>>>> Should we extend it to do the scan, and update the write pointer as
>>>>>>>>> well? I think this kind of feature already is baked into pblk?
>>>>>>>> This is already in place: we scan until empty page and take it from
>>>>>>>> there. This patch is only for the case in which we start a pblk instance
>>>>>>>> form scratch. On a device already owned by pblk, we would not have the
>>>>>>>> problem we are trying to solve here because we know the state.
>>>>>>> 
>>>>>>> Agree. What I meant was that when we anyway are recovering the state,
>>>>>>> we could just as well update ->wp and set to NVM_CHK_ST_OPEN and so
>>>>>>> forth for the initialization phase.
>>>>>> In 1.2 the use of chunk metadata is purely fictional. We respect the
>>>>>> chunk state machine as we transition lines, but all the write pointers
>>>>>> are ignored. Instead, we use the line bitmap to point to the next
>>>>>> writable entry. This is BTW the same way we it in open lines on 2.0 too.
>>>>> 
>>>>> Now I understand where you are coming from. I had the understanding
>>>>> that we where using the write pointer now that we moved to 2.0,
>>>>> looking through the code, that wasn't the case. :) Which means that
>>>>> pblk doesn't work with a devices that implements 2.0. Yikes... I knew
>>>>> I had forgot a detail when support was added into pblk.
>>>> I think you misunderstood; pblk does support 2.0 devices. What happens
>>>> is that we transform the per chunk WP in 2.0 into the line bitmap to
>>>> simplify the lookup. The point being that we do not need to create a
>>>> fictional chunk for 1.2 devices since we do the translation to the
>>>> bitmap directly. Does this make sense?
>>> 
>>> The chunk->wp isn't used anywhere. So it can't take wp into account.
>>> It uses the EMPTYPAGE marker from 1.2 instead. See pblk-core and
>>> pblk-recovery.
>> I see that the patches for this are still internal. Will post for 4.20
> 
> Thanks. Please also put a Fixes: on, so it gets backported appropriately.

Sure. I will.

Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)