linux-kernel - Re: [PATCH v2 2/2] dax,pmem: Implement pmem based dax data recovery

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <15f01d51-2611-3566-0d08-bdfbec53f88c@oracle.com>
Date:   Tue, 9 Nov 2021 19:58:48 +0000
From:   Jane Chu <jane.chu@...cle.com>
To:     Dan Williams <dan.j.williams@...el.com>,
        Christoph Hellwig <hch@...radead.org>
CC:     david <david@...morbit.com>, "Darrick J. Wong" <djwong@...nel.org>,
        Vishal L Verma <vishal.l.verma@...el.com>,
        Dave Jiang <dave.jiang@...el.com>,
        Alasdair Kergon <agk@...hat.com>,
        Mike Snitzer <snitzer@...hat.com>,
        device-mapper development <dm-devel@...hat.com>,
        "Weiny, Ira" <ira.weiny@...el.com>,
        Matthew Wilcox <willy@...radead.org>,
        Vivek Goyal <vgoyal@...hat.com>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Linux NVDIMM <nvdimm@...ts.linux.dev>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        linux-xfs <linux-xfs@...r.kernel.org>
Subject: Re: [PATCH v2 2/2] dax,pmem: Implement pmem based dax data recovery

On 11/9/2021 10:48 AM, Dan Williams wrote:
> On Mon, Nov 8, 2021 at 11:27 PM Christoph Hellwig <hch@...radead.org> wrote:
>>
>> On Fri, Nov 05, 2021 at 07:16:38PM -0600, Jane Chu wrote:
>>>   static size_t pmem_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff,
>>>                void *addr, size_t bytes, struct iov_iter *i, int mode)
>>>   {
>>> +     phys_addr_t pmem_off;
>>> +     size_t len, lead_off;
>>> +     struct pmem_device *pmem = dax_get_private(dax_dev);
>>> +     struct device *dev = pmem->bb.dev;
>>> +
>>> +     if (unlikely(mode == DAX_OP_RECOVERY)) {
>>> +             lead_off = (unsigned long)addr & ~PAGE_MASK;
>>> +             len = PFN_PHYS(PFN_UP(lead_off + bytes));
>>> +             if (is_bad_pmem(&pmem->bb, PFN_PHYS(pgoff) / 512, len)) {
>>> +                     if (lead_off || !(PAGE_ALIGNED(bytes))) {
>>> +                             dev_warn(dev, "Found poison, but addr(%p) and/or bytes(%#lx) not page aligned\n",
>>> +                                     addr, bytes);
>>> +                             return (size_t) -EIO;
>>> +                     }
>>> +                     pmem_off = PFN_PHYS(pgoff) + pmem->data_offset;
>>> +                     if (pmem_clear_poison(pmem, pmem_off, bytes) !=
>>> +                                             BLK_STS_OK)
>>> +                             return (size_t) -EIO;
>>> +             }
>>> +     }
>>
>> This is in the wrong spot.  As seen in my WIP series individual drivers
>> really should not hook into copying to and from the iter, because it
>> really is just one way to write to a nvdimm.  How would dm-writecache
>> clear the errors with this scheme?
>>
>> So IMHO going back to the separate recovery method as in your previous
>> patch really is the way to go.  If/when the 64-bit store happens we
>> need to figure out a good way to clear the bad block list for that.
> 
> I think we just make error management a first class citizen of a
> dax-device and stop abstracting it behind a driver callback. That way
> the driver that registers the dax-device can optionally register error
> management as well. Then fsdax path can do:
> 
>          rc = dax_direct_access(..., &kaddr, ...);
>          if (unlikely(rc)) {
>                  kaddr = dax_mk_recovery(kaddr);

Sorry, what does dax_mk_recovery(kaddr) do?

>                  dax_direct_access(..., &kaddr, ...);
>                  return dax_recovery_{read,write}(..., kaddr, ...);
>          }
>          return copy_{mc_to_iter,from_iter_flushcache}(...);
> 
> Where, the recovery version of dax_direct_access() has the opportunity
> to change the page permissions / use an alias mapping for the access,

again, sorry, what 'page permissions'?  memory_failure_dev_pagemap()
changes the poisoned page mem_type from 'rw' to 'uc-' (should be NP?),
do you mean to reverse the change?

> dax_recovery_read() allows reading the good cachelines out of a
> poisoned page, and dax_recovery_write() coordinates error list
> management and returning a poison page to full write-back caching
> operation when no more poisoned cacheline are detected in the page.
> 

How about to introduce 3 dax_recover_ APIs:
   dax_recover_direct_access(): similar to dax_direct_access except
      it ignores error list and return the kaddr, and hence is also
      optional, exported by device driver that has the ability to
      detect error;
   dax_recovery_read(): optional, supported by pmem driver only,
      reads as much data as possible up to the poisoned page;
   dax_recovery_write(): optional, supported by pmem driver only,
      first clear-poison, then write.

Should we worry about the dm targets?

Both dax_recovery_read/write() are hooked up to dax_iomap_iter().

Thanks,
-jane