Message-ID: <PU1P153MB01691648EF8354319EA0244CBF6F0@PU1P153MB0169.APCP153.PROD.OUTLOOK.COM>
Date:   Wed, 6 Feb 2019 00:57:48 +0000
From:   Dexuan Cui <decui@...rosoft.com>
To:     Dan Williams <dan.j.williams@...el.com>
CC:     "linux-nvdimm@...ts.01.org" <linux-nvdimm@...ts.01.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Michael Kelley <mikelley@...rosoft.com>
Subject: RE: [PATCH] libnvdimm/dimm: Add a no-BLK quirk based on NVDIMM family

> From: Dan Williams <dan.j.williams@...el.com>
> Sent: Tuesday, February 5, 2019 9:12 AM
> On Tue, Feb 5, 2019 at 8:53 AM Dexuan Cui <decui@...rosoft.com> wrote:
> >
> > > From: Dan Williams <dan.j.williams@...el.com>
> > > Sent: Sunday, February 3, 2019 11:14 AM
> > > > ...
> > > > As I understand, the essence of the issue is: Hyper-V emulates the
> > > > label mechanism (i.e. it supports _LSI and LSR), but doesn't do it
> > > > right (i.e. it doesn't support _LSW).
> > > >
> > > > To manage the namespaces, Linux can choose to use label or not.
> > > >
> > > > If _LSI/_LSR are supported, Linux assumes _LSW is supported as well
> > > > and chooses to use label (i.e. the label mode), but since Hyper-V doesn't
> > > > support _LSW, Linux fails to change the namespace configuration.
> > >
> > > No, that's not quite right. The reason Linux does not see the fsdax
> > > mode configuration is due to the missing "address abstraction GUID" in
> > > the label produced the default Hyper-V configuration.
> > Hi Dan,
> > Do you mean NVDIMM_DAX_GUID?
> 
> Correct.
FYI: in create_namespace_pmem(), label0->abstraction_guid is guid_null in
the case of Hyper-V NVDIMM. It looks Hyper-V does't use the guid.

> > > In label-less mode there is no "address abstraction GUID" to validate so
> > > it falls back to just using the info-block directly.
> > In the case of not using label storage area, as I understand the info-block
> > resides in regular data storage area. Can you please tell me where exactly
> > the info-block is and how its location is decided?
> > And I suppose the info-block contains the NVDIMM_DAX_GUID?
> 
> The info-block always lives in the data-storage area, even with
> labels. It's placed at namespace base address + 4K.
>
> When labels are present the label gives the namespace a uuid and the
> info-block "parent uuid" field must match that value. Without labels
> the "parent uuid" field is unused / filled with zero's. Also with v1.2
> labels the address abstraction uuid must match the info-block type.
Thanks for elaborating on this!

It looks the label mechanism is for advanced usages, e.g., a PMEM namespace
can consist of multiple NVDIMMs, or a namespace only uses part of the
capacity of a NVDIMM, or a NVDIMM contains both PMEM and Block-Mode
namespaces.

For a simple usage, e.g. if a NVDIMM only contains one PMEM namespace which
consumes all the storage space of the NVDIMM, it looks the namespace can
be normally used with the help of the info-block, and we don't really need
the label. IMO this is the case of Hyper-V, i.e. we don't use the label at all,
since _LSW is not implemented.

> > I'm asking because I found I lose ~500MBytes of the 32GBytes Hyper-V
> > NVDIMM device, when the namespace is in fsdax mode. When it's in
> > raw mode, I'm able to use all of the 32GB space.
> 
> Yes. A portion of the capacity is reserved for page structures.
Got it. It looks the size of the info-block (and the related padding?) is 2MB,
and "the overhead is 64-bytes per 4K (16GB per 1TB) on x86", so the total
overhead in my 32GB case is: 2MB + 32GB/(4096/64) = 514MB.

Thanks for sharing the link! Now I realized we can use the -M parameter
to not store the page metadata in the NVDIMM: 

-M; --map=
A pmem namespace in “fsdax” or “devdax” mode requires allocation of
per-page metadata. The allocation can be drawn from either:
“mem”: typical system memory
“dev”: persistent memory reserved from the namespace

Thanks,
--Dexuan

Powered by blists - more mailing lists