[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20200922120820.24978-1-sjpark@amazon.com>
Date: Tue, 22 Sep 2020 14:08:20 +0200
From: SeongJae Park <sjpark@...zon.com>
To: Jürgen Groß <jgross@...e.com>
CC: SeongJae Park <sjpark@...zon.com>,
Roger Pau Monné <roger.pau@...rix.com>,
<konrad.wilk@...cle.com>, SeongJae Park <sjpark@...zon.de>,
<axboe@...nel.dk>, <aliguori@...zon.com>, <amit@...nel.org>,
<mheyne@...zon.de>, <pdurrant@...zon.co.uk>,
<linux-block@...r.kernel.org>, <xen-devel@...ts.xenproject.org>,
<linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2 1/3] xen-blkback: add a parameter for disabling of persistent grants
On Tue, 22 Sep 2020 13:35:30 +0200 "Jürgen Groß" <jgross@...e.com> wrote:
> On 22.09.20 13:26, SeongJae Park wrote:
> > On Tue, 22 Sep 2020 13:12:59 +0200 "Roger Pau Monné" <roger.pau@...rix.com> wrote:
> >
> >> On Tue, Sep 22, 2020 at 12:52:07PM +0200, SeongJae Park wrote:
> >>> From: SeongJae Park <sjpark@...zon.de>
> >>>
> >>> Persistent grants feature provides high scalability. On some small
> >>> systems, however, it could incur data copy overheads[1] and thus it is
> >>> required to be disabled. But, there is no option to disable it. For
> >>> the reason, this commit adds a module parameter for disabling of the
> >>> feature.
> >>>
> >>> [1] https://wiki.xen.org/wiki/Xen_4.3_Block_Protocol_Scalability
> >>>
> >>> Signed-off-by: Anthony Liguori <aliguori@...zon.com>
> >>> Signed-off-by: SeongJae Park <sjpark@...zon.de>
> >>> ---
> >>> .../ABI/testing/sysfs-driver-xen-blkback | 9 ++++++
> >>> drivers/block/xen-blkback/xenbus.c | 28 ++++++++++++++-----
> >>> 2 files changed, 30 insertions(+), 7 deletions(-)
> >>>
> >>> diff --git a/Documentation/ABI/testing/sysfs-driver-xen-blkback b/Documentation/ABI/testing/sysfs-driver-xen-blkback
> >>> index ecb7942ff146..ac2947b98950 100644
> >>> --- a/Documentation/ABI/testing/sysfs-driver-xen-blkback
> >>> +++ b/Documentation/ABI/testing/sysfs-driver-xen-blkback
> >>> @@ -35,3 +35,12 @@ Description:
> >>> controls the duration in milliseconds that blkback will not
> >>> cache any page not backed by a grant mapping.
> >>> The default is 10ms.
> >>> +
> >>> +What: /sys/module/xen_blkback/parameters/feature_persistent
> >>> +Date: September 2020
> >>> +KernelVersion: 5.10
> >>> +Contact: SeongJae Park <sjpark@...zon.de>
> >>> +Description:
> >>> + Whether to enable the persistent grants feature or not. Note
> >>> + that this option only takes effect on newly created backends.
> >>> + The default is Y (enable).
> >>> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
> >>> index b9aa5d1ac10b..8a95ddd08b13 100644
> >>> --- a/drivers/block/xen-blkback/xenbus.c
> >>> +++ b/drivers/block/xen-blkback/xenbus.c
> >>> @@ -879,6 +879,12 @@ static void reclaim_memory(struct xenbus_device *dev)
> >>>
> >>> /* ** Connection ** */
> >>>
> >>> +/* Enable the persistent grants feature. */
> >>> +static bool feature_persistent = true;
> >>> +module_param(feature_persistent, bool, 0644);
> >>> +MODULE_PARM_DESC(feature_persistent,
> >>> + "Enables the persistent grants feature");
> >>> +
> >>> /*
> >>> * Write the physical details regarding the block device to the store, and
> >>> * switch to Connected state.
> >>> @@ -906,11 +912,15 @@ static void connect(struct backend_info *be)
> >>>
> >>> xen_blkbk_barrier(xbt, be, be->blkif->vbd.flush_support);
> >>>
> >>> - err = xenbus_printf(xbt, dev->nodename, "feature-persistent", "%u", 1);
> >>> - if (err) {
> >>> - xenbus_dev_fatal(dev, err, "writing %s/feature-persistent",
> >>> - dev->nodename);
> >>> - goto abort;
> >>> + if (feature_persistent) {
> >>> + err = xenbus_printf(xbt, dev->nodename, "feature-persistent",
> >>> + "%u", feature_persistent);
> >>> + if (err) {
> >>> + xenbus_dev_fatal(dev, err,
> >>> + "writing %s/feature-persistent",
> >>> + dev->nodename);
> >>> + goto abort;
> >>> + }
> >>> }
> >>>
> >>> err = xenbus_printf(xbt, dev->nodename, "sectors", "%llu",
> >>> @@ -1093,8 +1103,12 @@ static int connect_ring(struct backend_info *be)
> >>> xenbus_dev_fatal(dev, err, "unknown fe protocol %s", protocol);
> >>> return -ENOSYS;
> >>> }
> >>> - pers_grants = xenbus_read_unsigned(dev->otherend, "feature-persistent",
> >>> - 0);
> >>> + if (feature_persistent)
> >>> + pers_grants = xenbus_read_unsigned(dev->otherend,
> >>> + "feature-persistent", 0);
> >>> + else
> >>> + pers_grants = 0;
> >>> +
> >>
> >> Sorry for not realizing earlier, but looking at it again I think you
> >> need to cache the value of feature_persistent when it's first used in
> >> the blkback state data, so that it's consistent.
> >>
> >> What would happen for example with the following flow (assume a
> >> persistent grants enabled frontend):
> >>
> >> feature_persistent = false
> >>
> >> connect(...)
> >> feature-persistent is not written to xenstore
> >>
> >> User changes feature_persistent = true
> >>
> >> connect_ring(...)
> >> pers_grants = true, because feature-persistent is set unconditionally
> >> by the frontend and feature_persistent variable is now true.
> >>
> >> Then blkback will try to use persistent grants and the whole
> >> connection will malfunction because the frontend won't.
> >
> > Ah, you're right. I should also catch this before but didn't, sorry.
> >
> >>
> >> The other option is to prevent changing the variable when there are
> >> blkback instances already running.
> >
> > I think storing the option value in xenstore would be simpler. That said, if
> > you prefer this way, please let me know.
>
> No, Xenstore isn't the right place for that. This is a local
> implementation detail of blkback and shouldn't be exported to Xenstore.
Agreed. I will try using the 'feature_gnt_persistent' as Roger recommended.
Thanks,
SeongJae Park
Powered by blists - more mailing lists