Message-id: <171330870816.17212.18412845227056589318@noble.neil.brown.name>
Date: Wed, 17 Apr 2024 09:05:08 +1000
From: "NeilBrown" <neilb@...e.de>
To: "Jeff Layton" <jlayton@...nel.org>
Cc: "Lorenzo Bianconi" <lorenzo@...nel.org>, linux-nfs@...r.kernel.org,
 lorenzo.bianconi@...hat.com, chuck.lever@...cle.com, netdev@...r.kernel.org,
 kuba@...nel.org
Subject: Re: [PATCH v8 3/6] NFSD: add write_version to netlink command

On Wed, 17 Apr 2024, Jeff Layton wrote:
> On Wed, 2024-04-17 at 07:48 +1000, NeilBrown wrote:
> > On Tue, 16 Apr 2024, Jeff Layton wrote:
> > > On Tue, 2024-04-16 at 13:16 +1000, NeilBrown wrote:
> > > > On Tue, 16 Apr 2024, Lorenzo Bianconi wrote:
> > > > > Introduce write_version netlink command through a "declarative" interface.
> > > > > This patch introduces a change in behavior since for version-set userspace
> > > > > is expected to provide a NFS major/minor version list it wants to enable
> > > > > while all the other ones will be disabled. (procfs write_version
> > > > > command implements imperative interface where the admin writes +3/-3 to
> > > > > enable/disable a single version.
> > > > 
> > > > It seems a little weird to me that the interface always disables all
> > > > version, but then also allows individual versions to be disabled.
> > > > 
> > > > Would it be reasonable to simply ignore the "enabled" flag when setting
> > > > version, and just enable all versions listed??
> > > > 
> > > > Or maybe only enable those with the flag, and don't disable those
> > > > without the flag?
> > > > 
> > > > Those don't necessarily seem much better - but the current behaviour
> > > > still seems odd.
> > > > 
> > > 
> > > I think it makes sense.
> > > 
> > > We disable all versions, and enable any that have the "enabled" flag set
> > > in the call from userland. Userland technically needn't send down the
> > > versions that are disabled in the call, but the current userland program
> > > does.
> > > 
> > > I worry about imperative interfaces that might only say -- "enable v4.1,
> > > but disable v3" and leave the others in their current state. That
> > > requires that both the kernel and userland keep state about what
> > > versions are currently enabled and disabled, and it's possible to get
> > > that wrong.
> > 
> > I understand and support your aversion for imperative interfaces.
> > But this interface, as currently implemented, looks somewhat imperative.
> > The message sent to the kernel could enable some interfaces and then
> > disable them.  I know that isn't the intent, but it is what the code
> > supports.  Hence "weird".
> > 
> > We could add code to make that sort of thing impossible, but there isn't
> > much point.  Better to make it syntactically impossible.
> > 
> > Realistically there will never be NFSv4.3 as there is no need - new
> > features can be added incrementally. 
> > 
> 
> There is no need _so_far_. Who knows what the future will bring? Maybe
> we'll need v4.3 in order to add some needed feature?
> 
> >  So we could just pass an array of
> > 5 active flags: 2,3,4.0,4.1,4.2.  I suspect you wouldn't like that and
> > I'm not sure that I do either.  A "read" would return the same array
> > with 3 possible states: unavailable, disabled, enabled.  (Maybe the
> > array could be variable length so 5.0 could be added one day...).
> > 
> 
> A set of flags is basically what this interface is. They just happen to
> also be labeled with the major and minorversion. I think that's a good
> thing.

Good for whom?  Labelling allows for labelling inconsistencies.

Maybe the kernel should be able to provide an ordered list of labels,
and separately an array of which labels are enabled/disabled.
Userspace could send down a new set of enabled/disabled flags based on
the agreed set of labels.

Here is a question that is a bit of a diversion, but might help us
understand the context more fully:

  Why would anyone disable v4.2 separately from v4.1 ??

I understand that v2, v3, v4.0, v4.1 are effectively different protocols
and you might want to ensure that only the approved/tested protocol is
used.  But v4.2 is just a few enhancements on v4.1.  Why would you want
to disable it?

The answer I can think of that there might be bugs (perish the
thought!!) in some of those features so you might want to avoid using
them.
But in that case, it is really the features that you want to suppress,
not the protocol version.

Maybe I might want to disable delegation - to just write delegation.
Can I do that?  What if I just want to disable server-side copy, but
keep fallocate and umask support?

i.e.  is a list of versions really the granularity that we want to use
for this interface?

NeilBrown

> 
> >
> > I haven't managed to come up with anything *better* than the current
> > proposal and I don't want to stand in its way, but I wanted to highlight
> > that it looks weird - as much imperative as declarative - in case
> > someone else might be able to come up with a better alternative.
> > 
> 
> The intention was to create a symmetrical interface. We have 2
> operations: a "get" that asks "what versions are currently supported and
> are they enabled?", and a "set" that says "here is the new state of
> version enablement".
> 
> The userland tool always sends down a complete set of versions. The
> kernel is (currently) more forgiving, and treats omitted versions as
> being disabled. The kernel could require that every supported version be
> represented in the "set" operation, if that's more desirable.
> 
> > > 
> > > My thinking was that by using a declarative interface like this, we
> > > eliminate ambiguity in how these interfaces are supposed to work. The
> > > client sends down an entire version map in one fell swoop, and we know
> > > exactly what the result should look like.
> > > 
> > > Note that you can enable or disable just a single version with the
> > > userland tool, but this way all of that complexity stays in userland.
> > > 
> > > > Also for getting the version, the doc says:
> > > > 
> > > >      doc: get nfs enabled versions
> > > > 
> > > > which I don't think it quite right.  The code reports all supported
> > > > versions, and indicates which of those are enabled.  So maybe:
> > > > 
> > > >      doc: get enabled status for all supported versions
> > > > 
> > > > 
> > > > I think that fact that it actually lists all supported versions is
> > > > useful and worth making explicit.
> > > > 
> > > > 
> > > 
> > > Agreed. We should make that change before merging anything.
> > > 
> > > > > 
> > > > > Reviewed-by: Jeff Layton <jlayton@...nel.org>
> > > > > Tested-by: Jeff Layton <jlayton@...nel.org>
> > > > > Signed-off-by: Lorenzo Bianconi <lorenzo@...nel.org>
> > > > > ---
> > > > >  Documentation/netlink/specs/nfsd.yaml |  37 +++++++
> > > > >  fs/nfsd/netlink.c                     |  24 +++++
> > > > >  fs/nfsd/netlink.h                     |   5 +
> > > > >  fs/nfsd/netns.h                       |   1 +
> > > > >  fs/nfsd/nfsctl.c                      | 150 ++++++++++++++++++++++++++
> > > > >  fs/nfsd/nfssvc.c                      |   3 +-
> > > > >  include/uapi/linux/nfsd_netlink.h     |  18 ++++
> > > > >  7 files changed, 236 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/Documentation/netlink/specs/nfsd.yaml b/Documentation/netlink/specs/nfsd.yaml
> > > > > index cbe6c5fd6c4d..0396e8b3ea1f 100644
> > > > > --- a/Documentation/netlink/specs/nfsd.yaml
> > > > > +++ b/Documentation/netlink/specs/nfsd.yaml
> > > > > @@ -74,6 +74,26 @@ attribute-sets:
> > > > >        -
> > > > >          name: leasetime
> > > > >          type: u32
> > > > > +  -
> > > > > +    name: version
> > > > > +    attributes:
> > > > > +      -
> > > > > +        name: major
> > > > > +        type: u32
> > > > > +      -
> > > > > +        name: minor
> > > > > +        type: u32
> > > > > +      -
> > > > > +        name: enabled
> > > > > +        type: flag
> > > > > +  -
> > > > > +    name: server-proto
> > > > > +    attributes:
> > > > > +      -
> > > > > +        name: version
> > > > > +        type: nest
> > > > > +        nested-attributes: version
> > > > > +        multi-attr: true
> > > > >  
> > > > >  operations:
> > > > >    list:
> > > > > @@ -120,3 +140,20 @@ operations:
> > > > >              - threads
> > > > >              - gracetime
> > > > >              - leasetime
> > > > > +    -
> > > > > +      name: version-set
> > > > > +      doc: set nfs enabled versions
> > > > > +      attribute-set: server-proto
> > > > > +      flags: [ admin-perm ]
> > > > > +      do:
> > > > > +        request:
> > > > > +          attributes:
> > > > > +            - version
> > > > > +    -
> > > > > +      name: version-get
> > > > > +      doc: get nfs enabled versions
> > > > > +      attribute-set: server-proto
> > > > > +      do:
> > > > > +        reply:
> > > > > +          attributes:
> > > > > +            - version
> > > > > diff --git a/fs/nfsd/netlink.c b/fs/nfsd/netlink.c
> > > > > index 20a646af0324..bf5df9597288 100644
> > > > > --- a/fs/nfsd/netlink.c
> > > > > +++ b/fs/nfsd/netlink.c
> > > > > @@ -10,6 +10,13 @@
> > > > >  
> > > > >  #include <uapi/linux/nfsd_netlink.h>
> > > > >  
> > > > > +/* Common nested types */
> > > > > +const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1] = {
> > > > > +	[NFSD_A_VERSION_MAJOR] = { .type = NLA_U32, },
> > > > > +	[NFSD_A_VERSION_MINOR] = { .type = NLA_U32, },
> > > > > +	[NFSD_A_VERSION_ENABLED] = { .type = NLA_FLAG, },
> > > > > +};
> > > > > +
> > > > >  /* NFSD_CMD_THREADS_SET - do */
> > > > >  static const struct nla_policy nfsd_threads_set_nl_policy[NFSD_A_SERVER_WORKER_LEASETIME + 1] = {
> > > > >  	[NFSD_A_SERVER_WORKER_THREADS] = { .type = NLA_U32, },
> > > > > @@ -17,6 +24,11 @@ static const struct nla_policy nfsd_threads_set_nl_policy[NFSD_A_SERVER_WORKER_L
> > > > >  	[NFSD_A_SERVER_WORKER_LEASETIME] = { .type = NLA_U32, },
> > > > >  };
> > > > >  
> > > > > +/* NFSD_CMD_VERSION_SET - do */
> > > > > +static const struct nla_policy nfsd_version_set_nl_policy[NFSD_A_SERVER_PROTO_VERSION + 1] = {
> > > > > +	[NFSD_A_SERVER_PROTO_VERSION] = NLA_POLICY_NESTED(nfsd_version_nl_policy),
> > > > > +};
> > > > > +
> > > > >  /* Ops table for nfsd */
> > > > >  static const struct genl_split_ops nfsd_nl_ops[] = {
> > > > >  	{
> > > > > @@ -38,6 +50,18 @@ static const struct genl_split_ops nfsd_nl_ops[] = {
> > > > >  		.doit	= nfsd_nl_threads_get_doit,
> > > > >  		.flags	= GENL_CMD_CAP_DO,
> > > > >  	},
> > > > > +	{
> > > > > +		.cmd		= NFSD_CMD_VERSION_SET,
> > > > > +		.doit		= nfsd_nl_version_set_doit,
> > > > > +		.policy		= nfsd_version_set_nl_policy,
> > > > > +		.maxattr	= NFSD_A_SERVER_PROTO_VERSION,
> > > > > +		.flags		= GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
> > > > > +	},
> > > > > +	{
> > > > > +		.cmd	= NFSD_CMD_VERSION_GET,
> > > > > +		.doit	= nfsd_nl_version_get_doit,
> > > > > +		.flags	= GENL_CMD_CAP_DO,
> > > > > +	},
> > > > >  };
> > > > >  
> > > > >  struct genl_family nfsd_nl_family __ro_after_init = {
> > > > > diff --git a/fs/nfsd/netlink.h b/fs/nfsd/netlink.h
> > > > > index 4137fac477e4..c7c0da275481 100644
> > > > > --- a/fs/nfsd/netlink.h
> > > > > +++ b/fs/nfsd/netlink.h
> > > > > @@ -11,6 +11,9 @@
> > > > >  
> > > > >  #include <uapi/linux/nfsd_netlink.h>
> > > > >  
> > > > > +/* Common nested types */
> > > > > +extern const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1];
> > > > > +
> > > > >  int nfsd_nl_rpc_status_get_start(struct netlink_callback *cb);
> > > > >  int nfsd_nl_rpc_status_get_done(struct netlink_callback *cb);
> > > > >  
> > > > > @@ -18,6 +21,8 @@ int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
> > > > >  				  struct netlink_callback *cb);
> > > > >  int nfsd_nl_threads_set_doit(struct sk_buff *skb, struct genl_info *info);
> > > > >  int nfsd_nl_threads_get_doit(struct sk_buff *skb, struct genl_info *info);
> > > > > +int nfsd_nl_version_set_doit(struct sk_buff *skb, struct genl_info *info);
> > > > > +int nfsd_nl_version_get_doit(struct sk_buff *skb, struct genl_info *info);
> > > > >  
> > > > >  extern struct genl_family nfsd_nl_family;
> > > > >  
> > > > > diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
> > > > > index d4be519b5734..14ec15656320 100644
> > > > > --- a/fs/nfsd/netns.h
> > > > > +++ b/fs/nfsd/netns.h
> > > > > @@ -218,6 +218,7 @@ struct nfsd_net {
> > > > >  /* Simple check to find out if a given net was properly initialized */
> > > > >  #define nfsd_netns_ready(nn) ((nn)->sessionid_hashtbl)
> > > > >  
> > > > > +extern bool nfsd_support_version(int vers);
> > > > >  extern void nfsd_netns_free_versions(struct nfsd_net *nn);
> > > > >  
> > > > >  extern unsigned int nfsd_net_id;
> > > > > diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> > > > > index 38a5df03981b..2c8929ef79e9 100644
> > > > > --- a/fs/nfsd/nfsctl.c
> > > > > +++ b/fs/nfsd/nfsctl.c
> > > > > @@ -1757,6 +1757,156 @@ int nfsd_nl_threads_get_doit(struct sk_buff *skb, struct genl_info *info)
> > > > >  	return err;
> > > > >  }
> > > > >  
> > > > > +/**
> > > > > + * nfsd_nl_version_set_doit - set the nfs enabled versions
> > > > > + * @skb: reply buffer
> > > > > + * @info: netlink metadata and command arguments
> > > > > + *
> > > > > + * Return 0 on success or a negative errno.
> > > > > + */
> > > > > +int nfsd_nl_version_set_doit(struct sk_buff *skb, struct genl_info *info)
> > > > > +{
> > > > > +	const struct nlattr *attr;
> > > > > +	struct nfsd_net *nn;
> > > > > +	int i, rem;
> > > > > +
> > > > > +	if (GENL_REQ_ATTR_CHECK(info, NFSD_A_SERVER_PROTO_VERSION))
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	mutex_lock(&nfsd_mutex);
> > > > > +
> > > > > +	nn = net_generic(genl_info_net(info), nfsd_net_id);
> > > > > +	if (nn->nfsd_serv) {
> > > > > +		mutex_unlock(&nfsd_mutex);
> > > > > +		return -EBUSY;
> > > > > +	}
> > > > > +
> > > > > +	/* clear current supported versions. */
> > > > > +	nfsd_vers(nn, 2, NFSD_CLEAR);
> > > > > +	nfsd_vers(nn, 3, NFSD_CLEAR);
> > > > > +	for (i = 0; i <= NFSD_SUPPORTED_MINOR_VERSION; i++)
> > > > > +		nfsd_minorversion(nn, i, NFSD_CLEAR);
> > > > > +
> > > > > +	nlmsg_for_each_attr(attr, info->nlhdr, GENL_HDRLEN, rem) {
> > > > > +		struct nlattr *tb[NFSD_A_VERSION_MAX + 1];
> > > > > +		u32 major, minor = 0;
> > > > > +		bool enabled;
> > > > > +
> > > > > +		if (nla_type(attr) != NFSD_A_SERVER_PROTO_VERSION)
> > > > > +			continue;
> > > > > +
> > > > > +		if (nla_parse_nested(tb, NFSD_A_VERSION_MAX, attr,
> > > > > +				     nfsd_version_nl_policy, info->extack) < 0)
> > > > > +			continue;
> > > > > +
> > > > > +		if (!tb[NFSD_A_VERSION_MAJOR])
> > > > > +			continue;
> > > > > +
> > > > > +		major = nla_get_u32(tb[NFSD_A_VERSION_MAJOR]);
> > > > > +		if (tb[NFSD_A_VERSION_MINOR])
> > > > > +			minor = nla_get_u32(tb[NFSD_A_VERSION_MINOR]);
> > > > > +
> > > > > +		enabled = nla_get_flag(tb[NFSD_A_VERSION_ENABLED]);
> > > > > +
> > > > > +		switch (major) {
> > > > > +		case 4:
> > > > > +			nfsd_minorversion(nn, minor, enabled ? NFSD_SET : NFSD_CLEAR);
> > > > > +			break;
> > > > > +		case 3:
> > > > > +		case 2:
> > > > > +			if (!minor)
> > > > > +				nfsd_vers(nn, major, enabled ? NFSD_SET : NFSD_CLEAR);
> > > > > +			break;
> > > > > +		default:
> > > > > +			break;
> > > > > +		}
> > > > > +	}
> > > > > +
> > > > > +	mutex_unlock(&nfsd_mutex);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +/**
> > > > > + * nfsd_nl_version_get_doit - get the nfs enabled versions
> > > > > + * @skb: reply buffer
> > > > > + * @info: netlink metadata and command arguments
> > > > > + *
> > > > > + * Return 0 on success or a negative errno.
> > > > > + */
> > > > > +int nfsd_nl_version_get_doit(struct sk_buff *skb, struct genl_info *info)
> > > > > +{
> > > > > +	struct nfsd_net *nn;
> > > > > +	int i, err;
> > > > > +	void *hdr;
> > > > > +
> > > > > +	skb = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL);
> > > > > +	if (!skb)
> > > > > +		return -ENOMEM;
> > > > > +
> > > > > +	hdr = genlmsg_iput(skb, info);
> > > > > +	if (!hdr) {
> > > > > +		err = -EMSGSIZE;
> > > > > +		goto err_free_msg;
> > > > > +	}
> > > > > +
> > > > > +	mutex_lock(&nfsd_mutex);
> > > > > +	nn = net_generic(genl_info_net(info), nfsd_net_id);
> > > > > +
> > > > > +	for (i = 2; i <= 4; i++) {
> > > > > +		int j;
> > > > > +
> > > > > +		for (j = 0; j <= NFSD_SUPPORTED_MINOR_VERSION; j++) {
> > > > > +			struct nlattr *attr;
> > > > > +
> > > > > +			/* Don't record any versions the kernel doesn't have
> > > > > +			 * compiled in
> > > > > +			 */
> > > > > +			if (!nfsd_support_version(i))
> > > > > +				continue;
> > > > > +
> > > > > +			/* NFSv{2,3} does not support minor numbers */
> > > > > +			if (i < 4 && j)
> > > > > +				continue;
> > > > > +
> > > > > +			attr = nla_nest_start(skb,
> > > > > +					      NFSD_A_SERVER_PROTO_VERSION);
> > > > > +			if (!attr) {
> > > > > +				err = -EINVAL;
> > > > > +				goto err_nfsd_unlock;
> > > > > +			}
> > > > > +
> > > > > +			if (nla_put_u32(skb, NFSD_A_VERSION_MAJOR, i) ||
> > > > > +			    nla_put_u32(skb, NFSD_A_VERSION_MINOR, j)) {
> > > > > +				err = -EINVAL;
> > > > > +				goto err_nfsd_unlock;
> > > > > +			}
> > > > > +
> > > > > +			/* Set the enabled flag if the version is enabled */
> > > > > +			if (nfsd_vers(nn, i, NFSD_TEST) &&
> > > > > +			    (i < 4 || nfsd_minorversion(nn, j, NFSD_TEST)) &&
> > > > > +			    nla_put_flag(skb, NFSD_A_VERSION_ENABLED)) {
> > > > > +				err = -EINVAL;
> > > > > +				goto err_nfsd_unlock;
> > > > > +			}
> > > > > +
> > > > > +			nla_nest_end(skb, attr);
> > > > > +		}
> > > > > +	}
> > > > > +
> > > > > +	mutex_unlock(&nfsd_mutex);
> > > > > +	genlmsg_end(skb, hdr);
> > > > > +
> > > > > +	return genlmsg_reply(skb, info);
> > > > > +
> > > > > +err_nfsd_unlock:
> > > > > +	mutex_unlock(&nfsd_mutex);
> > > > > +err_free_msg:
> > > > > +	nlmsg_free(skb);
> > > > > +
> > > > > +	return err;
> > > > > +}
> > > > > +
> > > > >  /**
> > > > >   * nfsd_net_init - Prepare the nfsd_net portion of a new net namespace
> > > > >   * @net: a freshly-created network namespace
> > > > > diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
> > > > > index ca193f7ff0e1..4fc91f50138a 100644
> > > > > --- a/fs/nfsd/nfssvc.c
> > > > > +++ b/fs/nfsd/nfssvc.c
> > > > > @@ -133,8 +133,7 @@ struct svc_program		nfsd_program = {
> > > > >  	.pg_rpcbind_set		= nfsd_rpcbind_set,
> > > > >  };
> > > > >  
> > > > > -static bool
> > > > > -nfsd_support_version(int vers)
> > > > > +bool nfsd_support_version(int vers)
> > > > >  {
> > > > >  	if (vers >= NFSD_MINVERS && vers < NFSD_NRVERS)
> > > > >  		return nfsd_version[vers] != NULL;
> > > > > diff --git a/include/uapi/linux/nfsd_netlink.h b/include/uapi/linux/nfsd_netlink.h
> > > > > index ccc78a5ee650..8a0a2b344923 100644
> > > > > --- a/include/uapi/linux/nfsd_netlink.h
> > > > > +++ b/include/uapi/linux/nfsd_netlink.h
> > > > > @@ -38,10 +38,28 @@ enum {
> > > > >  	NFSD_A_SERVER_WORKER_MAX = (__NFSD_A_SERVER_WORKER_MAX - 1)
> > > > >  };
> > > > >  
> > > > > +enum {
> > > > > +	NFSD_A_VERSION_MAJOR = 1,
> > > > > +	NFSD_A_VERSION_MINOR,
> > > > > +	NFSD_A_VERSION_ENABLED,
> > > > > +
> > > > > +	__NFSD_A_VERSION_MAX,
> > > > > +	NFSD_A_VERSION_MAX = (__NFSD_A_VERSION_MAX - 1)
> > > > > +};
> > > > > +
> > > > > +enum {
> > > > > +	NFSD_A_SERVER_PROTO_VERSION = 1,
> > > > > +
> > > > > +	__NFSD_A_SERVER_PROTO_MAX,
> > > > > +	NFSD_A_SERVER_PROTO_MAX = (__NFSD_A_SERVER_PROTO_MAX - 1)
> > > > > +};
> > > > > +
> > > > >  enum {
> > > > >  	NFSD_CMD_RPC_STATUS_GET = 1,
> > > > >  	NFSD_CMD_THREADS_SET,
> > > > >  	NFSD_CMD_THREADS_GET,
> > > > > +	NFSD_CMD_VERSION_SET,
> > > > > +	NFSD_CMD_VERSION_GET,
> > > > >  
> > > > >  	__NFSD_CMD_MAX,
> > > > >  	NFSD_CMD_MAX = (__NFSD_CMD_MAX - 1)
> > > > > -- 
> > > > > 2.44.0
> > > > > 
> > > > > 
> > > > 
> > > 
> > > -- 
> > > Jeff Layton <jlayton@...nel.org>
> > > 
> > 
> 
> -- 
> Jeff Layton <jlayton@...nel.org>
> 

Powered by blists - more mailing lists