Message-Id: <1420730220-20224-3-git-send-email-hadarh@mellanox.com>
Date:	Thu,  8 Jan 2015 17:16:59 +0200
From:	Hadar Hen Zion <hadarh@...lanox.com>
To:	"David S. Miller" <davem@...emloft.net>, gregkh@...uxfoundation.org
Cc:	netdev@...r.kernel.org, Amir Vadai <amirv@...lanox.com>,
	Hadar Har-Zion <hadarh@...lanox.com>,
	Yevgeny Petrilin <yevgenyp@...lanox.com>,
	Or Gerlitz <ogerlitz@...lanox.com>, shannon.nelson@...el.com,
	Doug Ledford <dledford@...hat.com>, greearb@...delatech.com
Subject: [RFC net-next 2/3] devconf: Add configuration module for setting pre-load parameters

Introducing a new kernel infrastructure using configfs to allow the
configuration of low-level device functionality that needs to be sorted
out before a module is loaded. The suggested solution is generic and is
designed to suite any type of device.

The devconf module presented in this commit gives generic services for a
'dev_c_<driver name>' module. The 'dev_c_<driver name>' module will be
written by each vendor who wishes to implement a pre-load configuration
module for its driver.

Motivation and design for the devconf infrastructure is available under:
Documentation/filesystems/configfs/devconf.txt.

Signed-off-by: Hadar Hen Zion <hadarh@...lanox.com>
Signed-off-by: Amir Vadai <amirv@...lanox.com>
---
 Documentation/filesystems/configfs/devconf.txt |  135 ++++++++++++++++++++
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/devconf/Kconfig                        |    6 +
 drivers/devconf/Makefile                       |    3 +
 drivers/devconf/driver.c                       |  160 ++++++++++++++++++++++++
 drivers/devconf/main.c                         |  103 +++++++++++++++
 include/linux/devconf.h                        |   69 ++++++++++
 8 files changed, 479 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/filesystems/configfs/devconf.txt
 create mode 100644 drivers/devconf/Kconfig
 create mode 100644 drivers/devconf/Makefile
 create mode 100644 drivers/devconf/driver.c
 create mode 100644 drivers/devconf/main.c
 create mode 100644 include/linux/devconf.h

diff --git a/Documentation/filesystems/configfs/devconf.txt b/Documentation/filesystems/configfs/devconf.txt
new file mode 100644
index 0000000..a1b8039
--- /dev/null
+++ b/Documentation/filesystems/configfs/devconf.txt
@@ -0,0 +1,135 @@
+Configfs infrastructure for setting pre-load parameters for a module
+====================================================================
+
+When configuring a device at an early boot stage, most kernel drivers use
+module parameters (the parameters' settings can be determined in modprobe.d
+config files.) These parameters are difficult to manage, and one of the
+reasons is that module parameters are set per driver and not per device
+(NICs using the same driver cannot be set with different configurations).
+Furthermore, using other existing configuration tools like ethtool, ifconfig,
+ip link commands or sysfs entries is not applicable, since they all rely on
+having a netdev already set up.
+
+In the past, 'request_firmware' solution for configuration parameters was
+suggested by Shannon Nelson from Intel[1]. The idea was rejected by Greg KH, who
+claimed it was abusive of the request_firmware mechanism. Greg suggested using
+configfs for device configuration instead (as done by the USB gadget driver).
+
+As a solution, we introduce a new kernel infrastructure using configfs to
+allow the configuration of the device. The goal is to set low-level device
+functionality that needs to be sorted out before a module is loaded.
+
+Configfs Solution
+=====================
+The implemented configfs solution is composed of one generic module ('devconf')
+that can load configuration modules per driver. The devconf should be loaded
+at a very early boot stage, before other kernel modules are loaded (or
+compiled as in-tree). After configfs is mounted, it creates a new directory
+under configfs, called 'devices'.
+
+In the example presented below, systemd mechanism is being used, but the
+configuration can also work with older init systems.
+
+1. A systemd service, scheduled by the OS to run before kernel modules
+are loaded, creates a directory under 'devices' with the name of the driver.
+The devconf module will try to load a configuration module for this driver (if
+exists).
+
+2. Drivers using this mechanism will be made of two parts: a
+configuration module and the driver itself.
+
+3. Later on, when the OS loads the driver, it uses the configuration
+sub-tree.
+
+To avoid dependencies between the modules, in case the configuration module
+does not exist or is not loaded, the driver will use default values.
+
+Example
+=======
+The below mlx4_core configuration file is only an example. Since the
+infrastructure is generic, each vendor can choose how to identify its devices
+(for example: bdf, mac address, uuid etc.).
+
+If /etc/devconf.d/mlx4_core.conf is as follows (see below), systemd service
+will use it to run the following commands (see below).
+
+mlx4_core.conf file:
+--------------------
+[pdevs]
+	[0000:00:08.0]
+		dmfs = 0
+		[ports]
+			[1]
+			type = 2
+			[2]
+			type =2
+	[0000:00:04.0]
+		dmfs = 0
+		[ports]
+			[1]
+			type = 5
+			[2]
+			type =1
+
+Systemd commands:
+-----------------
+$ mkdir /sys/kernel/config/devices/mlx4_core
+
+devconf module will try to load dev_c_mlx4.ko
+
+$ mkdir /sys/kernel/config/devices/mlx4_core/pdevs
+$ mkdir /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/
+$ mkdir /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/ports
+$ mkdir /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/ports/1
+$ mkdir /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/ports/2
+
+$ echo 5 > /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/ports/1/type
+$ echo 1 > /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/ports/2/type
+$ echo 0 > /sys/kernel/config/devices/mlx4_core/pdevs/0000:00:08.0/dmfs
+[...]
+
+dev_c_mlx4 will reject the above configuration, printing a clear message to
+dmesg:
+"mlx4_core: Unknown port type '5' for device 0000:00:08.0. Use 0 for auto,
+1 for IB and 2 for ETH"
+
+The driver configuration module validates and prepares configfs sub-tree for
+the driver itself. A configuration sub-tree for a specific module will be
+created under /sys/kernel/config/devices/<driver name>, and propagated from a
+configuration file on the OS (located under: /etc/devconf.d/).
+
+Configfs Sub-tree Example (for mlx4_core):
+------------------------------------------
+$ cd /sys/kernel/config/
+$ tree devices/
+devices/
+ └── mlx4_core
+     └── pdevs
+           ├── 0000:00:04.0
+ 	    │   ├── dmfs
+ 	    │   └── ports
+ 	    │       ├── 1
+ 	    │       │   └── type
+ 	    │       └── 2
+ 	    │   	└── type
+ 	    └── 0000:00:08.0
+ 	    	├── dmfs
+ 	    	└── ports
+ 	    	    ├── 1
+ 	    	    │   └── type
+ 	    	    └── 2
+ 	    		└── type
+
+The systemd service that populates configfs is part of the sysinit target.
+The service is executed after mounting configfs and before udev load kernel
+modules.
+
+To use devconf infrastructure, the following should be included:
+1. Devconf systemd service
+2. Configuration file in the right format under: /etc/devconf.d/
+
+This suggested solution is generic and designed to suite any type of device.
+The goal is to make this solution generic enough so all kinds of drivers
+are able to use it.
+
+[1] - https://lkml.org/lkml/2013/1/10/606
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 694d5a7..5a489e3 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -186,4 +186,6 @@ source "drivers/thunderbolt/Kconfig"
 
 source "drivers/android/Kconfig"
 
+source "drivers/devconf/Kconfig"
+
 endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index 67d2334..05e0d48 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -163,3 +163,4 @@ obj-$(CONFIG_RAS)		+= ras/
 obj-$(CONFIG_THUNDERBOLT)	+= thunderbolt/
 obj-$(CONFIG_CORESIGHT)		+= coresight/
 obj-$(CONFIG_ANDROID)		+= android/
+obj-$(CONFIG_DEVCONF)		+= devconf/
diff --git a/drivers/devconf/Kconfig b/drivers/devconf/Kconfig
new file mode 100644
index 0000000..3a98617
--- /dev/null
+++ b/drivers/devconf/Kconfig
@@ -0,0 +1,6 @@
+config DEVCONF
+	tristate "Driver configuration module"
+	depends on CONFIGFS_FS
+	default n
+	---help---
+	 Allow Setting pre-load parameters for drivers using configfs file system.
diff --git a/drivers/devconf/Makefile b/drivers/devconf/Makefile
new file mode 100644
index 0000000..f2a5f1f
--- /dev/null
+++ b/drivers/devconf/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_DEVCONF)  += devconf.o
+
+devconf-y := main.o driver.o
diff --git a/drivers/devconf/driver.c b/drivers/devconf/driver.c
new file mode 100644
index 0000000..230afa6
--- /dev/null
+++ b/drivers/devconf/driver.c
@@ -0,0 +1,160 @@
+/*
+ * Copyright (c) 2015 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/err.h>
+
+#include <linux/devconf.h>
+
+static LIST_HEAD(config_drivers_list);
+static DEFINE_MUTEX(drivers_lock);
+
+static struct devconf_config_driver *find_config_driver(const char *name)
+{
+	struct devconf_config_driver *cd, *tmp;
+
+	cd = ERR_PTR(-ENOENT);
+	mutex_lock(&drivers_lock);
+	list_for_each_entry(tmp, &config_drivers_list, list) {
+		if (strcmp(name, tmp->name))
+			continue;
+		else
+			cd = tmp;
+		if (!try_module_get(cd->mod)) {
+			cd = ERR_PTR(-EBUSY);
+			break;
+		}
+		cd->set_config_object(cd, name);
+		break;
+	}
+	mutex_unlock(&drivers_lock);
+	return cd;
+}
+
+struct devconf_config_driver *devconf_create_config_driver(const char *name)
+{
+	struct devconf_config_driver *cdrv;
+	int ret;
+
+	cdrv = find_config_driver(name);
+	if (!IS_ERR(cdrv))
+		return cdrv;
+	if (PTR_ERR(cdrv) != -ENOENT)
+		return cdrv;
+	ret = request_module("devconf:%s", name);
+	if (ret < 0)
+		return ERR_PTR(ret);
+	return find_config_driver(name);
+}
+
+void devconf_put_config_driver(struct devconf_config_driver *cdrv)
+{
+	struct module *mod;
+
+	if (!cdrv)
+		return;
+
+	mod = cdrv->mod;
+	module_put(mod);
+}
+EXPORT_SYMBOL_GPL(devconf_put_config_driver);
+
+int devconf_device_register(struct devconf_config_driver *newd)
+{
+	struct devconf_config_driver *cd;
+	int ret = -EEXIST;
+
+	mutex_lock(&drivers_lock);
+	list_for_each_entry(cd, &config_drivers_list, list) {
+		if (!strcmp(cd->name, newd->name))
+			goto out;
+	}
+	ret = 0;
+	list_add_tail(&newd->list, &config_drivers_list);
+out:
+	mutex_unlock(&drivers_lock);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(devconf_device_register);
+
+void devconf_device_unregister(struct devconf_config_driver *cd)
+{
+	mutex_lock(&drivers_lock);
+	list_del(&cd->list);
+	mutex_unlock(&drivers_lock);
+}
+EXPORT_SYMBOL_GPL(devconf_device_unregister);
+
+struct devconf_config_driver *devconf_get_config_driver(const char *name)
+{
+	struct devconf_config_driver *cd, *tmp;
+
+	cd = ERR_PTR(-ENOENT);
+	mutex_lock(&drivers_lock);
+	list_for_each_entry(tmp, &config_drivers_list, list) {
+		if (!strcmp(name, tmp->name)) {
+			cd = tmp;
+			break;
+		}
+	}
+	mutex_unlock(&drivers_lock);
+	return cd;
+}
+EXPORT_SYMBOL(devconf_get_config_driver);
+
+struct devconf_config_object
+*devconf_get_config_object(struct config_group *group, const char *name)
+{
+	struct config_item *item;
+	struct devconf_config_object *cobj;
+
+	mutex_lock(&group->cg_subsys->su_mutex);
+	item = config_group_find_item(group, name);
+	mutex_unlock(&group->cg_subsys->su_mutex);
+
+	cobj = to_config_obj(item);
+	return cobj;
+}
+EXPORT_SYMBOL(devconf_get_config_object);
+
+ssize_t devconf_get_int_attr(struct devconf_config_driver *cdrv,
+			     struct devconf_config_object *cobj,
+			     const char *param, int *value)
+{
+	if (cdrv->get_int_attr)
+		return cdrv->get_int_attr(cobj, param, value);
+	else
+		return -ENOSYS;
+}
+EXPORT_SYMBOL(devconf_get_int_attr);
diff --git a/drivers/devconf/main.c b/drivers/devconf/main.c
new file mode 100644
index 0000000..e308d4b
--- /dev/null
+++ b/drivers/devconf/main.c
@@ -0,0 +1,103 @@
+/*
+ * Copyright (c) 2015 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/configfs.h>
+
+#include <linux/devconf.h>
+
+#define MAX_NAME_LEN	40
+
+static struct config_group *device_driver_make(struct config_group *group,
+					       const char *name)
+{
+	struct devconf_config_driver *cdrv;
+
+	cdrv = devconf_create_config_driver(name);
+	if (IS_ERR(cdrv))
+		return ERR_CAST(cdrv);
+
+	return &cdrv->cobj.group;
+}
+
+static struct configfs_group_operations devices_group_ops = {
+	.make_group	= device_driver_make,
+};
+
+static struct config_item_type devices_type = {
+	.ct_group_ops	= &devices_group_ops,
+	.ct_owner	= THIS_MODULE,
+};
+
+static struct configfs_subsystem devices_subsys = {
+	.su_group = {
+		.cg_item = {
+			.ci_namebuf = "devices",
+			.ci_type = &devices_type,
+		},
+	},
+};
+
+static int __init devconf_init(void)
+{
+	int ret;
+	struct configfs_subsystem *subsys = &devices_subsys;
+
+	config_group_init(&subsys->su_group);
+	mutex_init(&subsys->su_mutex);
+	ret = configfs_register_subsystem(subsys);
+	if (ret) {
+		pr_err("Error %d while registering subsystem %s\n",
+		       ret,
+		       subsys->su_group.cg_item.ci_namebuf);
+		goto out_unregister;
+	}
+	return 0;
+
+out_unregister:
+	configfs_unregister_subsystem(subsys);
+
+	return ret;
+}
+
+static void __exit devconf_exit(void)
+{
+	struct configfs_subsystem *subsys = &devices_subsys;
+
+	configfs_unregister_subsystem(subsys);
+}
+
+module_init(devconf_init);
+module_exit(devconf_exit);
+MODULE_LICENSE("GPL");
diff --git a/include/linux/devconf.h b/include/linux/devconf.h
new file mode 100644
index 0000000..ce78662
--- /dev/null
+++ b/include/linux/devconf.h
@@ -0,0 +1,69 @@
+#ifndef	__LINUX_DEVCONF_H
+#define	__LINUX_DEVCONF_H
+#include <linux/configfs.h>
+
+struct devconf_config_object {
+	struct config_group group;
+};
+
+struct devconf_config_driver {
+	struct list_head list;
+	struct devconf_config_object cobj;
+	const char *name;
+	struct module *mod;
+	void (*set_config_object)(struct devconf_config_driver *cdrv,
+				  const char *name);
+	int (*get_int_attr)(struct devconf_config_object *cobj,
+			    const char *attr_name, int *val);
+};
+
+struct devconf_config_driver *devconf_get_config_driver(const char *name);
+
+struct devconf_config_object
+*devconf_get_config_object(struct config_group *group, const char *name);
+
+ssize_t devconf_get_int_attr(struct devconf_config_driver *cdrv,
+			     struct devconf_config_object *cobj,
+			     const char *param, int *value);
+void devconf_device_unregister(struct devconf_config_driver *cd);
+int devconf_device_register(struct devconf_config_driver *new_cd);
+struct devconf_config_driver *devconf_create_config_driver(const char *name);
+void devconf_put_config_driver(struct devconf_config_driver *cdrv);
+
+static inline struct devconf_config_object
+*to_config_obj(struct config_item *item)
+{
+	return item ? container_of(item, struct devconf_config_object,
+				   group.cg_item) : NULL;
+}
+
+static inline struct devconf_config_driver
+*to_config_driver(struct config_item *item)
+{
+	return item ? container_of(item, struct devconf_config_driver,
+				   cobj.group.cg_item) : NULL;
+}
+
+#define DECLARE_DEVCONF_DRIVER(_name, _set_obj, _get_int_attr)		\
+	static struct devconf_config_driver _name ## devconf_driver = {	\
+		.name = __stringify(_name),				\
+		.mod  = THIS_MODULE,					\
+		.set_config_object = _set_obj,				\
+		.get_int_attr = _get_int_attr,				\
+	};								\
+	MODULE_ALIAS("devconf:" __stringify(_name))
+
+#define DECLARE_DEVCONF_DRIVER_INIT(_name, _set_obj, _get_int_attr)	\
+	DECLARE_DEVCONF_DRIVER(_name, _set_obj, _get_int_attr);		\
+	static int __init _name ## mod_init(void)			\
+	{								\
+		return devconf_device_register(&_name ## devconf_driver);\
+	}								\
+	static void __exit _name ## mod_exit(void)			\
+	{								\
+		devconf_device_unregister(&_name ## devconf_driver);	\
+	}								\
+	module_init(_name ## mod_init);					\
+	module_exit(_name ## mod_exit)
+
+#endif
-- 
1.7.8.2

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists