[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <Zpgxnszbsf8_uEzf@pengutronix.de>
Date: Wed, 17 Jul 2024 23:03:26 +0200
From: Michael Grzeschik <mgr@...gutronix.de>
To: Andrzej Pietrasiewicz <andrzej.p@...labora.com>
Cc: Eric Van Hensbergen <ericvh@...nel.org>,
Latchesar Ionkov <lucho@...kov.net>,
Dominique Martinet <asmadeus@...ewreck.org>,
Christian Schoenebeck <linux_oss@...debyte.com>,
Jonathan Corbet <corbet@....net>,
Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
v9fs@...ts.linux.dev, linux-doc@...r.kernel.org,
linux-kernel@...r.kernel.org, linux-usb@...r.kernel.org,
kernel@...gutronix.de, Jan Luebbe <jlu@...gutronix.de>
Subject: Re: [PATCH v6 3/3] tools: usb: p9_fwd: add usb gadget packet
forwarder script
Hi Andrzej,
On Mon, Jun 17, 2024 at 01:46:01PM +0200, Michael Grzeschik wrote:
>On Thu, Jun 06, 2024 at 10:19:33AM +0200, Andrzej Pietrasiewicz wrote:
>>Hi,
>>
>>W dniu 5.06.2024 o 15:32, Michael Grzeschik pisze:
>>>This patch is adding an small python tool to forward 9pfs requests
>>>from the USB gadget to an existing 9pfs TCP server. Since currently all
>>>9pfs servers lack support for the usb transport this tool is an useful
>>>helper to get started.
>>>
>>>Refer the Documentation section "USBG Example" in
>>>Documentation/filesystems/9p.rst on how to use it.
>>>
>>>Signed-off-by: Jan Luebbe <jlu@...gutronix.de>
>>>Signed-off-by: Michael Grzeschik <m.grzeschik@...gutronix.de>
>>>
>>
>>This time --id alone worked for me (as well as --id combined with --path)
>>but --path alone did not:
>
>This is currently right and expected. Why is using path alone of any
>more use? Since you could then grab any device with two bulk endpoints
>without even ensuring that it is supposed to behave as expected with
>this script.
Ping!
Can this series move on?
Regards,
Michael
>>$ sudo python3 tools/usb/p9_fwd.py --id 0xabcd:0xef01 list
>>Bus | Addr | Manufacturer | Product | ID | Path
>>--- | ---- | ---------------- | ---------------- | --------- | ----
>> 3 | 39 | unknown | unknown | abcd:ef01 | 3-1.3.3.5.4
>>
>>$ sudo python3 tools/usb/p9_fwd.py --id 0xabcd:0xef01 connect -p 9999
>>2024-06-06 10:14:45,368 INFO found device: 3/39 located at 3-1.3.3.5.4
>>2024-06-06 10:14:45,369 INFO claiming interface:
>> INTERFACE 0: Vendor Specific ===========================
>> bLength : 0x9 (9 bytes)
>> bDescriptorType : 0x4 Interface
>> bInterfaceNumber : 0x0
>> bAlternateSetting : 0x0
>> bNumEndpoints : 0x2
>> bInterfaceClass : 0xff Vendor Specific
>> bInterfaceSubClass : 0xff
>> bInterfaceProtocol : 0x9
>> iInterface : 0x1 usb9pfs input to output
>> ENDPOINT 0x81: Bulk IN ===============================
>> bLength : 0x7 (7 bytes)
>> bDescriptorType : 0x5 Endpoint
>> bEndpointAddress : 0x81 IN
>> bmAttributes : 0x2 Bulk
>> wMaxPacketSize : 0x200 (512 bytes)
>> bInterval : 0x0
>> ENDPOINT 0x1: Bulk OUT ===============================
>> bLength : 0x7 (7 bytes)
>> bDescriptorType : 0x5 Endpoint
>> bEndpointAddress : 0x1 OUT
>> bmAttributes : 0x2 Bulk
>> wMaxPacketSize : 0x200 (512 bytes)
>> bInterval : 0x0
>>2024-06-06 10:14:45,370 INFO interface claimed
>>2024-06-06 10:14:45,370 INFO connected to server
>>^C
>>
>>$ sudo python3 tools/usb/p9_fwd.py --path 3-1.3.3.5.4 connect -p 9999
>>Traceback (most recent call last):
>> File "/home/ap/Collabora/kernel-rk/tools/usb/p9_fwd.py", line 243, in <module>
>> main()
>> File "/home/ap/Collabora/kernel-rk/tools/usb/p9_fwd.py", line 239, in main
>> args.func(args)
>> File "/home/ap/Collabora/kernel-rk/tools/usb/p9_fwd.py", line 190, in connect
>> f = Forwarder(server=(args.server, args.port), vid=vid, pid=pid, path=args.path)
>> File "/home/ap/Collabora/kernel-rk/tools/usb/p9_fwd.py", line 60, in __init__
>> raise ValueError("Device not found")
>>ValueError: Device not found
>>
>>
>>>---
>>>v5 -> v6:
>>> - set path parameter to None when unused
>>>v4 -> v5:
>>> - updated documentation for new subcommands list/connect
>>> - run ruff format
>>> - make vid and pid parameterized
>>> - add list as subcommand to scan for devices
>>> - move connect to extra subcommand
>>>v3 -> v4: -
>>>v2 -> v3: -
>>>v1 -> v2:
>>> - added usbg 9pfs detailed instructions to 9p.rst doc
>>>---
>>> Documentation/filesystems/9p.rst | 41 +++++++
>>> tools/usb/p9_fwd.py | 243 +++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 284 insertions(+)
>>>
>>>diff --git a/Documentation/filesystems/9p.rst b/Documentation/filesystems/9p.rst
>>>index 10cf79dc287f8..2cc85f3e8659f 100644
>>>--- a/Documentation/filesystems/9p.rst
>>>+++ b/Documentation/filesystems/9p.rst
>>>@@ -67,6 +67,47 @@ To mount a 9p FS on a USB Host accessible via the gadget as root filesystem::
>>> where <device> is the tag associated by the usb gadget transport.
>>> It is defined by the configfs instance name.
>>>+USBG Example
>>>+============
>>>+
>>>+The USB host exports a filesystem, while the gadget on the USB device
>>>+side makes it mountable.
>>>+
>>>+Diod (9pfs server) and the forwarder are on the development host, where
>>>+the root filesystem is actually stored. The gadget is initialized during
>>>+boot (or later) on the embedded board. Then the forwarder will find it
>>>+on the USB bus and start forwarding requests.
>>>+
>>>+In this case the 9p requests come from the device and are handled by the
>>>+host. The reason is that USB device ports are normally not available on
>>>+PCs, so a connection in the other direction would not work.
>>>+
>>>+When using the usbg transport, for now there is no native usb host
>>>+service capable to handle the requests from the gadget driver. For
>>>+this we have to use the extra python tool p9_fwd.py from tools/usb.
>>>+
>>>+Just start the 9pfs capable network server like diod/nfs-ganesha e.g.:
>>>+
>>>+ $ diod -f -n -d 0 -S -l 0.0.0.0:9999 -e $PWD
>>>+
>>>+Optionaly scan your bus if there are more then one usbg gadgets to find their path:
>>>+
>>>+ $ python $kernel_dir/tools/usb/p9_fwd.py list
>>>+
>>>+ Bus | Addr | Manufacturer | Product | ID | Path
>>>+ --- | ---- | ---------------- | ---------------- | --------- | ----
>>>+ 2 | 67 | unknown | unknown | 1d6b:0109 | 2-1.1.2
>>>+ 2 | 68 | unknown | unknown | 1d6b:0109 | 2-1.1.3
>>>+
>>>+Then start the python transport:
>>>+
>>>+ $ python $kernel_dir/tools/usb/p9_fwd.py --path 2-1.1.2 connect -p 9999
>>>+
>>>+After that the gadget driver can be used as described above.
>>>+
>>>+One use-case is to use it as an alternative to NFS root booting during
>>>+the development of embedded Linux devices.
>>>+
>>> Options
>>> =======
>>>diff --git a/tools/usb/p9_fwd.py b/tools/usb/p9_fwd.py
>>>new file mode 100755
>>>index 0000000000000..12c76cbb046b7
>>>--- /dev/null
>>>+++ b/tools/usb/p9_fwd.py
>>>@@ -0,0 +1,243 @@
>>>+#!/usr/bin/env python3
>>>+# SPDX-License-Identifier: GPL-2.0
>>>+
>>>+import argparse
>>>+import errno
>>>+import logging
>>>+import socket
>>>+import struct
>>>+import time
>>>+
>>>+import usb.core
>>>+import usb.util
>>>+
>>>+
>>>+def path_from_usb_dev(dev):
>>>+ """Takes a pyUSB device as argument and returns a string.
>>>+ The string is a Path representation of the position of the USB device on the USB bus tree.
>>>+
>>>+ This path is used to find a USB device on the bus or all devices connected to a HUB.
>>>+ The path is made up of the number of the USB controller followed be the ports of the HUB tree."""
>>>+ if dev.port_numbers:
>>>+ dev_path = ".".join(str(i) for i in dev.port_numbers)
>>>+ return f"{dev.bus}-{dev_path}"
>>>+ return ""
>>>+
>>>+
>>>+HEXDUMP_FILTER = "".join(chr(x).isprintable() and chr(x) or "." for x in range(128)) + "." * 128
>>>+
>>>+
>>>+class Forwarder:
>>>+ @staticmethod
>>>+ def _log_hexdump(data):
>>>+ if not logging.root.isEnabledFor(logging.TRACE):
>>>+ return
>>>+ L = 16
>>>+ for c in range(0, len(data), L):
>>>+ chars = data[c : c + L]
>>>+ dump = " ".join(f"{x:02x}" for x in chars)
>>>+ printable = "".join(HEXDUMP_FILTER[x] for x in chars)
>>>+ line = f"{c:08x} {dump:{L*3}s} |{printable:{L}s}|"
>>>+ logging.root.log(logging.TRACE, "%s", line)
>>>+
>>>+ def __init__(self, server, vid, pid, path):
>>>+ self.stats = {
>>>+ "c2s packets": 0,
>>>+ "c2s bytes": 0,
>>>+ "s2c packets": 0,
>>>+ "s2c bytes": 0,
>>>+ }
>>>+ self.stats_logged = time.monotonic()
>>>+
>>>+ def find_filter(dev):
>>>+ dev_path = path_from_usb_dev(dev)
>>>+ if path is not None:
>>>+ return dev_path == path
>>>+ return True
>>>+
>>>+ dev = usb.core.find(idVendor=vid, idProduct=pid, custom_match=find_filter)
>>>+ if dev is None:
>>>+ raise ValueError("Device not found")
>>>+
>>>+ logging.info(f"found device: {dev.bus}/{dev.address} located at {path_from_usb_dev(dev)}")
>>>+
>>>+ # dev.set_configuration() is not necessary since g_multi has only one
>>>+ usb9pfs = None
>>>+ # g_multi adds 9pfs as last interface
>>>+ cfg = dev.get_active_configuration()
>>>+ for intf in cfg:
>>>+ # we have to detach the usb-storage driver from multi gadget since
>>>+ # stall option could be set, which will lead to spontaneous port
>>>+ # resets and our transfers will run dead
>>>+ if intf.bInterfaceClass == 0x08:
>>>+ if dev.is_kernel_driver_active(intf.bInterfaceNumber):
>>>+ dev.detach_kernel_driver(intf.bInterfaceNumber)
>>>+
>>>+ if intf.bInterfaceClass == 0xFF and intf.bInterfaceSubClass == 0xFF and intf.bInterfaceProtocol == 0x09:
>>>+ usb9pfs = intf
>>>+ if usb9pfs is None:
>>>+ raise ValueError("Interface not found")
>>>+
>>>+ logging.info(f"claiming interface:\n{usb9pfs}")
>>>+ usb.util.claim_interface(dev, usb9pfs.bInterfaceNumber)
>>>+ ep_out = usb.util.find_descriptor(
>>>+ usb9pfs,
>>>+ custom_match=lambda e: usb.util.endpoint_direction(e.bEndpointAddress) == usb.util.ENDPOINT_OUT,
>>>+ )
>>>+ assert ep_out is not None
>>>+ ep_in = usb.util.find_descriptor(
>>>+ usb9pfs,
>>>+ custom_match=lambda e: usb.util.endpoint_direction(e.bEndpointAddress) == usb.util.ENDPOINT_IN,
>>>+ )
>>>+ assert ep_in is not None
>>>+ logging.info("interface claimed")
>>>+
>>>+ self.ep_out = ep_out
>>>+ self.ep_in = ep_in
>>>+ self.dev = dev
>>>+
>>>+ # create and connect socket
>>>+ self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>>+ self.s.connect(server)
>>>+
>>>+ logging.info("connected to server")
>>>+
>>>+ def c2s(self):
>>>+ """forward a request from the USB client to the TCP server"""
>>>+ data = None
>>>+ while data is None:
>>>+ try:
>>>+ logging.log(logging.TRACE, "c2s: reading")
>>>+ data = self.ep_in.read(self.ep_in.wMaxPacketSize)
>>>+ except usb.core.USBTimeoutError:
>>>+ logging.log(logging.TRACE, "c2s: reading timed out")
>>>+ continue
>>>+ except usb.core.USBError as e:
>>>+ if e.errno == errno.EIO:
>>>+ logging.debug("c2s: reading failed with %s, retrying", repr(e))
>>>+ time.sleep(0.5)
>>>+ continue
>>>+ logging.error("c2s: reading failed with %s, aborting", repr(e))
>>>+ raise
>>>+ size = struct.unpack("<I", data[:4])[0]
>>>+ while len(data) < size:
>>>+ data += self.ep_in.read(size - len(data))
>>>+ logging.log(logging.TRACE, "c2s: writing")
>>>+ self._log_hexdump(data)
>>>+ self.s.send(data)
>>>+ logging.debug("c2s: forwarded %i bytes", size)
>>>+ self.stats["c2s packets"] += 1
>>>+ self.stats["c2s bytes"] += size
>>>+
>>>+ def s2c(self):
>>>+ """forward a response from the TCP server to the USB client"""
>>>+ logging.log(logging.TRACE, "s2c: reading")
>>>+ data = self.s.recv(4)
>>>+ size = struct.unpack("<I", data[:4])[0]
>>>+ while len(data) < size:
>>>+ data += self.s.recv(size - len(data))
>>>+ logging.log(logging.TRACE, "s2c: writing")
>>>+ self._log_hexdump(data)
>>>+ while data:
>>>+ written = self.ep_out.write(data)
>>>+ assert written > 0
>>>+ data = data[written:]
>>>+ if size % self.ep_out.wMaxPacketSize == 0:
>>>+ logging.log(logging.TRACE, "sending zero length packet")
>>>+ self.ep_out.write(b"")
>>>+ logging.debug("s2c: forwarded %i bytes", size)
>>>+ self.stats["s2c packets"] += 1
>>>+ self.stats["s2c bytes"] += size
>>>+
>>>+ def log_stats(self):
>>>+ logging.info("statistics:")
>>>+ for k, v in self.stats.items():
>>>+ logging.info(f" {k+':':14s} {v}")
>>>+
>>>+ def log_stats_interval(self, interval=5):
>>>+ if (time.monotonic() - self.stats_logged) < interval:
>>>+ return
>>>+
>>>+ self.log_stats()
>>>+ self.stats_logged = time.monotonic()
>>>+
>>>+
>>>+def try_get_usb_str(dev, name):
>>>+ try:
>>>+ with open(f"/sys/bus/usb/devices/{dev.bus}-{dev.address}/{name}") as f:
>>>+ return f.read().strip()
>>>+ except FileNotFoundError:
>>>+ return None
>>>+
>>>+
>>>+def list_usb(args):
>>>+ vid, pid = [int(x, 16) for x in args.id.split(":", 1)]
>>>+
>>>+ print("Bus | Addr | Manufacturer | Product | ID | Path")
>>>+ print("--- | ---- | ---------------- | ---------------- | --------- | ----")
>>>+ for dev in usb.core.find(find_all=True, idVendor=vid, idProduct=pid):
>>>+ path = path_from_usb_dev(dev) or ""
>>>+ manufacturer = try_get_usb_str(dev, "manufacturer") or "unknown"
>>>+ product = try_get_usb_str(dev, "product") or "unknown"
>>>+ print(
>>>+ f"{dev.bus:3} | {dev.address:4} | {manufacturer:16} | {product:16} | {dev.idVendor:04x}:{dev.idProduct:04x} | {path:18}"
>>>+ )
>>>+
>>>+
>>>+def connect(args):
>>>+ vid, pid = [int(x, 16) for x in args.id.split(":", 1)]
>>>+
>>>+ f = Forwarder(server=(args.server, args.port), vid=vid, pid=pid, path=args.path)
>>>+
>>>+ try:
>>>+ while True:
>>>+ f.c2s()
>>>+ f.s2c()
>>>+ f.log_stats_interval()
>>>+ finally:
>>>+ f.log_stats()
>>>+
>>>+
>>>+def main():
>>>+ parser = argparse.ArgumentParser(
>>>+ description="Forward 9PFS requests from USB to TCP",
>>>+ )
>>>+
>>>+ parser.add_argument("--id", type=str, default="1d6b:0109", help="vid:pid of target device")
>>>+ parser.add_argument("--path", type=str, required=False, help="path of target device")
>>>+ parser.add_argument("-v", "--verbose", action="count", default=0)
>>>+
>>>+ subparsers = parser.add_subparsers()
>>>+ subparsers.required = True
>>>+ subparsers.dest = "command"
>>>+
>>>+ parser_list = subparsers.add_parser("list", help="List all connected 9p gadgets")
>>>+ parser_list.set_defaults(func=list_usb)
>>>+
>>>+ parser_connect = subparsers.add_parser(
>>>+ "connect", help="Forward messages between the usb9pfs gadget and the 9p server"
>>>+ )
>>>+ parser_connect.set_defaults(func=connect)
>>>+ connect_group = parser_connect.add_argument_group()
>>>+ connect_group.required = True
>>>+ parser_connect.add_argument("-s", "--server", type=str, default="127.0.0.1", help="server hostname")
>>>+ parser_connect.add_argument("-p", "--port", type=int, default=564, help="server port")
>>>+
>>>+ args = parser.parse_args()
>>>+
>>>+ logging.TRACE = logging.DEBUG - 5
>>>+ logging.addLevelName(logging.TRACE, "TRACE")
>>>+
>>>+ if args.verbose >= 2:
>>>+ level = logging.TRACE
>>>+ elif args.verbose:
>>>+ level = logging.DEBUG
>>>+ else:
>>>+ level = logging.INFO
>>>+ logging.basicConfig(level=level, format="%(asctime)-15s %(levelname)-8s %(message)s")
>>>+
>>>+ args.func(args)
>>>+
>>>+
>>>+if __name__ == "__main__":
>>>+ main()
>>>
>>
>>
>
>--
>Pengutronix e.K. | |
>Steuerwalder Str. 21 | http://www.pengutronix.de/ |
>31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
>Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)
Powered by blists - more mailing lists