linux-kernel - Re: [PATCH 0/8] MUSE: Userspace backed MTD v3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAFLxGvzQo9H8h5XpLuVDYC8uaRrPmkH444yFv3jX0LNCs_nTmg@mail.gmail.com>
Date:   Mon, 1 Feb 2021 14:14:01 +0100
From:   Richard Weinberger <richard.weinberger@...il.com>
To:     Richard Weinberger <richard@....at>
Cc:     Miklos Szeredi <miklos@...redi.hu>,
        Miquel Raynal <miquel.raynal@...tlin.com>,
        Vignesh Raghavendra <vigneshr@...com>,
        Boris Brezillon <boris.brezillon@...labora.com>,
        rminnich@...gle.com, sven@...fation.org,
        LKML <linux-kernel@...r.kernel.org>,
        linux-mtd@...ts.infradead.org,
        "open list:FUSE: FILESYSTEM..." <fuse-devel@...ts.sourceforge.net>
Subject: Re: [PATCH 0/8] MUSE: Userspace backed MTD v3

*friendly FUSE maintainer ping* :-)

On Mon, Jan 25, 2021 at 12:24 AM Richard Weinberger <richard@....at> wrote:
>
> I'm happy to announce the first non-RFC version of this patch set.
> Over the xmas holidays I found some time to experiment with various userspace
> implementations of MTDs and gave the kernel side more fine-tuning.
>
> Rationale:
> ----------
>
> When working with flash devices a common task is emulating them to run various
> tests or inspect dumps from real hardware. To achieve that we have plenty of
> emulators in the MTD subsystem: mtdram, block2mtd, nandsim.
>
> Each of them implements an ad-hoc MTD and have various drawbacks.
> Over the last years some developers tried to extend them but these attempts
> often got rejected because they added just more adhoc feature instead of
> addressing overall problems.
>
> MUSE is a novel approach to address the need of advanced MTD emulators.
> Advanced means in this context supporting different (vendor specific) image
> formats, different ways for fault injection (fuzzing) and recoding/replaying
> IOs to emulate power cuts.
>
> The core goal of MUSE is having the complexity on the userspace side and
> only a small MTD driver in kernelspace.
> While playing with different approaches I realized that FUSE offers everything
> we need. So MUSE is a little like CUSE except that it does not implement a
> bare character device but an MTD.
>
> Notes:
> ------
>
> - OOB support is currently limited. Currently MUSE has no support for processing
>   in- and out-band in the same MTD operation. It is good enough to make JFFS2
>   happy. This limitation is because FUSE has no support more than one variable
>   length buffer in a FUSE request.
>   At least I didn’t find a good way to pass more than one buffer to a request.
>   Maybe FUSE folks can correct me. :-)
>
> - Every MTD read/write operation maps 1:1 to a MUSE_READ/WRITE opcode.
>   Since FUSE requests are not cheap, the amount of read/written bytes in a MTD
>   operation as a huge impact on the performance. Especially when NOR style MTDs
>   are implemented in userspace a large writebufsize should be requested to gain
>   good write performance.
>   On the other hand, MTD operations with lengths larger than writesize are *not*
>   split up into multiple MUSE_READ/WRITE requests. This means that userspace
>   has to split them manually when doing power-cut emulation.
>
> - MUSE is not super fast. On my i5 workstation nandsim is almost twice as fast
>   as a NAND flash in userspace. But MUSE is still magnitudes faster than any
>   real world MTD out there. So it is good enough for the use cases I have in
>   mind.
>
> Changelog:
> ----------
>
> Changes since v2 (RFC):
> - OOB support
> - MUSE_READ/WRITE opcodes are no longer a min IO MTD unit
> - MTD partitions support via mtdparts string
> - More code cleanup
> - Code rebased to 5.11-rc4
>
> Changes since v1 (RFC):
> - Rewrote IO path, fuse_direct_io() is no longer used.
>   Instead of cheating fuse_direct_io() use custom ops to implement
>   reading and writing. That way MUSE no longer needs a dummy file object
>   nor a fuse file object.
>   In MTD all IO is synchronous and operations on kernel buffers, this
>   makes IO processing simple for MUSE.
> - Support for bad blocks.
> - No more (ab)use of FUSE ops such as FUSE_FSYNC.
> - Major code cleanup.
>
> This series can also be found at:
> git://git.kernel.org/pub/scm/linux/kernel/git/rw/misc.git muse_v3
>
> Richard Weinberger (8):
>   fuse: Export fuse_simple_request
>   fuse: Export IO helpers
>   fuse: Make cuse_parse_one a common helper
>   mtd: Add MTD_MUSE flag
>   mtd: Allow passing a custom cmdline to cmdline line parser
>   fuse: Add MUSE specific defines FUSE interface
>   fuse: Implement MUSE - MTD in userspace
>   MAINTAINERS: Add entry for MUSE
>
>  Documentation/ABI/testing/sysfs-class-mtd |    8 +
>  MAINTAINERS                               |    7 +
>  drivers/mtd/parsers/cmdlinepart.c         |   73 +-
>  fs/fuse/Kconfig                           |   15 +
>  fs/fuse/Makefile                          |    2 +
>  fs/fuse/cuse.c                            |   58 +-
>  fs/fuse/dev.c                             |    1 +
>  fs/fuse/file.c                            |   16 +-
>  fs/fuse/fuse_i.h                          |   18 +
>  fs/fuse/helper.c                          |   70 ++
>  fs/fuse/muse.c                            | 1086 +++++++++++++++++++++
>  include/linux/mtd/partitions.h            |    2 +
>  include/uapi/linux/fuse.h                 |   76 ++
>  include/uapi/mtd/mtd-abi.h                |    1 +
>  14 files changed, 1346 insertions(+), 87 deletions(-)
>  create mode 100644 fs/fuse/helper.c
>  create mode 100644 fs/fuse/muse.c
>
> --
> 2.26.2
>


-- 
Thanks,
//richard