linux-kernel - Re: [PATCH v17 00/51] Introduce QC USB SND audio offloading support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <2024021922-privatize-runt-495e@gregkh>
Date: Mon, 19 Feb 2024 11:38:36 +0100
From: Greg KH <gregkh@...uxfoundation.org>
To: Mathias Nyman <mathias.nyman@...ux.intel.com>
Cc: Wesley Cheng <quic_wcheng@...cinc.com>, srinivas.kandagatla@...aro.org,
	mathias.nyman@...el.com, perex@...ex.cz, conor+dt@...nel.org,
	corbet@....net, lgirdwood@...il.com, andersson@...nel.org,
	krzysztof.kozlowski+dt@...aro.org, Thinh.Nguyen@...opsys.com,
	broonie@...nel.org, bgoswami@...cinc.com, tiwai@...e.com,
	robh+dt@...nel.org, konrad.dybcio@...aro.org,
	linux-kernel@...r.kernel.org, devicetree@...r.kernel.org,
	linux-sound@...r.kernel.org, linux-usb@...r.kernel.org,
	linux-arm-msm@...r.kernel.org, linux-doc@...r.kernel.org,
	alsa-devel@...a-project.org
Subject: Re: [PATCH v17 00/51] Introduce QC USB SND audio offloading support

On Mon, Feb 19, 2024 at 12:27:18PM +0200, Mathias Nyman wrote:
> On 17.2.2024 17.25, Greg KH wrote:
> > On Fri, Feb 16, 2024 at 04:09:26PM -0800, Wesley Cheng wrote:
> > > Several Qualcomm based chipsets can support USB audio offloading to a
> > > dedicated audio DSP, which can take over issuing transfers to the USB
> > > host controller.  The intention is to reduce the load on the main
> > > processors in the SoC, and allow them to be placed into lower power modes.
> > > There are several parts to this design:
> > >    1. Adding ASoC binding layer
> > >    2. Create a USB backend for Q6DSP
> > >    3. Introduce XHCI interrupter support
> > >    4. Create vendor ops for the USB SND driver
> > > 
> > >        USB                          |            ASoC
> > > --------------------------------------------------------------------
> > >                                     |  _________________________
> > >                                     | |sm8250 platform card     |
> > >                                     | |_________________________|
> > >                                     |         |           |
> > >                                     |      ___V____   ____V____
> > >                                     |     |Q6USB   | |Q6AFE    |
> > >                                     |     |"codec" | |"cpu"    |
> > >                                     |     |________| |_________|
> > >                                     |         ^  ^        ^
> > >                                     |         |  |________|
> > >                                     |      ___V____    |
> > >                                     |     |SOC-USB |   |
> > >     ________       ________               |        |   |
> > >    |USB SND |<--->|QC offld|<------------>|________|   |
> > >    |(card.c)|     |        |<----------                |
> > >    |________|     |________|___     | |                |
> > >        ^               ^       |    | |    ____________V_________
> > >        |               |       |    | |   |APR/GLINK             |
> > >     __ V_______________V_____  |    | |   |______________________|
> > >    |USB SND (endpoint.c)     | |    | |              ^
> > >    |_________________________| |    | |              |
> > >                ^               |    | |   ___________V___________
> > >                |               |    | |->|audio DSP              |
> > >     ___________V_____________  |    |    |_______________________|
> > >    |XHCI HCD                 |<-    |
> > >    |_________________________|      |
> > > 
> > > 
> > > Adding ASoC binding layer:
> > > soc-usb: Intention is to treat a USB port similar to a headphone jack.
> > > The port is always present on the device, but cable/pin status can be
> > > enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
> > > communicate with USB SND.
> > > 
> > > Create a USB backend for Q6DSP:
> > > q6usb: Basic backend driver that will be responsible for maintaining the
> > > resources needed to initiate a playback stream using the Q6DSP.  Will
> > > be the entity that checks to make sure the connected USB audio device
> > > supports the requested PCM format.  If it does not, the PCM open call will
> > > fail, and userpsace ALSA can take action accordingly.
> > > 
> > > Introduce XHCI interrupter support:
> > > XHCI HCD supports multiple interrupters, which allows for events to be routed
> > > to different event rings.  This is determined by "Interrupter Target" field
> > > specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
> > > 
> > > Events in the offloading case will be routed to an event ring that is assigned
> > > to the audio DSP.
> > > 
> > > Create vendor ops for the USB SND driver:
> > > qc_audio_offload: This particular driver has several components associated
> > > with it:
> > > - QMI stream request handler
> > > - XHCI interrupter and resource management
> > > - audio DSP memory management
> > > 
> > > When the audio DSP wants to enable a playback stream, the request is first
> > > received by the ASoC platform sound card.  Depending on the selected route,
> > > ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
> > > will send an AFE port start command (with enabling the USB playback path), and
> > > the audio DSP will handle the request accordingly.
> > > 
> > > Part of the AFE USB port start handling will have an exchange of control
> > > messages using the QMI protocol.  The qc_audio_offload driver will populate the
> > > buffer information:
> > > - Event ring base address
> > > - EP transfer ring base address
> > > 
> > > and pass it along to the audio DSP.  All endpoint management will now be handed
> > > over to the DSP, and the main processor is not involved in transfers.
> > > 
> > > Overall, implementing this feature will still expose separate sound card and PCM
> > > devices for both the platorm card and USB audio device:
> > >   0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
> > >                        SM8250-MTP-WCD9380-WSA8810-VA-DMIC
> > >   1 [Audio          ]: USB-Audio - USB Audio
> > >                        Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
> > > 
> > > This is to ensure that userspace ALSA entities can decide which route to take
> > > when executing the audio playback.  In the above, if card#1 is selected, then
> > > USB audio data will take the legacy path over the USB PCM drivers, etc...
> > > 
> > > This feature was validated using:
> > > - tinymix: set/enable the multimedia path to route to USB backend
> > > - tinyplay: issue playback on platform card
> > 
> > I've applied patches 1-10 and the 2 dts changes here, as those all had
> > acks from the relevant maintainers already.
> > 
> 
> Patch 10/10 is based on an old POC patch by me, but it's heavily modified.
> 
> It looks like it does a few minor things that are not optimal, like extra
> spinlock/unlock, and wait_for_completion_timeout() with magical timeout value.
> I haven't tested this version, but I guess any fixes or cleanups can be done
> later on top of it.

I can revert it now if you want, just let me know.

thanks,

greg k-h