Message-ID: <e1eac325-233e-43eb-9ca0-9746893462d7@quicinc.com>
Date: Thu, 3 Apr 2025 08:58:11 -0700
From: Wesley Cheng <quic_wcheng@...cinc.com>
To: Stephan Gerhold <stephan.gerhold@...aro.org>
CC: <srinivas.kandagatla@...aro.org>, <mathias.nyman@...el.com>,
        <perex@...ex.cz>, <conor+dt@...nel.org>, <dmitry.torokhov@...il.com>,
        <corbet@....net>, <broonie@...nel.org>, <lgirdwood@...il.com>,
        <krzk+dt@...nel.org>, <pierre-louis.bossart@...ux.intel.com>,
        <Thinh.Nguyen@...opsys.com>, <tiwai@...e.com>, <robh@...nel.org>,
        <gregkh@...uxfoundation.org>, <linux-kernel@...r.kernel.org>,
        <devicetree@...r.kernel.org>, <linux-sound@...r.kernel.org>,
        <linux-input@...r.kernel.org>, <linux-usb@...r.kernel.org>,
        <linux-arm-msm@...r.kernel.org>, <linux-doc@...r.kernel.org>,
        Luca Weiss
	<luca.weiss@...rphone.com>
Subject: Re: [PATCH v36 22/31] ASoC: qcom: qdsp6: Introduce USB AFE port to
 q6dsp

Hi Stephan,

On 4/3/2025 6:45 AM, Stephan Gerhold wrote:
> On Wed, Apr 02, 2025 at 05:54:45PM -0700, Wesley Cheng wrote:
>> On 4/2/2025 5:23 PM, Wesley Cheng wrote:
>>> On 4/2/2025 7:41 AM, Stephan Gerhold wrote:
>>>> On Tue, Apr 01, 2025 at 04:47:41PM -0700, Wesley Cheng wrote:
>>>>> On 4/1/2025 1:16 AM, Stephan Gerhold wrote:
>>>>>> On Mon, Mar 31, 2025 at 12:52:19PM -0700, Wesley Cheng wrote:
>>>>>>> On 3/26/2025 2:57 AM, Stephan Gerhold wrote:
>>>>>>>> On Tue, Mar 25, 2025 at 04:18:03PM -0700, Wesley Cheng wrote:
>>>>>>>>> On 3/25/2025 2:24 AM, Stephan Gerhold wrote:
>>>>>>>>>> On Tue, Mar 18, 2025 at 05:51:32PM -0700, Wesley Cheng wrote:
>>>>>>>>>>> The QC ADSP is able to support USB
>>>>>>>>>>> playback endpoints, so that the main
>>>>>>>>>>> application processor can be placed into
>>>>>>>>>>> lower CPU power modes.  This adds
>>>>>>>>>>> the required AFE port configurations and
>>>>>>>>>>> port start command to start an
>>>>>>>>>>> audio session.
>>>>>>>>>>>
>>>>>>>>>>> Specifically, the QC ADSP can support
>>>>>>>>>>> all potential endpoints that are
>>>>>>>>>>> exposed by the audio data interface. 
>>>>>>>>>>> This includes isochronous data
>>>>>>>>>>> endpoints, in either synchronous mode or
>>>>>>>>>>> asynchronous mode. In the latter
>>>>>>>>>>> case both implicit or explicit feedback
>>>>>>>>>>> endpoints are supported.  The size
>>>>>>>>>>> of audio samples sent per USB frame
>>>>>>>>>>> (microframe) will be adjusted based on
>>>>>>>>>>> information received on the feedback endpoint.
>>>>>>>>>>>
>>>>>>>>>>> Some pre-requisites are needed before
>>>>>>>>>>> issuing the AFE port start command,
>>>>>>>>>>> such as setting the USB AFE dev_token. 
>>>>>>>>>>> This carries information about the
>>>>>>>>>>> available USB SND cards and PCM devices
>>>>>>>>>>> that have been discovered on the
>>>>>>>>>>> USB bus.  The dev_token field is used by
>>>>>>>>>>> the audio DSP to notify the USB
>>>>>>>>>>> offload driver of which card and PCM index to enable playback on.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Wesley Cheng <quic_wcheng@...cinc.com>
>>>>>>>>>>> ---
>>>>>>>>>>>   sound/soc/qcom/qdsp6/q6afe-dai.c         |  60 +++++++
>>>>>>>>>>>  
>>>>>>>>>>> sound/soc/qcom/qdsp6/q6afe.c            
>>>>>>>>>>> | 192 ++++++++++++++++++++++-
>>>>>>>>>>>   sound/soc/qcom/qdsp6/q6afe.h             |  36 ++++-
>>>>>>>>>>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c |  23 +++
>>>>>>>>>>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h |   1 +
>>>>>>>>>>>   sound/soc/qcom/qdsp6/q6routing.c         |  32 +++-
>>>>>>>>>>>   6 files changed, 341 insertions(+), 3 deletions(-)
>>>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/sound/soc/qcom/qdsp6/q6routing.c
>>>>>>>>>>> b/sound/soc/qcom/qdsp6/q6routing.c
>>>>>>>>>>> index 90228699ba7d..b7439420b425 100644
>>>>>>>>>>> --- a/sound/soc/qcom/qdsp6/q6routing.c
>>>>>>>>>>> +++ b/sound/soc/qcom/qdsp6/q6routing.c
>>>>>>>>>>> @@ -435,6 +435,26 @@ static struct
>>>>>>>>>>> session_data *get_session_from_id(struct
>>>>>>>>>>> msm_routing_data *data,
>>>>>>>>>>>       return NULL;
>>>>>>>>>>>   }
>>>>>>>>>>> +
>>>>>>>>>>> +static bool is_usb_routing_enabled(struct msm_routing_data *data)
>>>>>>>>>>> +{
>>>>>>>>>>> +    int i;
>>>>>>>>>>> +
>>>>>>>>>>> +    /*
>>>>>>>>>>> +     * Loop through current sessions to
>>>>>>>>>>> see if there are active routes
>>>>>>>>>>> +     * to the USB_RX backend DAI.  The
>>>>>>>>>>> USB offload routing is designed
>>>>>>>>>>> +     * similarly to the non offload
>>>>>>>>>>> path.  If there are multiple PCM
>>>>>>>>>>> +     * devices associated with the ASoC
>>>>>>>>>>> platform card, only one active
>>>>>>>>>>> +     * path can be routed to the USB offloaded endpoint.
>>>>>>>>>>> +     */
>>>>>>>>>>> +    for (i = 0; i < MAX_SESSIONS; i++) {
>>>>>>>>>>> +        if (data->sessions[i].port_id == USB_RX)
>>>>>>>>>>> +            return true;
>>>>>>>>>>> +    }
>>>>>>>>>>> +
>>>>>>>>>>> +    return false;
>>>>>>>>>>> +}
>>>>>>>>>>
>>>>>>>>>> What is different about USB_RX compared to
>>>>>>>>>> other output ports we have in
>>>>>>>>>> Q6AFE? Obviously, we can only play one
>>>>>>>>>> stream on an output port. But
>>>>>>>>>> doesn't the ADSP mix streams together when
>>>>>>>>>> you have multiple routes?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This patch will limit the USB_RX from being able
>>>>>>>>> to be mixed to multiple
>>>>>>>>> q6adm paths.
>>>>>>>>>
>>>>>>>>>> Also, this doesn't actually check for *active* routes only. It just
>>>>>>>>>> looks if any other MultiMedia DAI is
>>>>>>>>>> configured to output to USB_RX.
>>>>>>>>>> That doesn't mean they will ever be active at the same time.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes, the main reason being that that is the
>>>>>>>>> mechanism we use to populate
>>>>>>>>> the active offload path within the USB SND card mixer.
>>>>>>>>>
>>>>>>>>>> I might for example want to have MultiMedia1 and MultiMedia2 both
>>>>>>>>>> configured to output to USB_RX. Let's assume
>>>>>>>>>> MultiMedia1 is a normal PCM
>>>>>>>>>> DAI, MultiMedia2 is a compress offload DAI. When I want to playback
>>>>>>>>>> normal audio, I go through MultiMedia1, when
>>>>>>>>>> I want to play compressed
>>>>>>>>>> audio, I go through MultiMedia2. Only one of them active at a time.
>>>>>>>>>> Why can't I set this up statically in the mixers?
>>>>>>>>>>
>>>>>>>>>> If you confirm that it is really impossible
>>>>>>>>>> to have multiple streams
>>>>>>>>>> mixed together to the USB_RX output in the
>>>>>>>>>> ADSP, then this should be a
>>>>>>>>>> runtime check instead when starting the stream IMO.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We can have multiple streams being mixed together, but it will get
>>>>>>>>> confusing because it changes the definition that
>>>>>>>>> we had discussed about in
>>>>>>>>> the past about the overall design for the interaction w/ userspace.
>>>>>>>>> Although we (QC) only support a single USB audio
>>>>>>>>> device for offloading,
>>>>>>>>> there could be other situations where the audio
>>>>>>>>> DSP can support multiple
>>>>>>>>> devices.  The assumption is that each MM path is
>>>>>>>>> assigned to a USB device.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Are you referring to the "USB Offload Playback Route
>>>>>>>> PCM#*" mixers here?
>>>>>>>> They could just refer to first of the configured MM paths, if someone
>>>>>>>> decides to route multiple paths to the USB backend. Looking at
>>>>>>>> q6usb_update_offload_route(), I think the implementation does that
>>>>>>>> already.
>>>>>>>>
>>>>>>>> I think it's fine that the userspace API for
>>>>>>>> automatically "probing" the
>>>>>>>> PCM device supports only a single path to the USB backend. But if
>>>>>>>> someone wants to bypass the automatic probing and configure a more
>>>>>>>> advanced setup, do we need to forbid that?
>>>>>>>>
>>>>>>>> Asked differently: what would happen if we remove this check here and
>>>>>>>> handle USB_RX like any other Q6AFE output port?
>>>>>>>> Would anything break for
>>>>>>>> the userspace interface?
>>>>>>>>
>>>>>>>
>>>>>>> So I took a look at seeing how the Q6ADM/ASM
>>>>>>> interactions would work for
>>>>>>> the situation where if user tried to start both MM1/2
>>>>>>> streams at the same
>>>>>>> time over the USB offload path.  In this scenario, we
>>>>>>> see that the Q6USB BE
>>>>>>> DAI operations, ie startup, hw_params, etc... gets
>>>>>>> called one time for the
>>>>>>> initial stream.  For example, if I start playback on MM1, then that
>>>>>>> triggers the USB BE DAI to be brought up.
>>>>>>>
>>>>>>> When I start playback on MM2, since MM1 already called
>>>>>>> dpcm_be_dai_startup(), then be->dpcm[stream].users will
>>>>>>> be greater than
>>>>>>> zero.  This would cause the __soc_pcm_open() to be
>>>>>>> skipped for the USB BE
>>>>>>> DAI, so I wouldn't be able to check the runtime status
>>>>>>> at the Q6USB backend
>>>>>>> DAI.  However, we do track current streaming sessions
>>>>>>> done over Q6 ADM and
>>>>>>> it does save the AFE port associated to each COPP
>>>>>>> allocation, so I think its
>>>>>>> reasonable to see if there is already a COPP entry for
>>>>>>> the USB AFE port, to
>>>>>>> fail the open() call associated to the FE DAI.
>>>>>>>
>>>>>>
>>>>>> This sounds like a reasonable approach *if* we have to prevent multiple
>>>>>> MM DAIs from streaming to the USB AFE port at the same time.
>>>>>>
>>>>>> It's still unclear to me why we have to introduce this
>>>>>> limitation in the
>>>>>> first place. I think the questions from my previous email are still
>>>>>> open. Can you check them again?
>>>>>>
>>>>>
>>>>> So I checked with our audio DSP folks, and they mentioned there isn't
>>>>> technically a limitation from mixing multiple ADM streams from
>>>>> their end.
>>>>> My observations are as follows:
>>>>> - Using tinyplay to open and play on different FE PCM devices
>>>>> (ie MM1 and
>>>>> MM2), both streams are audible on the USB headset (intermixed).
>>>>> - If starting playback on MM1 first, before MM2, then once playback is
>>>>> complete on MM1, the ADM close is also affecting the MM2 stream.
>>>>> Basically, MM2 stops when the MM1 audio file is completed.
>>>>> - Due to the abrupt/incomplete closing of the MM2 ADM stream, looks like
>>>>> the audio DSP is not handling that case well, so subsequent
>>>>> playbacks fail.
>>>>>
>>>>
>>>> Thanks a lot for testing this!
>>>>
>>>>> I did find a possible reason for this, and it seems to be due to
>>>>> some code
>>>>> unrelated to the USB offload path directly.  It looks like the Q6ADM is
>>>>> currently built in a way that you can only route streams to a single
>>>>> endpoint, even though we do have reference counting for each
>>>>> COPP profile.
>>>>> So even after the first MM1 ADM stream completes and the PCM device is
>>>>> closed, the MM1 ADM close callback will issue a q6adm_device_close() for
>>>>> the USB AFE port.
>>>>
>>>> Yeah, this sounds like a bug to me...
>>>>
>>>>>
>>>>> I made some test changes to account for the refcount before issuing the
>>>>> q6adm_device_close(), and that seemed to work.  Once the MM1
>>>>> stream closes,
>>>>> it allows for the MM2 stream to close/finish before issuing the command,
>>>>> and that allows for proper cleanup of the audio data.
>>>>>
>>>>> IMO, I would like to keep the initial behavior (ie, blocking the
>>>>> additional
>>>>> stream open from the kernel) until I can get some more testing done, and
>>>>> figure out if this is the correct approach.  If it is, I can submit a
>>>>> follow up series to address it.
>>>>>
>>>>
>>>> I'm fine with this approach. My only two concerns/comments are:
>>>>
>>>>   1. Let's make sure we don't waste too much time finding the ideal
>>>>      approach to implement the additional blocking, if this will likely
>>>>      end up getting removed again anyway.
>>>>
>>>>   2. How can we justify making this change only for USB? Your analysis
>>>>      sounds like the issue applies to all types of output ports. This was
>>>>      exactly my main concern with the check you added in this patch:
>>>>      I would expect QDSP6 treats all ports exactly the same. This means
>>>>      we should add this additional check for all the output ports...
>>>>
>>>
>>> Fair points...I will extend this check across all ports until I get back
>>> with seeing how we can address the mixing situation.
>>>
>>
>> Sorry for replying again.  I think I'll at least limit this to the playback
>> path only for now.  I'm not sure how the behavior yet is for the record
>> path, since I don't really have that use case to test with yet :).  I will
>> see how it looks like when the record USB offload is done.
>>
> 
> FWIW, I tested this on MSM8916:
> 
>  - MultiMedia1 -> PRI_MI2S_RX
>  - MultiMedia3 -> PRI_MI2S_RX
>  - TERT_MI2S_TX -> MultiMedia2
>  - TERT_MI2S_TX -> MultiMedia5
> 
> 2x speaker-test to MultiMedia1 and MultiMedia3 (pink noise + sine wave):
>  -> Audio playback correctly mixed together
>  -> Playback aborts for both when stopping one of the processes
>       Write error: -5,I/O error
>       xrun_recovery failed: -5,I/O error
>       Transfer failed: I/O error
>  -> Kernel log:
>     [  585.679421] q6routing 4080000.remoteproc:smd-edge:apr:service@8:routing: Found Matching Copp 0x0
>     [  587.917135] qcom-q6adm aprsvc:service:4:8: cmd = 0x10327 return error = 0x2
>     [  587.917378] qcom-q6adm aprsvc:service:4:8: DSP returned error[2]
>     [  587.917409] qcom-q6adm aprsvc:service:4:8: Failed to close copp -22
> 
> 2x arecord from MultiMedia2 and MultiMedia5:
>  -> Both processes record audio as expected
>  -> Recording aborts for both when stopping one of the processes
>       arecord: pcm_read:2272: read error: I/O error
>  -> Kernel log:
>     [  646.125612] q6routing 4080000.remoteproc:smd-edge:apr:service@8:routing: Found Matching Copp 0x0
>     [  654.247867] q6asm-dai 4080000.remoteproc:smd-edge:apr:service@7:dais: command[0x10bdb] not expecting rsp
>     [  654.815474] q6asm-dai 4080000.remoteproc:smd-edge:apr:service@7:dais: command[0x10bdb] not expecting rsp
>     [  654.825568] qcom-q6adm aprsvc:service:4:8: cmd = 0x10327 return error = 0x2
>     [  654.825955] qcom-q6adm aprsvc:service:4:8: DSP returned error[2]
>     [  654.826013] qcom-q6adm aprsvc:service:4:8: Failed to close copp -22
> 
> => Both playback and recording are affected. As you suspected it's
>    unrelated to USB.
> 

Great, thanks for confirming both paths show the same issue :).

> This feels like a bug to me rather than an intentional limitation. If
> you can share the fix you made I can test it here.
> 

Change I tried was fairly straightforward.  Sharing it below:

diff --git a/sound/soc/qcom/qdsp6/q6adm.c b/sound/soc/qcom/qdsp6/q6adm.c
index fe719c4c32b3..7880fbbe815f 100644
--- a/sound/soc/qcom/qdsp6/q6adm.c
+++ b/sound/soc/qcom/qdsp6/q6adm.c
@@ -578,12 +579,13 @@ int q6adm_close(struct device *dev, struct q6copp *copp)
     struct q6adm *adm = dev_get_drvdata(dev->parent);
     int ret = 0;

-    ret = q6adm_device_close(adm, copp, copp->afe_port, copp->copp_idx);
-    if (ret < 0) {
-        dev_err(adm->dev, "Failed to close copp %d\n", ret);
-        return ret;
+    if (kref_read(&copp->refcount) == 1) {
+        ret = q6adm_device_close(adm, copp, copp->afe_port, copp->copp_idx);
+        if (ret < 0) {
+            dev_err(adm->dev, "Failed to close copp %d\n", ret);
+            return ret;
+        }
     }
-
     kref_put(&copp->refcount, q6adm_free_copp);

     return 0;

Thanks
Wesley Cheng

Powered by blists - more mailing lists