linux-kernel - Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA+D8AN34-NVrgksRAG014PuHGUssTm0p-KR-HYGe+Pt8Yejxg@mail.gmail.com>
Date:   Wed, 23 Aug 2023 22:33:19 +0800
From:   Shengjiu Wang <shengjiu.wang@...il.com>
To:     Mark Brown <broonie@...nel.org>
Cc:     Takashi Iwai <tiwai@...e.de>, Hans Verkuil <hverkuil@...all.nl>,
        Shengjiu Wang <shengjiu.wang@....com>, sakari.ailus@....fi,
        tfiga@...omium.org, m.szyprowski@...sung.com, mchehab@...nel.org,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        Xiubo.Lee@...il.com, festevam@...il.com, nicoleotsuka@...il.com,
        lgirdwood@...il.com, perex@...ex.cz, tiwai@...e.com,
        alsa-devel@...a-project.org, linuxppc-dev@...ts.ozlabs.org
Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework

On Fri, Aug 11, 2023 at 7:05 PM Shengjiu Wang <shengjiu.wang@...il.com> wrote:
>
> Hi Mark, Takashi
>
> On Thu, Aug 3, 2023 at 9:11 PM Shengjiu Wang <shengjiu.wang@...il.com> wrote:
> >
> > On Thu, Aug 3, 2023 at 1:28 AM Mark Brown <broonie@...nel.org> wrote:
> > >
> > > On Wed, Aug 02, 2023 at 10:41:43PM +0800, Shengjiu Wang wrote:
> > >
> > > > Currently the ASRC in ALSA is to connect to another I2S device as
> > > > a sound card.  But we'd like to the ASRC can be used by user space directly
> > > > that user space application can get the output after conversion from ASRC.
> > >
> > > That sort of use case would be handled via DPCM at the minute, though
> > > persuading it to connect two front ends together might be fun (which is
> > > the sort of reason why we want to push digital information down into
> > > DAPM and make everything a component).
> >
> > Thanks.
> >
> > ASRC M2M case needs to run as fast as possible, no sync clock control.
> > If use sound card to handle ASRC M2M case,  the user application
> > should be aplay/arecord, then we need to consider xrun issue, buffer
> > may timeout, sync between aplay and arecord,  these should't be
> > considered by pure memory to memory operation.
> >
> > DPCM may achitect all the audio things in components and sound
> > card,  it is good. but for the M2M case, it is complcated. not sure
> > it is doable.
> >
>
> Beside the concern in previous mail,
>
> DPCM needs to separate ASRC to be two substreams (playback and capture).
>
> But the ASRC needs the sample rate & format of input and output first
> then start conversion.
>
> If the playback controls the rate & format of input,  capture substream
> controls the rate & format of output,  as a result
> one substream needs to get information(dma buffer address, size...
> rate, format) from another substream, then start both substreams in the
> last substream. How to synchronize these two substreams is a problem.
> One stream can be released but another stream doesn't know .
>
> So I don't think it is a good idea to use DPCM for pure M2M case.
>
> So can I persuade you to consider the V4L2 solution?
>

Just a summary:

Basic M2M conversion can work with DPCM, I have tried with some
workaround to make it work.

But there are several issues:
1. Need to create sound cards.  ASRC module support multi instances, then
need to create multi sound cards for each instance.

2. The ASRC is an entirety but with DPCM we need to separate input port and
output port to playback substream and capture stream. Synchronous between
playback substream and capture substream is a problem.
How to start them and stop them at the same time.

3. How to handle the xrun issue. pause or resume. which are brought by ALSA.

So shall we make the decision that we can go to the V4L2 solution?

Best regards
Wang Shengjiu