linux-kernel - Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA+D8AM_pqbjQaE59n6Qm5gypLb8zPAwOpJR+_ZEG89-q3B8pg@mail.gmail.com>
Date:   Fri, 25 Aug 2023 11:46:43 +0800
From:   Shengjiu Wang <shengjiu.wang@...il.com>
To:     Mark Brown <broonie@...nel.org>
Cc:     Takashi Iwai <tiwai@...e.de>, Hans Verkuil <hverkuil@...all.nl>,
        Shengjiu Wang <shengjiu.wang@....com>, sakari.ailus@....fi,
        tfiga@...omium.org, m.szyprowski@...sung.com, mchehab@...nel.org,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        Xiubo.Lee@...il.com, festevam@...il.com, nicoleotsuka@...il.com,
        lgirdwood@...il.com, perex@...ex.cz, tiwai@...e.com,
        alsa-devel@...a-project.org, linuxppc-dev@...ts.ozlabs.org
Subject: Re: [RFC PATCH v2 0/7] Add audio support in v4l2 framework

On Fri, Aug 25, 2023 at 4:21 AM Mark Brown <broonie@...nel.org> wrote:
>
> On Thu, Aug 24, 2023 at 07:03:09PM +0200, Takashi Iwai wrote:
> > Shengjiu Wang wrote:
>
> > > But there are several issues:
> > > 1. Need to create sound cards.  ASRC module support multi instances, then
> > > need to create multi sound cards for each instance.
>
> > Hm, why can't it be multiple PCM instances instead?
>
> I'm having a hard time following this one too.
>
> > > 2. The ASRC is an entirety but with DPCM we need to separate input port and
> > > output port to playback substream and capture stream. Synchronous between
> > > playback substream and capture substream is a problem.
> > > How to start them and stop them at the same time.
>
> > This could be done by enforcing the full duplex and linking the both
> > PCM streams, I suppose.
>
> > > So shall we make the decision that we can go to the V4L2 solution?
>
> > Honestly speaking, I don't mind much whether it's implemented in V2L4
> > or not -- at least for the kernel part, we can reorganize / refactor
> > things internally.  But, the biggest remaining question to me is
> > whether this user-space interface is the most suitable one.  Is it
> > well defined, usable and maintained for the audio applications?  Or
> > is it meant to be a stop-gap for a specific use case?
>
> I'm having a really hard time summoning much enthusiasm for using v4l
> here, it feels like this is heading down the same bodge route as DPCM
> but directly as ABI so even harder to fix properly.  That said all the
> ALSA APIs are really intended to be used in real time and this sounds
> like a non real time application?  I don't fully understand what the
> actual use case is here.

Thanks for your reply.

This asrc memory to memory (memory ->asrc->memory) case is a non
real time use case.

User fills the input buffer to the asrc module,  after conversion, then asrc
sends back the output buffer to user. So it is not a traditional ALSA playback
and capture case. I think it is not good to create sound card for it,  it is
not a sound card actually.

It is a specific use case,  there is no reference in current kernel.
v4l2 memory to memory is the closed implementation,  v4l2 current
support video, image, radio, tuner, touch devices, so it is not
complicated to add support for this specific audio case.

Maybe you can go through these patches first.  Because we
had implemented the "memory -> asrc ->i2s device-> codec"
use case in ALSA.  Now the "memory->asrc->memory" needs
to reuse the code in asrc driver, so the first 3 patches is for refining
the code to make it can be shared by the "memory->asrc->memory"
driver.

The main change is in the v4l2 side, A /dev/vl42-audio will be created,
user applications only use the ioctl of v4l2 framework.

Best regards
Wang Shengjiu