| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <0898cc9b-5eaa-41b4-9fc3-2483044c537a@xs4all.nl>
Date: Tue, 19 Mar 2024 08:21:40 +0100
From: Hans Verkuil <hverkuil@...all.nl>
To: Shengjiu Wang <shengjiu.wang@...il.com>
Cc: Shengjiu Wang <shengjiu.wang@....com>, sakari.ailus@....fi,
tfiga@...omium.org, m.szyprowski@...sung.com, mchehab@...nel.org,
linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
Xiubo.Lee@...il.com, festevam@...il.com, nicoleotsuka@...il.com,
lgirdwood@...il.com, broonie@...nel.org, perex@...ex.cz, tiwai@...e.com,
alsa-devel@...a-project.org, linuxppc-dev@...ts.ozlabs.org
Subject: Re: [PATCH v14 00/16] Add audio support in v4l2 framework
On 19/03/2024 4:14 am, Shengjiu Wang wrote:
> On Tue, Mar 19, 2024 at 10:24 AM Shengjiu Wang <shengjiu.wang@...il.com> wrote:
>>
>> On Mon, Mar 18, 2024 at 11:56 PM Hans Verkuil <hverkuil@...all.nl> wrote:
>>>
>>> Hi Shengjiu,
>>>
>>> I was about to prepare a pull request for this when I realized that
>>> entries for the new drivers need to be added to the MAINTAINERS file.
>>>
>>> Can you post patches 'v14 17/16' and 'v14/18/16' adding entries for each
>>> new driver?
>>
>> Ok, How about the change below?
>>
>> VIRTUAL MEM2MEM DRIVER FOR AUDIO
>> M: Hans Verkuil <hverkuil@...all.nl>
>> M: Shengjiu Wang <shengjiu.wang@...il.com>
>> L: linux-media@...r.kernel.org
>> S: Maintained
>> W: https://linuxtv.org
>> T: git git://linuxtv.org/media_tree.git
>> F: drivers/media/test-drivers/vim2m-audio.c
>>
>> NXP ASRC V4L2 MEM2MEM DRIVERS
>> M: Shengjiu Wang <shengjiu.wang@...il.com>
>> L: linux-media@...r.kernel.org
>> S: Maintained
>> W: https://linuxtv.org
>> T: git git://linuxtv.org/media_tree.git
>> F: drivers/media/platform/nxp/imx-asrc.c
>>
>
> BTW, should I merge this change to PATCH 15/16 and PATCH 16/16?
> That I will resend the whole patch series. Seems this is a better way.
That's fine!
Regards,
Hans
>
> Best regrads
> Shengjiu Wang
>
>> best regards
>> wang shengjiu
>>
>>>
>>> Thank you,
>>>
>>> Hans
>>>
>>> On 11/03/2024 10:53 am, Shengjiu Wang wrote:
>>>> Audio signal processing also has the requirement for memory to
>>>> memory similar as Video.
>>>>
>>>> This asrc memory to memory (memory ->asrc->memory) case is a non
>>>> real time use case.
>>>>
>>>> User fills the input buffer to the asrc module, after conversion, then asrc
>>>> sends back the output buffer to user. So it is not a traditional ALSA playback
>>>> and capture case.
>>>>
>>>> It is a specific use case, there is no reference in current kernel.
>>>> v4l2 memory to memory is the closed implementation, v4l2 current
>>>> support video, image, radio, tuner, touch devices, so it is not
>>>> complicated to add support for this specific audio case.
>>>>
>>>> Because we had implemented the "memory -> asrc ->i2s device-> codec"
>>>> use case in ALSA. Now the "memory->asrc->memory" needs
>>>> to reuse the code in asrc driver, so the first 3 patches is for refining
>>>> the code to make it can be shared by the "memory->asrc->memory"
>>>> driver.
>>>>
>>>> The main change is in the v4l2 side, A /dev/vl4-audioX will be created,
>>>> user applications only use the ioctl of v4l2 framework.
>>>>
>>>> Other change is to add memory to memory support for two kinds of i.MX ASRC
>>>> module.
>>>>
>>>> changes in v14:
>>>> - document the reservation of 'AUXX' fourcc format.
>>>> - add v4l2_audfmt_to_fourcc() definition.
>>>>
>>>> changes in v13
>>>> - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst
>>>> - add more description for clock drift in ext-ctrls-audio-m2m.rst
>>>> - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans
>>>> to avoid build issue for kernel test robot
>>>>
>>>> changes in v12
>>>> - minor changes according to comments
>>>> - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag
>>>> - drop bus_info
>>>>
>>>> changes in v11
>>>> - add add-fixed-point-test-controls in vivid.
>>>> - add v4l2_ctrl_fp_compose() helper function for min and max
>>>>
>>>> changes in v10
>>>> - remove FIXED_POINT type
>>>> - change code base on media: v4l2-ctrls: add support for fraction_bits
>>>> - fix issue reported by kernel test robot
>>>> - remove module_alias
>>>>
>>>> changes in v9:
>>>> - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
>>>> - add MEDIA_INTF_T_V4L_AUDIO
>>>> - add media controller support
>>>> - refine the vim2m-audio to support 8k<->16k conversion.
>>>>
>>>> changes in v8:
>>>> - refine V4L2_CAP_AUDIO_M2M to be 0x00000008
>>>> - update doc for FIXED_POINT
>>>> - address comments for imx-asrc
>>>>
>>>> changes in v7:
>>>> - add acked-by from Mark
>>>> - separate commit for fixed point, m2m audio class, audio rate controls
>>>> - use INTEGER_MENU for rate, FIXED_POINT for rate offset
>>>> - remove used fmts
>>>> - address other comments for Hans
>>>>
>>>> changes in v6:
>>>> - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace
>>>> m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two.
>>>> - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT
>>>> - fix warning by kernel test rebot
>>>> - remove some unused format V4L2_AUDIO_FMT_XX
>>>> - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver.
>>>> - rename audm2m to viaudm2m.
>>>>
>>>> changes in v5:
>>>> - remove V4L2_AUDIO_FMT_LPCM
>>>> - define audio pixel format like V4L2_AUDIO_FMT_S8...
>>>> - remove rate and format in struct v4l2_audio_format.
>>>> - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls
>>>> - updata document accordingly.
>>>>
>>>> changes in v4:
>>>> - update document style
>>>> - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit
>>>>
>>>> changes in v3:
>>>> - Modify documents for adding audio m2m support
>>>> - Add audio virtual m2m driver
>>>> - Defined V4L2_AUDIO_FMT_LPCM format type for audio.
>>>> - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case.
>>>> - with modification in v4l-utils, pass v4l2-compliance test.
>>>>
>>>> changes in v2:
>>>> - decouple the implementation in v4l2 and ALSA
>>>> - implement the memory to memory driver as a platfrom driver
>>>> and move it to driver/media
>>>> - move fsl_asrc_common.h to include/sound folder
>>>>
>>>> Hans Verkuil (1):
>>>> media: v4l2-ctrls: add support for fraction_bits
>>>>
>>>> Shengjiu Wang (15):
>>>> ASoC: fsl_asrc: define functions for memory to memory usage
>>>> ASoC: fsl_easrc: define functions for memory to memory usage
>>>> ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
>>>> ASoC: fsl_asrc: register m2m platform device
>>>> ASoC: fsl_easrc: register m2m platform device
>>>> media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
>>>> media: v4l2: Add audio capture and output support
>>>> media: uapi: Define audio sample format fourcc type
>>>> media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
>>>> media: uapi: Add audio rate controls support
>>>> media: uapi: Declare interface types for Audio
>>>> media: uapi: Add an entity type for audio resampler
>>>> media: vivid: add fixed point test controls
>>>> media: imx-asrc: Add memory to memory driver
>>>> media: vim2m-audio: add virtual driver for audio memory to memory
>>>>
>>>> .../media/mediactl/media-types.rst | 11 +
>>>> .../userspace-api/media/v4l/buffer.rst | 6 +
>>>> .../userspace-api/media/v4l/common.rst | 1 +
>>>> .../media/v4l/dev-audio-mem2mem.rst | 71 +
>>>> .../userspace-api/media/v4l/devices.rst | 1 +
>>>> .../media/v4l/ext-ctrls-audio-m2m.rst | 59 +
>>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++
>>>> .../userspace-api/media/v4l/pixfmt.rst | 1 +
>>>> .../media/v4l/vidioc-enum-fmt.rst | 2 +
>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
>>>> .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 +
>>>> .../media/v4l/vidioc-querycap.rst | 3 +
>>>> .../media/v4l/vidioc-queryctrl.rst | 11 +-
>>>> .../media/videodev2.h.rst.exceptions | 3 +
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 4 +
>>>> drivers/media/platform/nxp/Kconfig | 13 +
>>>> drivers/media/platform/nxp/Makefile | 1 +
>>>> drivers/media/platform/nxp/imx-asrc.c | 1256 +++++++++++++++++
>>>> drivers/media/test-drivers/Kconfig | 10 +
>>>> drivers/media/test-drivers/Makefile | 1 +
>>>> drivers/media/test-drivers/vim2m-audio.c | 793 +++++++++++
>>>> drivers/media/test-drivers/vivid/vivid-core.h | 2 +
>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 26 +
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +
>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +-
>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 10 +
>>>> drivers/media/v4l2-core/v4l2-dev.c | 21 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 66 +
>>>> drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +-
>>>> include/media/v4l2-ctrls.h | 13 +-
>>>> include/media/v4l2-dev.h | 2 +
>>>> include/media/v4l2-ioctl.h | 34 +
>>>> .../fsl => include/sound}/fsl_asrc_common.h | 60 +
>>>> include/uapi/linux/media.h | 2 +
>>>> include/uapi/linux/v4l2-controls.h | 9 +
>>>> include/uapi/linux/videodev2.h | 50 +-
>>>> sound/soc/fsl/fsl_asrc.c | 144 ++
>>>> sound/soc/fsl/fsl_asrc.h | 4 +-
>>>> sound/soc/fsl/fsl_asrc_dma.c | 2 +-
>>>> sound/soc/fsl/fsl_easrc.c | 233 +++
>>>> sound/soc/fsl/fsl_easrc.h | 6 +-
>>>> 42 files changed, 3128 insertions(+), 27 deletions(-)
>>>> create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>>>> create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
>>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>>> create mode 100644 drivers/media/platform/nxp/imx-asrc.c
>>>> create mode 100644 drivers/media/test-drivers/vim2m-audio.c
>>>> rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%)
>>>>
>>>
>
Powered by blists - more mailing lists