[RFC PATCH v2 0/7] Add audio support in v4l2 framework

[Shengjiu Wang]

On Wed, Aug 2, 2023 at 8:08 PM Takashi Iwai <tiwai at suse.de> wrote:
>
> On Wed, 02 Aug 2023 14:02:29 +0200,
> Shengjiu Wang wrote:
> >
> > On Wed, Aug 2, 2023 at 7:22 PM Takashi Iwai <tiwai at suse.de> wrote:
> > >
> > > On Wed, 02 Aug 2023 09:32:37 +0200,
> > > Hans Verkuil wrote:
> > > >
> > > > Hi all,
> > > >
> > > > On 25/07/2023 08:12, Shengjiu Wang wrote:
> > > > > Audio signal processing has the requirement for memory to
> > > > > memory similar as Video.
> > > > >
> > > > > This patch is to add this support in v4l2 framework, defined
> > > > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > > > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > > > > for audio case usage.
> > > > >
> > > > > The created audio device is named "/dev/audioX".
> > > > >
> > > > > And add memory to memory support for two kinds of i.MX ASRC
> > > > > module
> > > >
> > > > Before I spend time on this: are the audio maintainers OK with doing
> > > > this in V4L2?
> > > >
> > > > I do want to have a clear statement on this as it is not something I
> > > > can decide.
> > >
> > > Well, I personally don't mind to have some audio capability in v4l2
> > > layer.  But, the only uncertain thing for now is whether this is a
> > > must-have or not.
> > >
> >
> > Thanks,  I am also not sure about this.  I am also confused that why
> > there is no m2m implementation for audio in the kernel.  Audio also
> > has similar decoder encoder post-processing as video.
> >
> > >
> > > IIRC, the implementation in the sound driver side was never done just
> > > because there was no similar implementation?  If so, and if the
> > > extension to the v4l2 core layer is needed, shouldn't it be more
> > > considered for the possible other route?
> > >
> >
> > Actually I'd like someone could point me to the other route. I'd like to
> > try.
> >
> > The reason why I select to extend v4l2 for such audio usage is that v4l2
> > looks best for this audio m2m implementation.  v4l2 is designed for m2m
> > usage.  if we need implement another 'route',  I don't think it can do better
> > that v4l2.
> >
> > I appreciate that someone can share his ideas or doable solutions.
> > And please don't ignore my request, ignore my patch.
>
> Can you explain a bit more details of your demand?
> At least, a "big picture" showing how your hardware is implemented and
> what is exactly necessary would be helpful for understanding the
> problem.
>

We have the hardware IP: ASRC,  asynchronous sample rate converter.

Currently the ASRC in ALSA is to connect to another I2S device as
a sound card.  But we'd like to the ASRC can be used by user space directly
that user space application can get the output after conversion from ASRC.

The ASRC can be integrated into a multimedia framework (gstreamer) as a plugin.

best regards
wang shengjiu