linux-kernel - Re: [PATCH v2] media: uvcvideo: Implement V4L2_EVENT_FRAME

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 7 Nov 2023 11:58:00 +0100
From:   Ricardo Ribalda <ribalda@...omium.org>
To:     Sakari Ailus <sakari.ailus@....fi>,
        Hans Verkuil <hverkuil@...all.nl>
Cc:     Esker Wong <esker@...omium.org>,
        Laurent Pinchart <laurent.pinchart@...asonboard.com>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        Tomasz Figa <tfiga@...gle.com>
Subject: Re: [PATCH v2] media: uvcvideo: Implement V4L2_EVENT_FRAME_SYNC

Hi Sakari

On Tue, 7 Nov 2023 at 08:37, Sakari Ailus <sakari.ailus@....fi> wrote:
>
> Hi Esker,
>
> On Tue, Nov 07, 2023 at 01:06:20PM +0800, Esker Wong wrote:
> > [send again in text mode]
> > Hi Sakari,
> >
> > Sequence number is important to us. We need it to measure the latency
> > from this event to the time we display the frame.
>
> I think we could also add the sequence number to V4L2_EVENT_VSYNC.
>
> Cc Hans.
I think you forgot to add him :)

@Hans

What do you prefer?

- Use V4L2_EVENT_FRAME_SYNC because we trigger it as soon as the
driver knows about the frame
- Use V4L2_EVENT_VSYNC, but extending it to support the frame sequence
- Use a new EVENT

Thanks!


>
> > On Mon, Nov 6, 2023 at 7:06 PM Sakari Ailus <sakari.ailus@....fi> wrote:
> > >
> > > Hi Ricardo,
> > >
> > > On Mon, Nov 06, 2023 at 10:52:27AM +0000, Ricardo Ribalda wrote:
> > > > Add support for the frame_sync event, so user-space can become aware
> > > > earlier of new frames.
> > > >
> > > > Suggested-by: Esker Wong <esker@...omium.org>
> > > > Tested-by: Esker Wong <esker@...omium.org>
> > > > Signed-off-by: Ricardo Ribalda <ribalda@...omium.org>
> > > > ---
> > > > We have measured a latency of around 30msecs between frame sync
> > > > and dqbuf.
> > > > ---
> > > > Changes in v2:
> > > > - Suggested by Laurent. Split sequence++ and event init.
> > > > - Link to v1: https://lore.kernel.org/r/20231020-uvc-event-v1-1-3baa0e9f6952@chromium.org
> > > > ---
> > > >  drivers/media/usb/uvc/uvc_v4l2.c  | 2 ++
> > > >  drivers/media/usb/uvc/uvc_video.c | 7 +++++++
> > > >  2 files changed, 9 insertions(+)
> > > >
> > > > diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
> > > > index f4988f03640a..9f3fb5fd2375 100644
> > > > --- a/drivers/media/usb/uvc/uvc_v4l2.c
> > > > +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> > > > @@ -1352,6 +1352,8 @@ static int uvc_ioctl_subscribe_event(struct v4l2_fh *fh,
> > > >       switch (sub->type) {
> > > >       case V4L2_EVENT_CTRL:
> > > >               return v4l2_event_subscribe(fh, sub, 0, &uvc_ctrl_sub_ev_ops);
> > > > +     case V4L2_EVENT_FRAME_SYNC:
> > > > +             return v4l2_event_subscribe(fh, sub, 0, NULL);
> > > >       default:
> > > >               return -EINVAL;
> > > >       }
> > > > diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
> > > > index 28dde08ec6c5..4f3a510ca4fe 100644
> > > > --- a/drivers/media/usb/uvc/uvc_video.c
> > > > +++ b/drivers/media/usb/uvc/uvc_video.c
> > > > @@ -1073,9 +1073,16 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
> > > >        * that discontinuous sequence numbers always indicate lost frames.
> > > >        */
> > > >       if (stream->last_fid != fid) {
> > > > +             struct v4l2_event event = {
> > > > +                     .type = V4L2_EVENT_FRAME_SYNC,
> > > > +             };
> > > > +
> > > >               stream->sequence++;
> > > >               if (stream->sequence)
> > > >                       uvc_video_stats_update(stream);
> > > > +
> > > > +             event.u.frame_sync.frame_sequence = stream->sequence,
> > > > +             v4l2_event_queue(&stream->vdev, &event);
> > >
> > > uvc_video_decode_start() is called when the reception of the entire frame
> > > has been completed. However, the documentation for V4L2_EVENT_FRAME_SYNC
> > > says that the event is "Triggered immediately when the reception of a frame
> > > has begun.". The functionality here doesn't seem to fit to this patch.
> > >
> > > Wouldn't V4L2_EVENT_VSYNC be a better fit, even if we don't really have a
> > > concept of vertical sync in the case of USB? That event doesn't have the
> > > sequence though but I guess it's not an issue at least if your case.
> > >
> > > Another technically correct option could be to create a new event for this
> > > but I'm not sure it's worth it.
> > >
> > > >       }
> > > >
> > > >       uvc_video_clock_decode(stream, buf, data, len);
> > > >
> > >
> > > --
> > > Regards,
> > >
> > > Sakari Ailus
>
> --
> Regards,
>
> Sakari Ailus



-- 
Ricardo Ribalda