[v7,0/7] media: Implement UVC v1.5 ROI

  This patch set implements UVC v1.5 region of interest using V4L2
control API.

ROI control is consisted two uvc specific controls.
1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
2. An auto control with type bitmask.

V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.

Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
and calling ioctls.

1/7 add V4L2_CTRL_TYPE_RECT.
2/7 and 3/7 support compound types in UVC.
4/7 implement ROI in UVC.
5/7 is a cherry-pick for Hans' implementation of
V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
6/7 support MIN/MAX in UVC.
7/7 document the changes.

Changelog since v6:
-Add patch 2 and 3 to support compound types properly in UVC and
implement ROI on top of them.
-Reorder the patches.

Changelog since v5:
-Add a __uvc_ctrl_get_p_rect_to_user instead of modifying
 __uvc_ctrl_get.
-Support V4L2_CTRL_FLAG_NEXT_COMPOUND correctly.
-Fix formats.

Changelog since v4:
-Cherry-pick the original patch
 "v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL".
-Split patch "media: uvcvideo: implement UVC v1.5 ROI" into two patches.
 The codes for supporting min/max in uvc are in patch 4/5 now.
-Minor fixes. Detailed changelog in patches

Changelog since v3:
- Reordered/sliced the patches.
  1. Add rect type.
  2. Add min/max.
  3. Add the roi controls (including init to default).
  4. Document the roi controls.
- Define the roi controls as uvc-specific in uvcvideo.h.
- Modified documentation.
- Removed the vivid change. Given the controls are now uvc-specific.
  I'm not sure how valuable it is to add it in vivid. Let me know
  otherwise.

Hans Verkuil (1):
  v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL

Yunke Cao (6):
  media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
  media: uvcvideo: add uvc_ctrl_get_fixed for getting default value
  media: uvcvideo: Add support for compound controls
  media: uvcvideo: implement UVC v1.5 ROI
  media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
  media: uvcvideo: document UVC v1.5 ROI

 .../userspace-api/media/drivers/uvcvideo.rst  |  61 +++
 .../media/v4l/vidioc-g-ext-ctrls.rst          |  11 +-
 .../media/videodev2.h.rst.exceptions          |   3 +
 drivers/media/i2c/imx214.c                    |   5 +-
 .../media/platform/qcom/venus/venc_ctrls.c    |   4 +
 drivers/media/usb/uvc/uvc_ctrl.c              | 479 ++++++++++++++++--
 drivers/media/usb/uvc/uvc_v4l2.c              |  20 +-
 drivers/media/usb/uvc/uvcvideo.h              |  14 +
 drivers/media/v4l2-core/v4l2-ctrls-api.c      |  51 +-
 drivers/media/v4l2-core/v4l2-ctrls-core.c     | 155 +++++-
 drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
 include/media/v4l2-ctrls.h                    |  34 +-
 include/uapi/linux/usb/video.h                |   1 +
 include/uapi/linux/uvcvideo.h                 |  13 +
 include/uapi/linux/v4l2-controls.h            |   8 +
 include/uapi/linux/videodev2.h                |   4 +
 16 files changed, 788 insertions(+), 79 deletions(-)
  

Hi Laurent,

Do you have time to review this version? Ricardo has already reviewed
it, I hope it is easier to review now.

Thanks in advance!
Yunke



On Tue, Jun 28, 2022 at 4:57 PM Yunke Cao <yunkec@google.com> wrote:
>
> This patch set implements UVC v1.5 region of interest using V4L2
> control API.
>
> ROI control is consisted two uvc specific controls.
> 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> 2. An auto control with type bitmask.
>
> V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
>
> Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> and calling ioctls.
>
> 1/7 add V4L2_CTRL_TYPE_RECT.
> 2/7 and 3/7 support compound types in UVC.
> 4/7 implement ROI in UVC.
> 5/7 is a cherry-pick for Hans' implementation of
> V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> 6/7 support MIN/MAX in UVC.
> 7/7 document the changes.
>
> Changelog since v6:
> -Add patch 2 and 3 to support compound types properly in UVC and
> implement ROI on top of them.
> -Reorder the patches.
>
> Changelog since v5:
> -Add a __uvc_ctrl_get_p_rect_to_user instead of modifying
>  __uvc_ctrl_get.
> -Support V4L2_CTRL_FLAG_NEXT_COMPOUND correctly.
> -Fix formats.
>
> Changelog since v4:
> -Cherry-pick the original patch
>  "v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL".
> -Split patch "media: uvcvideo: implement UVC v1.5 ROI" into two patches.
>  The codes for supporting min/max in uvc are in patch 4/5 now.
> -Minor fixes. Detailed changelog in patches
>
> Changelog since v3:
> - Reordered/sliced the patches.
>   1. Add rect type.
>   2. Add min/max.
>   3. Add the roi controls (including init to default).
>   4. Document the roi controls.
> - Define the roi controls as uvc-specific in uvcvideo.h.
> - Modified documentation.
> - Removed the vivid change. Given the controls are now uvc-specific.
>   I'm not sure how valuable it is to add it in vivid. Let me know
>   otherwise.
>
> Hans Verkuil (1):
>   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
>
> Yunke Cao (6):
>   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
>   media: uvcvideo: add uvc_ctrl_get_fixed for getting default value
>   media: uvcvideo: Add support for compound controls
>   media: uvcvideo: implement UVC v1.5 ROI
>   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
>   media: uvcvideo: document UVC v1.5 ROI
>
>  .../userspace-api/media/drivers/uvcvideo.rst  |  61 +++
>  .../media/v4l/vidioc-g-ext-ctrls.rst          |  11 +-
>  .../media/videodev2.h.rst.exceptions          |   3 +
>  drivers/media/i2c/imx214.c                    |   5 +-
>  .../media/platform/qcom/venus/venc_ctrls.c    |   4 +
>  drivers/media/usb/uvc/uvc_ctrl.c              | 479 ++++++++++++++++--
>  drivers/media/usb/uvc/uvc_v4l2.c              |  20 +-
>  drivers/media/usb/uvc/uvcvideo.h              |  14 +
>  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  51 +-
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 155 +++++-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
>  include/media/v4l2-ctrls.h                    |  34 +-
>  include/uapi/linux/usb/video.h                |   1 +
>  include/uapi/linux/uvcvideo.h                 |  13 +
>  include/uapi/linux/v4l2-controls.h            |   8 +
>  include/uapi/linux/videodev2.h                |   4 +
>  16 files changed, 788 insertions(+), 79 deletions(-)
>
> --
> 2.37.0.rc0.161.g10f37bed90-goog
>
  

Hi Yunke,

On Fri, Jul 15, 2022 at 08:25:20AM +0900, Yunke Cao wrote:
> Hi Laurent,
> 
> Do you have time to review this version? Ricardo has already reviewed
> it, I hope it is easier to review now.

I'll try find time, but I doubt it will be before a couple of weeks.

> On Tue, Jun 28, 2022 at 4:57 PM Yunke Cao <yunkec@google.com> wrote:
> >
> > This patch set implements UVC v1.5 region of interest using V4L2
> > control API.
> >
> > ROI control is consisted two uvc specific controls.
> > 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> > 2. An auto control with type bitmask.
> >
> > V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
> >
> > Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> > and calling ioctls.
> >
> > 1/7 add V4L2_CTRL_TYPE_RECT.
> > 2/7 and 3/7 support compound types in UVC.
> > 4/7 implement ROI in UVC.
> > 5/7 is a cherry-pick for Hans' implementation of
> > V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> > 6/7 support MIN/MAX in UVC.
> > 7/7 document the changes.
> >
> > Changelog since v6:
> > -Add patch 2 and 3 to support compound types properly in UVC and
> > implement ROI on top of them.
> > -Reorder the patches.
> >
> > Changelog since v5:
> > -Add a __uvc_ctrl_get_p_rect_to_user instead of modifying
> >  __uvc_ctrl_get.
> > -Support V4L2_CTRL_FLAG_NEXT_COMPOUND correctly.
> > -Fix formats.
> >
> > Changelog since v4:
> > -Cherry-pick the original patch
> >  "v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL".
> > -Split patch "media: uvcvideo: implement UVC v1.5 ROI" into two patches.
> >  The codes for supporting min/max in uvc are in patch 4/5 now.
> > -Minor fixes. Detailed changelog in patches
> >
> > Changelog since v3:
> > - Reordered/sliced the patches.
> >   1. Add rect type.
> >   2. Add min/max.
> >   3. Add the roi controls (including init to default).
> >   4. Document the roi controls.
> > - Define the roi controls as uvc-specific in uvcvideo.h.
> > - Modified documentation.
> > - Removed the vivid change. Given the controls are now uvc-specific.
> >   I'm not sure how valuable it is to add it in vivid. Let me know
> >   otherwise.
> >
> > Hans Verkuil (1):
> >   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
> >
> > Yunke Cao (6):
> >   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
> >   media: uvcvideo: add uvc_ctrl_get_fixed for getting default value
> >   media: uvcvideo: Add support for compound controls
> >   media: uvcvideo: implement UVC v1.5 ROI
> >   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
> >   media: uvcvideo: document UVC v1.5 ROI
> >
> >  .../userspace-api/media/drivers/uvcvideo.rst  |  61 +++
> >  .../media/v4l/vidioc-g-ext-ctrls.rst          |  11 +-
> >  .../media/videodev2.h.rst.exceptions          |   3 +
> >  drivers/media/i2c/imx214.c                    |   5 +-
> >  .../media/platform/qcom/venus/venc_ctrls.c    |   4 +
> >  drivers/media/usb/uvc/uvc_ctrl.c              | 479 ++++++++++++++++--
> >  drivers/media/usb/uvc/uvc_v4l2.c              |  20 +-
> >  drivers/media/usb/uvc/uvcvideo.h              |  14 +
> >  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  51 +-
> >  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 155 +++++-
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
> >  include/media/v4l2-ctrls.h                    |  34 +-
> >  include/uapi/linux/usb/video.h                |   1 +
> >  include/uapi/linux/uvcvideo.h                 |  13 +
> >  include/uapi/linux/v4l2-controls.h            |   8 +
> >  include/uapi/linux/videodev2.h                |   4 +
> >  16 files changed, 788 insertions(+), 79 deletions(-)
  

Hi Laurent,

Thanks for the review! I've been working on v8 based on your feedback.

I have some questions about your comments and replied in some of the patches.
Would you mind taking another look?

Best,
Yunke

On Fri, Jul 15, 2022 at 9:52 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Yunke,
>
> On Fri, Jul 15, 2022 at 08:25:20AM +0900, Yunke Cao wrote:
> > Hi Laurent,
> >
> > Do you have time to review this version? Ricardo has already reviewed
> > it, I hope it is easier to review now.
>
> I'll try find time, but I doubt it will be before a couple of weeks.
>
> > On Tue, Jun 28, 2022 at 4:57 PM Yunke Cao <yunkec@google.com> wrote:
> > >
> > > This patch set implements UVC v1.5 region of interest using V4L2
> > > control API.
> > >
> > > ROI control is consisted two uvc specific controls.
> > > 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> > > 2. An auto control with type bitmask.
> > >
> > > V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
> > >
> > > Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> > > and calling ioctls.
> > >
> > > 1/7 add V4L2_CTRL_TYPE_RECT.
> > > 2/7 and 3/7 support compound types in UVC.
> > > 4/7 implement ROI in UVC.
> > > 5/7 is a cherry-pick for Hans' implementation of
> > > V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> > > 6/7 support MIN/MAX in UVC.
> > > 7/7 document the changes.
> > >
> > > Changelog since v6:
> > > -Add patch 2 and 3 to support compound types properly in UVC and
> > > implement ROI on top of them.
> > > -Reorder the patches.
> > >
> > > Changelog since v5:
> > > -Add a __uvc_ctrl_get_p_rect_to_user instead of modifying
> > >  __uvc_ctrl_get.
> > > -Support V4L2_CTRL_FLAG_NEXT_COMPOUND correctly.
> > > -Fix formats.
> > >
> > > Changelog since v4:
> > > -Cherry-pick the original patch
> > >  "v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL".
> > > -Split patch "media: uvcvideo: implement UVC v1.5 ROI" into two patches.
> > >  The codes for supporting min/max in uvc are in patch 4/5 now.
> > > -Minor fixes. Detailed changelog in patches
> > >
> > > Changelog since v3:
> > > - Reordered/sliced the patches.
> > >   1. Add rect type.
> > >   2. Add min/max.
> > >   3. Add the roi controls (including init to default).
> > >   4. Document the roi controls.
> > > - Define the roi controls as uvc-specific in uvcvideo.h.
> > > - Modified documentation.
> > > - Removed the vivid change. Given the controls are now uvc-specific.
> > >   I'm not sure how valuable it is to add it in vivid. Let me know
> > >   otherwise.
> > >
> > > Hans Verkuil (1):
> > >   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
> > >
> > > Yunke Cao (6):
> > >   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
> > >   media: uvcvideo: add uvc_ctrl_get_fixed for getting default value
> > >   media: uvcvideo: Add support for compound controls
> > >   media: uvcvideo: implement UVC v1.5 ROI
> > >   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
> > >   media: uvcvideo: document UVC v1.5 ROI
> > >
> > >  .../userspace-api/media/drivers/uvcvideo.rst  |  61 +++
> > >  .../media/v4l/vidioc-g-ext-ctrls.rst          |  11 +-
> > >  .../media/videodev2.h.rst.exceptions          |   3 +
> > >  drivers/media/i2c/imx214.c                    |   5 +-
> > >  .../media/platform/qcom/venus/venc_ctrls.c    |   4 +
> > >  drivers/media/usb/uvc/uvc_ctrl.c              | 479 ++++++++++++++++--
> > >  drivers/media/usb/uvc/uvc_v4l2.c              |  20 +-
> > >  drivers/media/usb/uvc/uvcvideo.h              |  14 +
> > >  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  51 +-
> > >  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 155 +++++-
> > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
> > >  include/media/v4l2-ctrls.h                    |  34 +-
> > >  include/uapi/linux/usb/video.h                |   1 +
> > >  include/uapi/linux/uvcvideo.h                 |  13 +
> > >  include/uapi/linux/v4l2-controls.h            |   8 +
> > >  include/uapi/linux/videodev2.h                |   4 +
> > >  16 files changed, 788 insertions(+), 79 deletions(-)
>
> --
> Regards,
>
> Laurent Pinchart