[v11,00/11] media: Implement UVC v1.5 ROI

  From: Yunke Cao <yunkec@chromium.org>

Hi,

This patch set implements UVC v1.5 region of interest using V4L2
control API. I rebased v10 and resended.

ROI control is consisted two uvc specific controls.
1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
2. An auto control with type bitmask.

V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.

Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
and calling ioctls.

1/11 adds V4L2_CTRL_TYPE_RECT.
2/11, 3/11, 4/11 refactors uvc_ctrl.c.
5/11 adds support for compound controls.
6/11 is a cherry-pick for Hans' implementation of
V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
7/11 documents min/max for the rectangle control.
8/11 supports MIN/MAX in UVC.
9/11 implements ROI in UVC.
10/11 initializes ROI control to default value.
11/11 documents the changes.

Changelog since v9:
- Reordered patches, move MIN/MAX related patches (patch 6, 7, 8) before
  implementing ROI control in UVC. Clamping set current ROI value based on
  MIN/MAX values.
- Fixed some minor issues. Changelog in patches.
Changelog since v9:
- Rebased onto Linus' master branch.
- Fixed some minor issues. Changelog in patches.
Changelog since v9:
- Rebased onto Linus' master branch.
- Fixed some minor issues. Changelog in patches.
Changelog since v8:
- Rebased onto media-stage master (v6.1-rc2).
- Addressed Hans' comments in 8/11 v4l2-ctrls: add support for
  V4L2_CTRL_WHICH_MIN/MAX_VAL. Including adding a new documentation patch
  9/11.
Changelog since v7:
- Rebased onto media-stage master.
- Move the factoring logic from 5/10 to 2/10 and 3/10 (new patch in v8).
- In patch 4/10, split uvc_control_mapping.size to v4l2 and data size.
- Move initializing ROI control from 5/10 into its own patch 7/10.
- Address some comments. Changelogs are in patches.

Hans Verkuil (1):
  v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL

Yunke Cao (10):
  media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
  media: uvcvideo: add uvc_ctrl_get_boundary for getting default value
  media: uvcvideo: introduce __uvc_ctrl_get_std()
  media: uvcvideo: Split uvc_control_mapping.size to v4l2 and data size
  media: uvcvideo: Add support for compound controls
  media: v4l2: document the usage of min/max for V4L2_CTRL_TYPE_RECT
  media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
  media: uvcvideo: implement UVC v1.5 ROI
  media: uvcvideo: initilaize ROI control to default value
  media: uvcvideo: document UVC v1.5 ROI

 .../userspace-api/media/drivers/uvcvideo.rst  |  64 +-
 .../media/v4l/vidioc-g-ext-ctrls.rst          |  15 +-
 .../media/v4l/vidioc-queryctrl.rst            |  11 +
 .../media/videodev2.h.rst.exceptions          |   3 +
 drivers/media/i2c/imx214.c                    |   5 +-
 .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
 drivers/media/usb/uvc/uvc_ctrl.c              | 702 ++++++++++++++----
 drivers/media/usb/uvc/uvc_v4l2.c              |  18 +-
 drivers/media/usb/uvc/uvcvideo.h              |  24 +-
 drivers/media/v4l2-core/v4l2-ctrls-api.c      |  57 +-
 drivers/media/v4l2-core/v4l2-ctrls-core.c     | 171 ++++-
 drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
 include/media/v4l2-ctrls.h                    |  36 +-
 include/uapi/linux/usb/video.h                |   1 +
 include/uapi/linux/uvcvideo.h                 |  13 +
 include/uapi/linux/v4l2-controls.h            |   9 +
 include/uapi/linux/videodev2.h                |   4 +
 17 files changed, 942 insertions(+), 204 deletions(-)
  

Hi Hans and Laurent,

On Wed, Apr 26, 2023 at 5:29 PM Yunke Cao <yunkec@google.com> wrote:
>
> From: Yunke Cao <yunkec@chromium.org>
>
> Hi,
>
> This patch set implements UVC v1.5 region of interest using V4L2
> control API. I rebased v10 and resended.
>
> ROI control is consisted two uvc specific controls.
> 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> 2. An auto control with type bitmask.
>
> V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
>
> Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> and calling ioctls.
>
> 1/11 adds V4L2_CTRL_TYPE_RECT.
> 2/11, 3/11, 4/11 refactors uvc_ctrl.c.
> 5/11 adds support for compound controls.
> 6/11 is a cherry-pick for Hans' implementation of
> V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> 7/11 documents min/max for the rectangle control.
> 8/11 supports MIN/MAX in UVC.
> 9/11 implements ROI in UVC.
> 10/11 initializes ROI control to default value.
> 11/11 documents the changes.
>

Would you have some time to take a look at this series, please? Thanks
in advance!

I think we need Hans's acks for 1, 6 and 7, since they modify the
control framework and then from Laurent for the uvc driver parts.

Best regards,
Tomasz

> Changelog since v9:
> - Reordered patches, move MIN/MAX related patches (patch 6, 7, 8) before
>   implementing ROI control in UVC. Clamping set current ROI value based on
>   MIN/MAX values.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v9:
> - Rebased onto Linus' master branch.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v9:
> - Rebased onto Linus' master branch.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v8:
> - Rebased onto media-stage master (v6.1-rc2).
> - Addressed Hans' comments in 8/11 v4l2-ctrls: add support for
>   V4L2_CTRL_WHICH_MIN/MAX_VAL. Including adding a new documentation patch
>   9/11.
> Changelog since v7:
> - Rebased onto media-stage master.
> - Move the factoring logic from 5/10 to 2/10 and 3/10 (new patch in v8).
> - In patch 4/10, split uvc_control_mapping.size to v4l2 and data size.
> - Move initializing ROI control from 5/10 into its own patch 7/10.
> - Address some comments. Changelogs are in patches.
>
> Hans Verkuil (1):
>   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
>
> Yunke Cao (10):
>   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
>   media: uvcvideo: add uvc_ctrl_get_boundary for getting default value
>   media: uvcvideo: introduce __uvc_ctrl_get_std()
>   media: uvcvideo: Split uvc_control_mapping.size to v4l2 and data size
>   media: uvcvideo: Add support for compound controls
>   media: v4l2: document the usage of min/max for V4L2_CTRL_TYPE_RECT
>   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
>   media: uvcvideo: implement UVC v1.5 ROI
>   media: uvcvideo: initilaize ROI control to default value
>   media: uvcvideo: document UVC v1.5 ROI
>
>  .../userspace-api/media/drivers/uvcvideo.rst  |  64 +-
>  .../media/v4l/vidioc-g-ext-ctrls.rst          |  15 +-
>  .../media/v4l/vidioc-queryctrl.rst            |  11 +
>  .../media/videodev2.h.rst.exceptions          |   3 +
>  drivers/media/i2c/imx214.c                    |   5 +-
>  .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
>  drivers/media/usb/uvc/uvc_ctrl.c              | 702 ++++++++++++++----
>  drivers/media/usb/uvc/uvc_v4l2.c              |  18 +-
>  drivers/media/usb/uvc/uvcvideo.h              |  24 +-
>  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  57 +-
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 171 ++++-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
>  include/media/v4l2-ctrls.h                    |  36 +-
>  include/uapi/linux/usb/video.h                |   1 +
>  include/uapi/linux/uvcvideo.h                 |  13 +
>  include/uapi/linux/v4l2-controls.h            |   9 +
>  include/uapi/linux/videodev2.h                |   4 +
>  17 files changed, 942 insertions(+), 204 deletions(-)
>
> --
> 2.40.1.495.gc816e09b53d-goog
>
  

On (23/04/26 17:29), Yunke Cao wrote:
> Hi,
> 
> This patch set implements UVC v1.5 region of interest using V4L2
> control API. I rebased v10 and resended.
> 
> ROI control is consisted two uvc specific controls.
> 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> 2. An auto control with type bitmask.
> 
> V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
> 
> Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> and calling ioctls.
> 
> 1/11 adds V4L2_CTRL_TYPE_RECT.
> 2/11, 3/11, 4/11 refactors uvc_ctrl.c.
> 5/11 adds support for compound controls.
> 6/11 is a cherry-pick for Hans' implementation of
> V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> 7/11 documents min/max for the rectangle control.
> 8/11 supports MIN/MAX in UVC.
> 9/11 implements ROI in UVC.
> 10/11 initializes ROI control to default value.
> 11/11 documents the changes.

Hello folks,

Can we please get some reviews/feedback on this?
  

On 26/04/2023 10:29, Yunke Cao wrote:
> From: Yunke Cao <yunkec@chromium.org>
> 
> Hi,
> 
> This patch set implements UVC v1.5 region of interest using V4L2
> control API. I rebased v10 and resended.
> 
> ROI control is consisted two uvc specific controls.
> 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> 2. An auto control with type bitmask.
> 
> V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
> 
> Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> and calling ioctls.
> 
> 1/11 adds V4L2_CTRL_TYPE_RECT.
> 2/11, 3/11, 4/11 refactors uvc_ctrl.c.
> 5/11 adds support for compound controls.
> 6/11 is a cherry-pick for Hans' implementation of
> V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> 7/11 documents min/max for the rectangle control.
> 8/11 supports MIN/MAX in UVC.
> 9/11 implements ROI in UVC.
> 10/11 initializes ROI control to default value.
> 11/11 documents the changes.

I just looked at the core control code, and except for some minor issues
it looks good.

One thing needs to be added here, though: I want to have a patch adding
a rectangle control + min/max support to the vivid driver, similar to
the existing VIVID_CID_AREA control. This makes it possible to do regression
tests without requiring hardware that supports this.

v4l2-ctl/compliance also need to be adapted for min/max.

Regards,

	Hans

> 
> Changelog since v9:
> - Reordered patches, move MIN/MAX related patches (patch 6, 7, 8) before
>   implementing ROI control in UVC. Clamping set current ROI value based on
>   MIN/MAX values.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v9:
> - Rebased onto Linus' master branch.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v9:
> - Rebased onto Linus' master branch.
> - Fixed some minor issues. Changelog in patches.
> Changelog since v8:
> - Rebased onto media-stage master (v6.1-rc2).
> - Addressed Hans' comments in 8/11 v4l2-ctrls: add support for
>   V4L2_CTRL_WHICH_MIN/MAX_VAL. Including adding a new documentation patch
>   9/11.
> Changelog since v7:
> - Rebased onto media-stage master.
> - Move the factoring logic from 5/10 to 2/10 and 3/10 (new patch in v8).
> - In patch 4/10, split uvc_control_mapping.size to v4l2 and data size.
> - Move initializing ROI control from 5/10 into its own patch 7/10.
> - Address some comments. Changelogs are in patches.
> 
> Hans Verkuil (1):
>   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
> 
> Yunke Cao (10):
>   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
>   media: uvcvideo: add uvc_ctrl_get_boundary for getting default value
>   media: uvcvideo: introduce __uvc_ctrl_get_std()
>   media: uvcvideo: Split uvc_control_mapping.size to v4l2 and data size
>   media: uvcvideo: Add support for compound controls
>   media: v4l2: document the usage of min/max for V4L2_CTRL_TYPE_RECT
>   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
>   media: uvcvideo: implement UVC v1.5 ROI
>   media: uvcvideo: initilaize ROI control to default value
>   media: uvcvideo: document UVC v1.5 ROI
> 
>  .../userspace-api/media/drivers/uvcvideo.rst  |  64 +-
>  .../media/v4l/vidioc-g-ext-ctrls.rst          |  15 +-
>  .../media/v4l/vidioc-queryctrl.rst            |  11 +
>  .../media/videodev2.h.rst.exceptions          |   3 +
>  drivers/media/i2c/imx214.c                    |   5 +-
>  .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
>  drivers/media/usb/uvc/uvc_ctrl.c              | 702 ++++++++++++++----
>  drivers/media/usb/uvc/uvc_v4l2.c              |  18 +-
>  drivers/media/usb/uvc/uvcvideo.h              |  24 +-
>  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  57 +-
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 171 ++++-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
>  include/media/v4l2-ctrls.h                    |  36 +-
>  include/uapi/linux/usb/video.h                |   1 +
>  include/uapi/linux/uvcvideo.h                 |  13 +
>  include/uapi/linux/v4l2-controls.h            |   9 +
>  include/uapi/linux/videodev2.h                |   4 +
>  17 files changed, 942 insertions(+), 204 deletions(-)
>
  

Hi Hans,

Thanks for the review!!

On Wed, Aug 9, 2023 at 4:19 PM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
>
> On 26/04/2023 10:29, Yunke Cao wrote:
> > From: Yunke Cao <yunkec@chromium.org>
> >
> > Hi,
> >
> > This patch set implements UVC v1.5 region of interest using V4L2
> > control API. I rebased v10 and resended.
> >
> > ROI control is consisted two uvc specific controls.
> > 1. A rectangle control with a newly added type V4L2_CTRL_TYPE_RECT.
> > 2. An auto control with type bitmask.
> >
> > V4L2_CTRL_WHICH_MIN/MAX_VAL is added to support the rectangle control.
> >
> > Tested on two different usb cameras using v4l2-compliance, v4l2-ctl
> > and calling ioctls.
> >
> > 1/11 adds V4L2_CTRL_TYPE_RECT.
> > 2/11, 3/11, 4/11 refactors uvc_ctrl.c.
> > 5/11 adds support for compound controls.
> > 6/11 is a cherry-pick for Hans' implementation of
> > V4L2_CTRL_WHICH_MIN/MAX_VAL in v4l2-core.
> > 7/11 documents min/max for the rectangle control.
> > 8/11 supports MIN/MAX in UVC.
> > 9/11 implements ROI in UVC.
> > 10/11 initializes ROI control to default value.
> > 11/11 documents the changes.
>
> I just looked at the core control code, and except for some minor issues
> it looks good.
>
> One thing needs to be added here, though: I want to have a patch adding
> a rectangle control + min/max support to the vivid driver, similar to
> the existing VIVID_CID_AREA control. This makes it possible to do regression
> tests without requiring hardware that supports this.

Sounds good, I can add a patch with VIVID_CID_RECT control in the next version.

>
> v4l2-ctl/compliance also need to be adapted for min/max.

This patchset doesn't break v4l2-compliance for the cameras I tested with.
I can try to come up with something to verify min/max .

Laurent, Daniel,
Do you have time to look at the UVC patches?

Best,
Yunke

>
> Regards,
>
>         Hans
>
> >
> > Changelog since v9:
> > - Reordered patches, move MIN/MAX related patches (patch 6, 7, 8) before
> >   implementing ROI control in UVC. Clamping set current ROI value based on
> >   MIN/MAX values.
> > - Fixed some minor issues. Changelog in patches.
> > Changelog since v9:
> > - Rebased onto Linus' master branch.
> > - Fixed some minor issues. Changelog in patches.
> > Changelog since v9:
> > - Rebased onto Linus' master branch.
> > - Fixed some minor issues. Changelog in patches.
> > Changelog since v8:
> > - Rebased onto media-stage master (v6.1-rc2).
> > - Addressed Hans' comments in 8/11 v4l2-ctrls: add support for
> >   V4L2_CTRL_WHICH_MIN/MAX_VAL. Including adding a new documentation patch
> >   9/11.
> > Changelog since v7:
> > - Rebased onto media-stage master.
> > - Move the factoring logic from 5/10 to 2/10 and 3/10 (new patch in v8).
> > - In patch 4/10, split uvc_control_mapping.size to v4l2 and data size.
> > - Move initializing ROI control from 5/10 into its own patch 7/10.
> > - Address some comments. Changelogs are in patches.
> >
> > Hans Verkuil (1):
> >   v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
> >
> > Yunke Cao (10):
> >   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
> >   media: uvcvideo: add uvc_ctrl_get_boundary for getting default value
> >   media: uvcvideo: introduce __uvc_ctrl_get_std()
> >   media: uvcvideo: Split uvc_control_mapping.size to v4l2 and data size
> >   media: uvcvideo: Add support for compound controls
> >   media: v4l2: document the usage of min/max for V4L2_CTRL_TYPE_RECT
> >   media: uvcvideo: support V4L2_CTRL_WHICH_MIN/MAX_VAL
> >   media: uvcvideo: implement UVC v1.5 ROI
> >   media: uvcvideo: initilaize ROI control to default value
> >   media: uvcvideo: document UVC v1.5 ROI
> >
> >  .../userspace-api/media/drivers/uvcvideo.rst  |  64 +-
> >  .../media/v4l/vidioc-g-ext-ctrls.rst          |  15 +-
> >  .../media/v4l/vidioc-queryctrl.rst            |  11 +
> >  .../media/videodev2.h.rst.exceptions          |   3 +
> >  drivers/media/i2c/imx214.c                    |   5 +-
> >  .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
> >  drivers/media/usb/uvc/uvc_ctrl.c              | 702 ++++++++++++++----
> >  drivers/media/usb/uvc/uvc_v4l2.c              |  18 +-
> >  drivers/media/usb/uvc/uvcvideo.h              |  24 +-
> >  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  57 +-
> >  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 171 ++++-
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
> >  include/media/v4l2-ctrls.h                    |  36 +-
> >  include/uapi/linux/usb/video.h                |   1 +
> >  include/uapi/linux/uvcvideo.h                 |  13 +
> >  include/uapi/linux/v4l2-controls.h            |   9 +
> >  include/uapi/linux/videodev2.h                |   4 +
> >  17 files changed, 942 insertions(+), 204 deletions(-)
> >
>