[0/2] New V4L2 controls V4L2_CID_NOTIFY_GAIN_XXX

  Hi

I'd like to propose some new V4L2 controls as defined in the attached
patches. The controls are:

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

The purpose of these controls is to be able to notify a raw sensor
what colour gains will be applied by subsequent processing (such as by
an ISP). Equivalently we can think of them as telling the sensor what
the white point is. Note that the sensor is told these gains but does
not apply them - which the choice of name is trying to convey.

Some sensors need to know these numbers for the processing that they
perform. Here I'm thinking in particular of so-called "quad Bayer"
(also sometimes "tetracell") sensors that have a special read-out mode
that converts the non-standard Bayer pattern into a standard one, also
at full resolution.

Sensors of this type are becoming quite common on cell phones where,
for example, a 48MP sensor may be able to deliver multiple exposures
at 12MP (for HDR processing perhaps) but they may also have a mode as
described above where they can generate a standard Bayer output at
48MP. This processing works better - we might expect less colour
aliasing? - when the sensor knows what values correspond to "white".

One question in my mind is whether it's worth having a control for
each green channel. The sensor I'm currently looking at only wants a
single green gain, but perhaps it's one of those instances where it
would be annoying to put in a single gain and discover, sometime
later, a sensor that wants both. Opinions on the matter always
appreciated!

Thanks very much and best regards

David

David Plowman (2):
  media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAIN_XXX controls
  media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls.c          |  4 +++
 include/uapi/linux/v4l2-controls.h            |  4 +++
 3 files changed, 36 insertions(+)
  

Hi David,

On 17/05/2021 12:02, David Plowman wrote:
> Hi
> 
> I'd like to propose some new V4L2 controls as defined in the attached
> patches. The controls are:
> 
> V4L2_CID_NOTIFY_GAIN_RED
> V4L2_CID_NOTIFY_GAIN_GREENR
> V4L2_CID_NOTIFY_GAIN_BLUE
> V4L2_CID_NOTIFY_GAIN_GREENB
> 
> The purpose of these controls is to be able to notify a raw sensor
> what colour gains will be applied by subsequent processing (such as by
> an ISP). Equivalently we can think of them as telling the sensor what
> the white point is. Note that the sensor is told these gains but does
> not apply them - which the choice of name is trying to convey.
> 
> Some sensors need to know these numbers for the processing that they
> perform. Here I'm thinking in particular of so-called "quad Bayer"
> (also sometimes "tetracell") sensors that have a special read-out mode
> that converts the non-standard Bayer pattern into a standard one, also
> at full resolution.
> 
> Sensors of this type are becoming quite common on cell phones where,
> for example, a 48MP sensor may be able to deliver multiple exposures
> at 12MP (for HDR processing perhaps) but they may also have a mode as
> described above where they can generate a standard Bayer output at
> 48MP. This processing works better - we might expect less colour
> aliasing? - when the sensor knows what values correspond to "white".
> 
> One question in my mind is whether it's worth having a control for
> each green channel. The sensor I'm currently looking at only wants a
> single green gain, but perhaps it's one of those instances where it
> would be annoying to put in a single gain and discover, sometime
> later, a sensor that wants both. Opinions on the matter always
> appreciated!

I think I would like to see this series in combination with an actual
sensor driver + ISP that uses them.

I am a bit concerned about the fact that the units are sensor-specific,
so how would an ISP driver be able to set them in a sensor-independent
manner? It seems that this would require knowledge about the sensor
internals, which is something you want to avoid in a ISP driver.

Perhaps this should be a multiplication factor instead? I.e. if the
value is X, then the ISP calculated red as the original red value times
(X / 100). So if X = 212, then the red gain factor is 2.12.

It's probably best to have two controls for green IMHO.

Regards,

	Hans

> 
> Thanks very much and best regards
> 
> David
> 
> David Plowman (2):
>   media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAIN_XXX controls
>   media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls
> 
>  .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls.c          |  4 +++
>  include/uapi/linux/v4l2-controls.h            |  4 +++
>  3 files changed, 36 insertions(+)
>