mbox series

[v1,0/5] Adds media driver for Sony IMX492 sensor

Message ID 20221125153120.541298-1-petko.manolov@konsulko.com (mailing list archive)
Headers
Series Adds media driver for Sony IMX492 sensor |

Message

Petko Manolov Nov. 25, 2022, 3:31 p.m. UTC 
  This patch adds dt-bindings documentation, driver source, etc for Sony imx492
image sensor.

imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
digital output makes it possible to output the signals with high definition for
moving pictures. It is programmable through I2C interface. The I2C client
address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
through MIPI CSI-2.

Petko Manolov (5):
  DT bindings for imx492
  media: i2c: imx492: driver's header
  media: i2c: imx492: driver's source
  media: i2c: add imx492 config entry
  media: i2c: add imx492 entry for make

 .../bindings/media/i2c/sony,imx492.yaml       |  101 ++
 drivers/media/i2c/Kconfig                     |   14 +
 drivers/media/i2c/Makefile                    |    1 +
 drivers/media/i2c/imx492.c                    | 1092 +++++++++++++++++
 drivers/media/i2c/imx492.h                    |  555 +++++++++
 5 files changed, 1763 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx492.yaml
 create mode 100644 drivers/media/i2c/imx492.c
 create mode 100644 drivers/media/i2c/imx492.h

Comments

Kieran Bingham Nov. 28, 2022, 12:45 p.m. UTC | #1 
Hi Petko,

What device/ISP are you testing this sensor on?

Quoting Petko Manolov (2022-11-25 15:31:15)
> This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> image sensor.
> 
> imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> digital output makes it possible to output the signals with high definition for
> moving pictures. It is programmable through I2C interface. The I2C client
> address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> through MIPI CSI-2.
> 
> Petko Manolov (5):
>   DT bindings for imx492
>   media: i2c: imx492: driver's header
>   media: i2c: imx492: driver's source
>   media: i2c: add imx492 config entry
>   media: i2c: add imx492 entry for make

Could you squash patches 2, 3, 4, and 5 into a single patch please?

Also - I think you'll need an entry in MAINTAINERS.

I can't see any reference to the selection controls:
 - V4L2_SEL_TGT_CROP_DEFAULT
 - V4L2_SEL_TGT_CROP
 - V4L2_SEL_TGT_CROP_BOUNDS

Do you have the datasheet or sufficient information to be able to
implement these controls?

--
Kieran

>  .../bindings/media/i2c/sony,imx492.yaml       |  101 ++
>  drivers/media/i2c/Kconfig                     |   14 +
>  drivers/media/i2c/Makefile                    |    1 +
>  drivers/media/i2c/imx492.c                    | 1092 +++++++++++++++++
>  drivers/media/i2c/imx492.h                    |  555 +++++++++
>  5 files changed, 1763 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx492.yaml
>  create mode 100644 drivers/media/i2c/imx492.c
>  create mode 100644 drivers/media/i2c/imx492.h
> 
> -- 
> 2.30.2
>
Petko Manolov Nov. 28, 2022, 1:11 p.m. UTC | #2 
On 22-11-28 12:45:32, Kieran Bingham wrote:
> Hi Petko,
> 
> What device/ISP are you testing this sensor on?

I'll tell you, but only if you promise to not laugh too much.  Deal?

i.MX8M-Mini

I hope nobody will notice the above line...

> Quoting Petko Manolov (2022-11-25 15:31:15)
> > This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> > image sensor.
> > 
> > imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> > monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> > digital output makes it possible to output the signals with high definition for
> > moving pictures. It is programmable through I2C interface. The I2C client
> > address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> > through MIPI CSI-2.
> > 
> > Petko Manolov (5):
> >   DT bindings for imx492
> >   media: i2c: imx492: driver's header
> >   media: i2c: imx492: driver's source
> >   media: i2c: add imx492 config entry
> >   media: i2c: add imx492 entry for make
> 
> Could you squash patches 2, 3, 4, and 5 into a single patch please?

This is the exact opposite to what the networking folks would prefer, but it's
OK with me.

> Also - I think you'll need an entry in MAINTAINERS.

Right, that one too.

> I can't see any reference to the selection controls:
>  - V4L2_SEL_TGT_CROP_DEFAULT
>  - V4L2_SEL_TGT_CROP
>  - V4L2_SEL_TGT_CROP_BOUNDS

These are in a very early phase of implementing.

> Do you have the datasheet or sufficient information to be able to implement
> these controls?

Of course not.  We are brute forcing our way forward by mostly reverse
engineering another solution that has an FPGA on the other end of the CSI link.

> --
> Kieran
> 
> >  .../bindings/media/i2c/sony,imx492.yaml       |  101 ++
> >  drivers/media/i2c/Kconfig                     |   14 +
> >  drivers/media/i2c/Makefile                    |    1 +
> >  drivers/media/i2c/imx492.c                    | 1092 +++++++++++++++++
> >  drivers/media/i2c/imx492.h                    |  555 +++++++++
> >  5 files changed, 1763 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx492.yaml
> >  create mode 100644 drivers/media/i2c/imx492.c
> >  create mode 100644 drivers/media/i2c/imx492.h
> > 
> > -- 
> > 2.30.2
> >


		Petko
Kieran Bingham Nov. 28, 2022, 1:38 p.m. UTC | #3 
Quoting Petko Manolov (2022-11-28 13:11:39)
> On 22-11-28 12:45:32, Kieran Bingham wrote:
> > Hi Petko,
> > 
> > What device/ISP are you testing this sensor on?
> 
> I'll tell you, but only if you promise to not laugh too much.  Deal?
> 
> i.MX8M-Mini
> 
> I hope nobody will notice the above line...

I won't laugh at all. We've just merged the ISI pipeline handler for
libcamera, which should let you use libcamera on this target (I think).

I would hope sometime in the future you'd be able to use the ISI
pipeline handler with a GPU based ISP with this sensor too.

How are you managing the image 3a? the imx492 is a raw sensor isn't it ?


> > Quoting Petko Manolov (2022-11-25 15:31:15)
> > > This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> > > image sensor.
> > > 
> > > imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> > > monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> > > digital output makes it possible to output the signals with high definition for
> > > moving pictures. It is programmable through I2C interface. The I2C client
> > > address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> > > through MIPI CSI-2.
> > > 
> > > Petko Manolov (5):
> > >   DT bindings for imx492
> > >   media: i2c: imx492: driver's header
> > >   media: i2c: imx492: driver's source
> > >   media: i2c: add imx492 config entry
> > >   media: i2c: add imx492 entry for make
> > 
> > Could you squash patches 2, 3, 4, and 5 into a single patch please?
> 
> This is the exact opposite to what the networking folks would prefer, but it's
> OK with me.

Haha ... yes - isn't it great working in mulitple subsystems....

> > Also - I think you'll need an entry in MAINTAINERS.
> 
> Right, that one too.
> 
> > I can't see any reference to the selection controls:
> >  - V4L2_SEL_TGT_CROP_DEFAULT
> >  - V4L2_SEL_TGT_CROP
> >  - V4L2_SEL_TGT_CROP_BOUNDS
> 
> These are in a very early phase of implementing.

Ok

> > Do you have the datasheet or sufficient information to be able to implement
> > these controls?
> 
> Of course not.  We are brute forcing our way forward by mostly reverse
> engineering another solution that has an FPGA on the other end of the CSI link.

Understood ;-) (and ... I think what I was fearing/anticipating).

--
Kieran




> > --
> > Kieran
> > 
> > >  .../bindings/media/i2c/sony,imx492.yaml       |  101 ++
> > >  drivers/media/i2c/Kconfig                     |   14 +
> > >  drivers/media/i2c/Makefile                    |    1 +
> > >  drivers/media/i2c/imx492.c                    | 1092 +++++++++++++++++
> > >  drivers/media/i2c/imx492.h                    |  555 +++++++++
> > >  5 files changed, 1763 insertions(+)
> > >  create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx492.yaml
> > >  create mode 100644 drivers/media/i2c/imx492.c
> > >  create mode 100644 drivers/media/i2c/imx492.h
> > > 
> > > -- 
> > > 2.30.2
> > >
> 
> 
>                 Petko
Laurent Pinchart Nov. 28, 2022, 1:46 p.m. UTC | #4 
On Mon, Nov 28, 2022 at 01:38:12PM +0000, Kieran Bingham wrote:
> Quoting Petko Manolov (2022-11-28 13:11:39)
> > On 22-11-28 12:45:32, Kieran Bingham wrote:
> > > Hi Petko,
> > > 
> > > What device/ISP are you testing this sensor on?
> > 
> > I'll tell you, but only if you promise to not laugh too much.  Deal?
> > 
> > i.MX8M-Mini
> > 
> > I hope nobody will notice the above line...
> 
> I won't laugh at all. We've just merged the ISI pipeline handler for
> libcamera, which should let you use libcamera on this target (I think).

The i.MX8MM doesn't have an ISI. The i.MX8MN does, but with a single
pipeline, so it won't work out of the box with libcamera, but it should
be easy to support it.

The i.MX8MM has a CSI bridge (CSI stands for CMOS Sensor Interface
there, it's a DMA engine, not to be confused with the CSI-2 receiver)
which should work with the simple pipeline handler.

> I would hope sometime in the future you'd be able to use the ISI
> pipeline handler with a GPU based ISP with this sensor too.
> 
> How are you managing the image 3a? the imx492 is a raw sensor isn't it ?
> 
> > > Quoting Petko Manolov (2022-11-25 15:31:15)
> > > > This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> > > > image sensor.
> > > > 
> > > > imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> > > > monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> > > > digital output makes it possible to output the signals with high definition for
> > > > moving pictures. It is programmable through I2C interface. The I2C client
> > > > address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> > > > through MIPI CSI-2.
> > > > 
> > > > Petko Manolov (5):
> > > >   DT bindings for imx492
> > > >   media: i2c: imx492: driver's header
> > > >   media: i2c: imx492: driver's source
> > > >   media: i2c: add imx492 config entry
> > > >   media: i2c: add imx492 entry for make
> > > 
> > > Could you squash patches 2, 3, 4, and 5 into a single patch please?
> > 
> > This is the exact opposite to what the networking folks would prefer, but it's
> > OK with me.
> 
> Haha ... yes - isn't it great working in mulitple subsystems....
> 
> > > Also - I think you'll need an entry in MAINTAINERS.
> > 
> > Right, that one too.
> > 
> > > I can't see any reference to the selection controls:
> > >  - V4L2_SEL_TGT_CROP_DEFAULT
> > >  - V4L2_SEL_TGT_CROP
> > >  - V4L2_SEL_TGT_CROP_BOUNDS
> > 
> > These are in a very early phase of implementing.
> 
> Ok
> 
> > > Do you have the datasheet or sufficient information to be able to implement
> > > these controls?
> > 
> > Of course not.  We are brute forcing our way forward by mostly reverse
> > engineering another solution that has an FPGA on the other end of the CSI link.
> 
> Understood ;-) (and ... I think what I was fearing/anticipating).
>
> > > >  .../bindings/media/i2c/sony,imx492.yaml       |  101 ++
> > > >  drivers/media/i2c/Kconfig                     |   14 +
> > > >  drivers/media/i2c/Makefile                    |    1 +
> > > >  drivers/media/i2c/imx492.c                    | 1092 +++++++++++++++++
> > > >  drivers/media/i2c/imx492.h                    |  555 +++++++++
> > > >  5 files changed, 1763 insertions(+)
> > > >  create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx492.yaml
> > > >  create mode 100644 drivers/media/i2c/imx492.c
> > > >  create mode 100644 drivers/media/i2c/imx492.h
Petko Manolov Nov. 28, 2022, 2:07 p.m. UTC | #5 
On 22-11-28 13:38:12, Kieran Bingham wrote:
> Quoting Petko Manolov (2022-11-28 13:11:39)
> > On 22-11-28 12:45:32, Kieran Bingham wrote:
> > > Hi Petko,
> > > 
> > > What device/ISP are you testing this sensor on?
> > 
> > I'll tell you, but only if you promise to not laugh too much.  Deal?
> > 
> > i.MX8M-Mini
> > 
> > I hope nobody will notice the above line...
> 
> I won't laugh at all. We've just merged the ISI pipeline handler for
> libcamera, which should let you use libcamera on this target (I think).

I think i saw your patch in the ML.

> I would hope sometime in the future you'd be able to use the ISI pipeline
> handler with a GPU based ISP with this sensor too.

Both of the above are present in i.MX8M-Plus, not the Mini.

> How are you managing the image 3a? the imx492 is a raw sensor isn't it ?

Not sure what you mean by "image 3a" and yes, it's a raw sensor.

> > > Quoting Petko Manolov (2022-11-25 15:31:15)
> > > > This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> > > > image sensor.
> > > > 
> > > > imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> > > > monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> > > > digital output makes it possible to output the signals with high definition for
> > > > moving pictures. It is programmable through I2C interface. The I2C client
> > > > address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> > > > through MIPI CSI-2.
> > > > 
> > > > Petko Manolov (5):
> > > >   DT bindings for imx492
> > > >   media: i2c: imx492: driver's header
> > > >   media: i2c: imx492: driver's source
> > > >   media: i2c: add imx492 config entry
> > > >   media: i2c: add imx492 entry for make
> > > 
> > > Could you squash patches 2, 3, 4, and 5 into a single patch please?
> > 
> > This is the exact opposite to what the networking folks would prefer, but it's
> > OK with me.
> 
> Haha ... yes - isn't it great working in mulitple subsystems....

Yeah, easy to get confused... :)

> > > Also - I think you'll need an entry in MAINTAINERS.
> > 
> > Right, that one too.
> > 
> > > I can't see any reference to the selection controls:
> > >  - V4L2_SEL_TGT_CROP_DEFAULT
> > >  - V4L2_SEL_TGT_CROP
> > >  - V4L2_SEL_TGT_CROP_BOUNDS
> > 
> > These are in a very early phase of implementing.
> 
> Ok
> 
> > > Do you have the datasheet or sufficient information to be able to implement
> > > these controls?
> > 
> > Of course not.  We are brute forcing our way forward by mostly reverse
> > engineering another solution that has an FPGA on the other end of the CSI link.
> 
> Understood ;-) (and ... I think what I was fearing/anticipating).

I would have been too easy otherwise.


		Petko
Kieran Bingham Nov. 28, 2022, 3:04 p.m. UTC | #6 
Quoting Petko Manolov (2022-11-28 14:07:22)
> On 22-11-28 13:38:12, Kieran Bingham wrote:
> > Quoting Petko Manolov (2022-11-28 13:11:39)
> > > On 22-11-28 12:45:32, Kieran Bingham wrote:
> > > > Hi Petko,
> > > > 
> > > > What device/ISP are you testing this sensor on?
> > > 
> > > I'll tell you, but only if you promise to not laugh too much.  Deal?
> > > 
> > > i.MX8M-Mini
> > > 
> > > I hope nobody will notice the above line...
> > 
> > I won't laugh at all. We've just merged the ISI pipeline handler for
> > libcamera, which should let you use libcamera on this target (I think).
> 
> I think i saw your patch in the ML.
> 
> > I would hope sometime in the future you'd be able to use the ISI pipeline
> > handler with a GPU based ISP with this sensor too.
> 
> Both of the above are present in i.MX8M-Plus, not the Mini.
> 
> > How are you managing the image 3a? the imx492 is a raw sensor isn't it ?
> 
> Not sure what you mean by "image 3a" and yes, it's a raw sensor.

I meant how are you handling "Auto-exposure" "Auto-gain"
"Auto-whitebalance" (known as 3a).

There's more than that of course, but those are the basics that are
expected to be able to get a 'usable' image. Unless of course you're
doing everything with manual controls ... which is the alternative for a
fixed/known lighting environment.

But without an ISP on the device anyway, perhaps there's already more
going on elsewhere?

--
Kieran.



> > > > Quoting Petko Manolov (2022-11-25 15:31:15)
> > > > > This patch adds dt-bindings documentation, driver source, etc for Sony imx492
> > > > > image sensor.
> > > > > 
> > > > > imx492 is a diagonal 23.1 mm (Type 1.4) CMOS image sensor with color or
> > > > > monochrome square pixel array and approximately 47.08 M effective pixels. 12-bit
> > > > > digital output makes it possible to output the signals with high definition for
> > > > > moving pictures. It is programmable through I2C interface. The I2C client
> > > > > address can be either 0x10 or 0x1a depending on SLASEL pin. Image data is sent
> > > > > through MIPI CSI-2.
> > > > > 
> > > > > Petko Manolov (5):
> > > > >   DT bindings for imx492
> > > > >   media: i2c: imx492: driver's header
> > > > >   media: i2c: imx492: driver's source
> > > > >   media: i2c: add imx492 config entry
> > > > >   media: i2c: add imx492 entry for make
> > > > 
> > > > Could you squash patches 2, 3, 4, and 5 into a single patch please?
> > > 
> > > This is the exact opposite to what the networking folks would prefer, but it's
> > > OK with me.
> > 
> > Haha ... yes - isn't it great working in mulitple subsystems....
> 
> Yeah, easy to get confused... :)
> 
> > > > Also - I think you'll need an entry in MAINTAINERS.
> > > 
> > > Right, that one too.
> > > 
> > > > I can't see any reference to the selection controls:
> > > >  - V4L2_SEL_TGT_CROP_DEFAULT
> > > >  - V4L2_SEL_TGT_CROP
> > > >  - V4L2_SEL_TGT_CROP_BOUNDS
> > > 
> > > These are in a very early phase of implementing.
> > 
> > Ok
> > 
> > > > Do you have the datasheet or sufficient information to be able to implement
> > > > these controls?
> > > 
> > > Of course not.  We are brute forcing our way forward by mostly reverse
> > > engineering another solution that has an FPGA on the other end of the CSI link.
> > 
> > Understood ;-) (and ... I think what I was fearing/anticipating).
> 
> I would have been too easy otherwise.
> 
> 
>                 Petko