diff mbox series

[v5,05/10] media: v4l: Add definition for the Aspeed JPEG format

Message ID 20211118074030.685-6-jammy_huang@aspeedtech.com (mailing list archive)
State Changes Requested, archived
Delegated to: Hans Verkuil
Headers
Series add aspeed-jpeg support for aspeed-video |

Commit Message

Jammy Huang Nov. 18, 2021, 7:40 a.m. UTC 
  This introduces support for the Aspeed JPEG format, where the new frame
can refer to previous frame to reduce the amount of compressed data. The
concept is similar to I/P frame of video compression. I will compare the
new frame with previous one to decide which macroblock's data is
changed, and only the changed macroblocks will be compressed.

This Aspeed JPEG format is used by the video engine on Aspeed platforms,
which is generally adapted for remote KVM.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - new
---
 Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
 include/uapi/linux/videodev2.h                   |  1 +
 3 files changed, 14 insertions(+)

Comments

Sakari Ailus Nov. 18, 2021, 12:33 p.m. UTC | #1 
Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. I will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - new
> ---
>  Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>  include/uapi/linux/videodev2.h                   |  1 +
>  3 files changed, 14 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> index b2cd155e691b..23c05063133d 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> +        for more information.

Is this datasheet publicly available? Do you have a URL?
Jammy Huang Nov. 19, 2021, 2:02 a.m. UTC | #2 
Hi Sakari,

On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. I will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>   include/uapi/linux/videodev2.h                   |  1 +
>>   3 files changed, 14 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> index b2cd155e691b..23c05063133d 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>> +        for more information.
> Is this datasheet publicly available? Do you have a URL?

Sorry, this datasheet is not publicly available.
Hans mentioned this as well in the discussion below:

https://lkml.org/lkml/2021/11/10/101
Paul Menzel Nov. 19, 2021, 8:39 a.m. UTC | #3 
Dear Jammy,


Am 19.11.21 um 03:02 schrieb Jammy Huang:

> On 2021/11/18 下午 08:33, Sakari Ailus wrote:

>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>> This introduces support for the Aspeed JPEG format, where the new frame
>>> can refer to previous frame to reduce the amount of compressed data. The
>>> concept is similar to I/P frame of video compression. I will compare the
>>> new frame with previous one to decide which macroblock's data is
>>> changed, and only the changed macroblocks will be compressed.
>>>
>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>> which is generally adapted for remote KVM.
>>>
>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>> ---
>>> v5:
>>>    - no update
>>> v4:
>>>    - new
>>> ---
>>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>   include/uapi/linux/videodev2.h                   |  1 +
>>>   3 files changed, 14 insertions(+)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst 
>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> index b2cd155e691b..23c05063133d 100644
>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media 
>>> mailing list.
>>>       of tiles, resulting in 32-aligned resolutions for the luminance 
>>> plane
>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>       subsampling).
>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>> +
>>> +      - ``V4L2_PIX_FMT_AJPG``
>>> +      - 'AJPG'
>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>> +        which is generally adapted for remote KVM.
>>> +        On each frame compression, I will compare the new frame with previous
>>> +        one to decide which macroblock's data is changed, and only the changed
>>> +        macroblocks will be compressed.
>>> +
>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>> +        for more information.
>> Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If questions come up during review, please also add the answers to the 
commit message of the next iteration. ;-) Maybe:

> The implementation is based on datasheet *Name goes here*, revision
> X, which is not publicly available.

Kind regards,

Paul
Jammy Huang Nov. 19, 2021, 9:37 a.m. UTC | #4 
Dear Paul,

OK, thanks for your suggestion.

On 2021/11/19 下午 04:39, Paul Menzel wrote:
> Dear Jammy,
>
>
> Am 19.11.21 um 03:02 schrieb Jammy Huang:
>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media
>>>> mailing list.
>>>>        of tiles, resulting in 32-aligned resolutions for the luminance
>>>> plane
>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>        subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If questions come up during review, please also add the answers to the
> commit message of the next iteration. ;-) Maybe:
>
>> The implementation is based on datasheet *Name goes here*, revision
>> X, which is not publicly available.
> Kind regards,
>
> Paul
Sakari Ailus Nov. 23, 2021, 10:23 a.m. UTC | #5 
Hi Jammy,

On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
> Hi Sakari,
> 
> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> > Hi Jammy,
> > 
> > On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> > > This introduces support for the Aspeed JPEG format, where the new frame
> > > can refer to previous frame to reduce the amount of compressed data. The
> > > concept is similar to I/P frame of video compression. I will compare the
> > > new frame with previous one to decide which macroblock's data is
> > > changed, and only the changed macroblocks will be compressed.
> > > 
> > > This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> > > which is generally adapted for remote KVM.
> > > 
> > > Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> > > ---
> > > v5:
> > >    - no update
> > > v4:
> > >    - new
> > > ---
> > >   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
> > >   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
> > >   include/uapi/linux/videodev2.h                   |  1 +
> > >   3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > index b2cd155e691b..23c05063133d 100644
> > > --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
> > >   	of tiles, resulting in 32-aligned resolutions for the luminance plane
> > >   	and 16-aligned resolutions for the chrominance plane (with 2x2
> > >   	subsampling).
> > > +    * .. _V4L2-PIX-FMT-AJPG:
> > > +
> > > +      - ``V4L2_PIX_FMT_AJPG``
> > > +      - 'AJPG'
> > > +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> > > +        which is generally adapted for remote KVM.
> > > +        On each frame compression, I will compare the new frame with previous
> > > +        one to decide which macroblock's data is changed, and only the changed
> > > +        macroblocks will be compressed.
> > > +
> > > +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> > > +        for more information.
> > Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If the vendor documentation is not publicly available, you'll need to have
the format documented here. Alternatively an open source implementation
(e.g. LGPL) is presumably fine, too.
Jammy Huang Nov. 26, 2021, 8:08 a.m. UTC | #6 
Hi Sakari,

On 2021/11/23 下午 06:23, Sakari Ailus wrote:
> Hi Jammy,
>
> On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
>> Hi Sakari,
>>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> Hi Jammy,
>>>
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>>>    	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>    	and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>    	subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If the vendor documentation is not publicly available, you'll need to have
> the format documented here. Alternatively an open source implementation
> (e.g. LGPL) is presumably fine, too.
Please refer to 
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf.
At Page 132, 'Video stream data format – ASPEED mode compression' 
describes the data format.
Is above information  enough??

Besides, I have already had openbmc's kvm support this aspeed format.
The source code of the implementation will be released after performance 
tuning finished.
diff mbox series

Patch

diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index b2cd155e691b..23c05063133d 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -264,6 +264,18 @@  please make a proposal on the linux-media mailing list.
 	of tiles, resulting in 32-aligned resolutions for the luminance plane
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
+    * .. _V4L2-PIX-FMT-AJPG:
+
+      - ``V4L2_PIX_FMT_AJPG``
+      - 'AJPG'
+      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
+        which is generally adapted for remote KVM.
+        On each frame compression, I will compare the new frame with previous
+        one to decide which macroblock's data is changed, and only the changed
+        macroblocks will be compressed.
+
+        You could reference to chapter 36, Video Engine, of AST2600's datasheet
+        for more information.
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 24db33f803c0..00dde01d2f97 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1378,6 +1378,7 @@  static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3210b3c82a4a..994eb6155ea9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -726,6 +726,7 @@  struct v4l2_pix_format {
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
+#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */