From patchwork Thu Jun 4 18:57:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jernej_=C5=A0krabec?= X-Patchwork-Id: 64289 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([23.128.96.18]) by www.linuxtv.org with esmtp (Exim 4.92) (envelope-from ) id 1jguxv-0008AO-AZ; Thu, 04 Jun 2020 18:51:47 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729131AbgFDSzU (ORCPT + 1 other); Thu, 4 Jun 2020 14:55:20 -0400 Received: from mailoutvs25.siol.net ([185.57.226.216]:52662 "EHLO mail.siol.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729070AbgFDSzT (ORCPT ); Thu, 4 Jun 2020 14:55:19 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTP id ED5AE5210C2; Thu, 4 Jun 2020 20:55:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at psrvmta10.zcs-production.pri Received: from mail.siol.net ([127.0.0.1]) by localhost (psrvmta10.zcs-production.pri [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id i_V4WpUo9TTS; Thu, 4 Jun 2020 20:55:15 +0200 (CEST) Received: from mail.siol.net (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTPS id 7D9AE521138; Thu, 4 Jun 2020 20:55:15 +0200 (CEST) Received: from kista.localdomain (cpe-194-152-20-232.static.triera.net [194.152.20.232]) (Authenticated sender: 031275009) by mail.siol.net (Postfix) with ESMTPSA id 0FD1E52118B; Thu, 4 Jun 2020 20:55:13 +0200 (CEST) From: Jernej Skrabec To: paul.kocialkowski@bootlin.com, mripard@kernel.org Cc: mchehab@kernel.org, wens@csie.org, hverkuil-cisco@xs4all.nl, gregkh@linuxfoundation.org, jonas@kwiboo.se, nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devel@driverdev.osuosl.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/3] media: uapi: h264: update reference lists Date: Thu, 4 Jun 2020 20:57:43 +0200 Message-Id: <20200604185745.23568-2-jernej.skrabec@siol.net> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200604185745.23568-1-jernej.skrabec@siol.net> References: <20200604185745.23568-1-jernej.skrabec@siol.net> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-LSpam-Score: -2.4 (--) X-LSpam-Report: No, score=-2.4 required=5.0 tests=BAYES_00=-1.9,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1 autolearn=ham autolearn_force=no When dealing with with interlaced frames, reference lists must tell if each particular reference is meant for top or bottom field. This info is currently not provided at all in the H264 related controls. Make reference lists hold a structure which will also hold flags along index into DPB array. Flags will tell if reference is meant for top or bottom field. Currently the only user of these lists is Cedrus which is just compile fixed here. Actual usage of newly introduced flags will come in following commit. Signed-off-by: Jernej Skrabec Reviewed-by: Nicolas Dufresne --- .../media/v4l/ext-ctrls-codec.rst | 40 ++++++++++++++++++- .../staging/media/sunxi/cedrus/cedrus_h264.c | 6 +-- include/media/h264-ctrls.h | 12 +++++- 3 files changed, 51 insertions(+), 7 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index d0d506a444b1..6c36d298db20 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1843,10 +1843,10 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - __u32 - ``slice_group_change_cycle`` - - * - __u8 + * - struct :c:type:`v4l2_h264_reference` - ``ref_pic_list0[32]`` - Reference picture list after applying the per-slice modifications - * - __u8 + * - struct :c:type:`v4l2_h264_reference` - ``ref_pic_list1[32]`` - Reference picture list after applying the per-slice modifications * - __u32 @@ -1926,6 +1926,42 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``chroma_offset[32][2]`` - +``Picture Reference`` + +.. c:type:: v4l2_h264_reference + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_h264_reference + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u16 + - ``flags`` + - See :ref:`Picture Reference Flags ` + * - __u8 + - ``index`` + - + +.. _h264_reference_flags: + +``Picture Reference Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_REFERENCE_FLAG_TOP_FIELD`` + - 0x00000001 + - + * - ``V4L2_H264_REFERENCE_FLAG_BOTTOM_FIELD`` + - 0x00000002 + - + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` Specifies the decode parameters (as extracted from the bitstream) for the associated H264 slice data. This includes the necessary diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index 54ee2aa423e2..cce527bbdf86 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -166,8 +166,8 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, struct cedrus_run *run, - const u8 *ref_list, u8 num_ref, - enum cedrus_h264_sram_off sram) + const struct v4l2_h264_reference *ref_list, + u8 num_ref, enum cedrus_h264_sram_off sram) { const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params; struct vb2_queue *cap_q; @@ -188,7 +188,7 @@ static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, int buf_idx; u8 dpb_idx; - dpb_idx = ref_list[i]; + dpb_idx = ref_list[i].index; dpb = &decode->dpb[dpb_idx]; if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 080fd1293c42..9b1cbc9bc38e 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -140,6 +140,14 @@ struct v4l2_h264_pred_weight_table { #define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 #define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 +#define V4L2_H264_REFERENCE_FLAG_TOP_FIELD 0x01 +#define V4L2_H264_REFERENCE_FLAG_BOTTOM_FIELD 0x02 + +struct v4l2_h264_reference { + __u8 flags; + __u8 index; +}; + struct v4l2_ctrl_h264_slice_params { /* Size in bytes, including header */ __u32 size; @@ -182,8 +190,8 @@ struct v4l2_ctrl_h264_slice_params { * Entries on each list are indices into * v4l2_ctrl_h264_decode_params.dpb[]. */ - __u8 ref_pic_list0[32]; - __u8 ref_pic_list1[32]; + struct v4l2_h264_reference ref_pic_list0[32]; + struct v4l2_h264_reference ref_pic_list1[32]; __u32 flags; }; From patchwork Thu Jun 4 18:57:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jernej_=C5=A0krabec?= X-Patchwork-Id: 64288 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([23.128.96.18]) by www.linuxtv.org with esmtp (Exim 4.92) (envelope-from ) id 1jguxu-0008AO-5y; Thu, 04 Jun 2020 18:51:46 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729151AbgFDSzW (ORCPT + 1 other); Thu, 4 Jun 2020 14:55:22 -0400 Received: from mailoutvs43.siol.net ([185.57.226.234]:52691 "EHLO mail.siol.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728476AbgFDSzV (ORCPT ); Thu, 4 Jun 2020 14:55:21 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTP id 35193521138; Thu, 4 Jun 2020 20:55:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at psrvmta10.zcs-production.pri Received: from mail.siol.net ([127.0.0.1]) by localhost (psrvmta10.zcs-production.pri [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id XznPkMjjbkV1; Thu, 4 Jun 2020 20:55:17 +0200 (CEST) Received: from mail.siol.net (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTPS id D034552118B; Thu, 4 Jun 2020 20:55:17 +0200 (CEST) Received: from kista.localdomain (cpe-194-152-20-232.static.triera.net [194.152.20.232]) (Authenticated sender: 031275009) by mail.siol.net (Postfix) with ESMTPSA id 82AC452123C; Thu, 4 Jun 2020 20:55:15 +0200 (CEST) From: Jernej Skrabec To: paul.kocialkowski@bootlin.com, mripard@kernel.org Cc: mchehab@kernel.org, wens@csie.org, hverkuil-cisco@xs4all.nl, gregkh@linuxfoundation.org, jonas@kwiboo.se, nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devel@driverdev.osuosl.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/3] media: cedrus: h264: Properly configure reference field Date: Thu, 4 Jun 2020 20:57:44 +0200 Message-Id: <20200604185745.23568-3-jernej.skrabec@siol.net> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200604185745.23568-1-jernej.skrabec@siol.net> References: <20200604185745.23568-1-jernej.skrabec@siol.net> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-LSpam-Score: -2.4 (--) X-LSpam-Report: No, score=-2.4 required=5.0 tests=BAYES_00=-1.9,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1 autolearn=ham autolearn_force=no When interlaced H264 content is being decoded, references must indicate which field is being referenced. Currently this was done by checking capture buffer flags. However, that is not correct because capture buffer may hold both fields. Fix this by checking newly introduced flags in reference lists. Signed-off-by: Jernej Skrabec Reviewed-by: Nicolas Dufresne --- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index cce527bbdf86..c87717d17ec5 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -183,7 +183,6 @@ static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, for (i = 0; i < num_ref; i++) { const struct v4l2_h264_dpb_entry *dpb; const struct cedrus_buffer *cedrus_buf; - const struct vb2_v4l2_buffer *ref_buf; unsigned int position; int buf_idx; u8 dpb_idx; @@ -198,12 +197,11 @@ static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, if (buf_idx < 0) continue; - ref_buf = to_vb2_v4l2_buffer(cap_q->bufs[buf_idx]); - cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); + cedrus_buf = vb2_to_cedrus_buffer(cap_q->bufs[buf_idx]); position = cedrus_buf->codec.h264.position; sram_array[i] |= position << 1; - if (ref_buf->field == V4L2_FIELD_BOTTOM) + if (ref_list[i].flags & V4L2_H264_REFERENCE_FLAG_BOTTOM_FIELD) sram_array[i] |= BIT(0); } From patchwork Thu Jun 4 18:57:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jernej_=C5=A0krabec?= X-Patchwork-Id: 64287 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([23.128.96.18]) by www.linuxtv.org with esmtp (Exim 4.92) (envelope-from ) id 1jguxo-0008A8-Ek; Thu, 04 Jun 2020 18:51:41 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729178AbgFDSzY (ORCPT + 1 other); Thu, 4 Jun 2020 14:55:24 -0400 Received: from mailoutvs56.siol.net ([185.57.226.247]:52722 "EHLO mail.siol.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729141AbgFDSzX (ORCPT ); Thu, 4 Jun 2020 14:55:23 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTP id 77EC0521370; Thu, 4 Jun 2020 20:55:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at psrvmta10.zcs-production.pri Received: from mail.siol.net ([127.0.0.1]) by localhost (psrvmta10.zcs-production.pri [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 22HNnnPf64Hw; Thu, 4 Jun 2020 20:55:20 +0200 (CEST) Received: from mail.siol.net (localhost [127.0.0.1]) by mail.siol.net (Postfix) with ESMTPS id 2AD98521295; Thu, 4 Jun 2020 20:55:20 +0200 (CEST) Received: from kista.localdomain (cpe-194-152-20-232.static.triera.net [194.152.20.232]) (Authenticated sender: 031275009) by mail.siol.net (Postfix) with ESMTPSA id D2A915211F9; Thu, 4 Jun 2020 20:55:17 +0200 (CEST) From: Jernej Skrabec To: paul.kocialkowski@bootlin.com, mripard@kernel.org Cc: mchehab@kernel.org, wens@csie.org, hverkuil-cisco@xs4all.nl, gregkh@linuxfoundation.org, jonas@kwiboo.se, nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devel@driverdev.osuosl.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 3/3] media: cedrus: h264: Fix frame list construction Date: Thu, 4 Jun 2020 20:57:45 +0200 Message-Id: <20200604185745.23568-4-jernej.skrabec@siol.net> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200604185745.23568-1-jernej.skrabec@siol.net> References: <20200604185745.23568-1-jernej.skrabec@siol.net> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-LSpam-Score: -2.4 (--) X-LSpam-Report: No, score=-2.4 required=5.0 tests=BAYES_00=-1.9,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1 autolearn=ham autolearn_force=no Current frame list construction algorithm assumes that decoded image will be output into its own buffer. That is true for progressive content but not for interlaced where each field is decoded separately into same buffer. Fix that by checking if capture buffer is listed in DPB. If it is, reuse it. Signed-off-by: Jernej Skrabec --- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index c87717d17ec5..4f79386315ae 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -102,7 +102,7 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, struct cedrus_dev *dev = ctx->dev; unsigned long used_dpbs = 0; unsigned int position; - unsigned int output = 0; + int output = -1; unsigned int i; cap_q = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE); @@ -125,6 +125,11 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, position = cedrus_buf->codec.h264.position; used_dpbs |= BIT(position); + if (run->dst->vb2_buf.timestamp == dpb->reference_ts) { + output = position; + continue; + } + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; @@ -132,13 +137,11 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, dpb->top_field_order_cnt, dpb->bottom_field_order_cnt, &pic_list[position]); - - output = max(position, output); } - position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM, - output); - if (position >= CEDRUS_H264_FRAME_NUM) + if (output >= 0) + position = output; + else position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM); output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf);