From patchwork Fri Oct 11 09:32:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59416 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI3-0001pj-CA; Fri, 11 Oct 2019 09:32:51 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727670AbfJKJct (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:49 -0400 Received: from lb3-smtp-cloud7.xs4all.net ([194.109.24.31]:55371 "EHLO lb3-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726885AbfJKJct (ORCPT ); Fri, 11 Oct 2019 05:32:49 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHyiRKiy; Fri, 11 Oct 2019 11:32:47 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Hans Verkuil Subject: [PATCHv4 1/6] vb2: add V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF Date: Fri, 11 Oct 2019 11:32:40 +0200 Message-Id: <20191011093245.33366-2-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfFv/7qNpfmVbnIJ/xmr9gUdIdi33yDJQ9mfG03ySrqcndY7GpTCgB697IjgR55ob4jHfPPDO4c9aONuhGHEiQ7rnoWv/4S4l8pcVM924VrlxEGQ+L3dy qnbf4xhH17soiy8VW29pfyad9zvbR0J3fbb9jtxuoQdRYonjlnmay+mx33L/H3gv5xCmWj5c4h1k41m2Tsigb40BP5Vg78WSoP7419I/qum8VgjJhuKPQjRi jwuq5xlDzGP+BM9B2lFR3cZE6NA2IQtS1GX+QOdViC1TzYluBVjwU6JQa2eY3wumNYT/FhTdc4OFhz1P35hC5TIthxgB0Q047qfUteJJ+dDhIokqxyzKH1uB ndrhrBUeNslspSbo2b6tWOJ0rQbNLLJLDwdgau6m6Ihy6NUZUZFHTUToMscci/8ahnir24A3QRc94tQQvXroC+NA2K4/DyS806T881DbD+r2BVp0vjmHrP3x LaIs/YVYpO95jPGC Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This patch adds support for the V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF flag. It also adds a new V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF capability. Drivers should set vb2_queue->subsystem_flags to VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF to indicate support for this flag. Signed-off-by: Hans Verkuil --- Documentation/media/uapi/v4l/buffer.rst | 13 +++++++++++++ Documentation/media/uapi/v4l/vidioc-reqbufs.rst | 6 ++++++ drivers/media/common/videobuf2/videobuf2-v4l2.c | 12 ++++++++++-- include/media/videobuf2-core.h | 3 +++ include/media/videobuf2-v4l2.h | 5 +++++ include/uapi/linux/videodev2.h | 13 ++++++++----- 6 files changed, 45 insertions(+), 7 deletions(-) diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst index 1cbd9cde57f3..9149b57728e5 100644 --- a/Documentation/media/uapi/v4l/buffer.rst +++ b/Documentation/media/uapi/v4l/buffer.rst @@ -607,6 +607,19 @@ Buffer Flags applications shall use this flag for output buffers if the data in this buffer has not been created by the CPU but by some DMA-capable unit, in which case caches have not been used. + * .. _`V4L2-BUF-FLAG-M2M-HOLD-CAPTURE-BUF`: + + - ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` + - 0x00000200 + - Only valid if ``V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF`` is + set. It is typically used with stateless decoders where multiple + output buffers each decode to a slice of the decoded frame. + Applications can set this flag when queueing the output buffer + to prevent the driver from dequeueing the capture buffer after + the output buffer has been decoded (i.e. the capture buffer is + 'held'). If the timestamp of this output buffer differs from that + of the previous output buffer, then that indicates the start of a + new frame and the previously held capture buffer is dequeued. * .. _`V4L2-BUF-FLAG-LAST`: - ``V4L2_BUF_FLAG_LAST`` diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst index d7faef10e39b..d0c643db477a 100644 --- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst +++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst @@ -125,6 +125,7 @@ aborting or finishing any DMA in progress, an implicit .. _V4L2-BUF-CAP-SUPPORTS-DMABUF: .. _V4L2-BUF-CAP-SUPPORTS-REQUESTS: .. _V4L2-BUF-CAP-SUPPORTS-ORPHANED-BUFS: +.. _V4L2-BUF-CAP-SUPPORTS-M2M-HOLD-CAPTURE-BUF: .. cssclass:: longtable @@ -150,6 +151,11 @@ aborting or finishing any DMA in progress, an implicit - The kernel allows calling :ref:`VIDIOC_REQBUFS` while buffers are still mapped or exported via DMABUF. These orphaned buffers will be freed when they are unmapped or when the exported DMABUF fds are closed. + * - ``V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF`` + - 0x00000020 + - Only valid for stateless decoders. If set, then userspace can set the + ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag to hold off on returning the + capture buffer until the OUTPUT timestamp changes. Return Value ============ diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index 5a9ba3846f0a..e652f4318284 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -49,8 +49,11 @@ module_param(debug, int, 0644); V4L2_BUF_FLAG_REQUEST_FD | \ V4L2_BUF_FLAG_TIMESTAMP_MASK) /* Output buffer flags that should be passed on to the driver */ -#define V4L2_BUFFER_OUT_FLAGS (V4L2_BUF_FLAG_PFRAME | V4L2_BUF_FLAG_BFRAME | \ - V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_TIMECODE) +#define V4L2_BUFFER_OUT_FLAGS (V4L2_BUF_FLAG_PFRAME | \ + V4L2_BUF_FLAG_BFRAME | \ + V4L2_BUF_FLAG_KEYFRAME | \ + V4L2_BUF_FLAG_TIMECODE | \ + V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF) /* * __verify_planes_array() - verify that the planes array passed in struct @@ -194,6 +197,7 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b } vbuf->sequence = 0; vbuf->request_fd = -1; + vbuf->is_held = false; if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) { switch (b->memory) { @@ -321,6 +325,8 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b */ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE; vbuf->field = b->field; + if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) + vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; } else { /* Zero any output buffer flags as this is a capture buffer */ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS; @@ -654,6 +660,8 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps) *caps |= V4L2_BUF_CAP_SUPPORTS_USERPTR; if (q->io_modes & VB2_DMABUF) *caps |= V4L2_BUF_CAP_SUPPORTS_DMABUF; + if (q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF) + *caps |= V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF; #ifdef CONFIG_MEDIA_CONTROLLER_REQUEST_API if (q->supports_requests) *caps |= V4L2_BUF_CAP_SUPPORTS_REQUESTS; diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index 640aabe69450..a2b2208b02da 100644 --- a/include/media/videobuf2-core.h +++ b/include/media/videobuf2-core.h @@ -505,6 +505,8 @@ struct vb2_buf_ops { * @buf_ops: callbacks to deliver buffer information. * between user-space and kernel-space. * @drv_priv: driver private data. + * @subsystem_flags: Flags specific to the subsystem (V4L2/DVB/etc.). Not used + * by the vb2 core. * @buf_struct_size: size of the driver-specific buffer structure; * "0" indicates the driver doesn't want to use a custom buffer * structure type. for example, ``sizeof(struct vb2_v4l2_buffer)`` @@ -571,6 +573,7 @@ struct vb2_queue { const struct vb2_buf_ops *buf_ops; void *drv_priv; + u32 subsystem_flags; unsigned int buf_struct_size; u32 timestamp_flags; gfp_t gfp_flags; diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h index 8a10889dc2fd..59bf33a12648 100644 --- a/include/media/videobuf2-v4l2.h +++ b/include/media/videobuf2-v4l2.h @@ -33,6 +33,7 @@ * @timecode: frame timecode. * @sequence: sequence count of this frame. * @request_fd: the request_fd associated with this buffer + * @is_held: if true, then this capture buffer was held * @planes: plane information (userptr/fd, length, bytesused, data_offset). * * Should contain enough information to be able to cover all the fields @@ -46,9 +47,13 @@ struct vb2_v4l2_buffer { struct v4l2_timecode timecode; __u32 sequence; __s32 request_fd; + bool is_held; struct vb2_plane planes[VB2_MAX_PLANES]; }; +/* VB2 V4L2 flags as set in vb2_queue.subsystem_flags */ +#define VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF (1 << 0) + /* * to_vb2_v4l2_buffer() - cast struct vb2_buffer * to struct vb2_v4l2_buffer * */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index b3c0961b62a0..9f4e66affac4 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -920,11 +920,12 @@ struct v4l2_requestbuffers { }; /* capabilities for struct v4l2_requestbuffers and v4l2_create_buffers */ -#define V4L2_BUF_CAP_SUPPORTS_MMAP (1 << 0) -#define V4L2_BUF_CAP_SUPPORTS_USERPTR (1 << 1) -#define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2) -#define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3) -#define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4) +#define V4L2_BUF_CAP_SUPPORTS_MMAP (1 << 0) +#define V4L2_BUF_CAP_SUPPORTS_USERPTR (1 << 1) +#define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2) +#define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3) +#define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4) +#define V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF (1 << 5) /** * struct v4l2_plane - plane info for multi-planar buffers @@ -1046,6 +1047,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv) #define V4L2_BUF_FLAG_IN_REQUEST 0x00000080 /* timecode field is valid */ #define V4L2_BUF_FLAG_TIMECODE 0x00000100 +/* Don't return the capture buffer until OUTPUT timestamp changes */ +#define V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF 0x00000200 /* Buffer is prepared for queuing */ #define V4L2_BUF_FLAG_PREPARED 0x00000400 /* Cache handling flags */ From patchwork Fri Oct 11 09:32:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59421 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI8-0001pj-9a; Fri, 11 Oct 2019 09:32:56 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727671AbfJKJct (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:49 -0400 Received: from lb3-smtp-cloud7.xs4all.net ([194.109.24.31]:55671 "EHLO lb3-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727604AbfJKJct (ORCPT ); Fri, 11 Oct 2019 05:32:49 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHziRKjB; Fri, 11 Oct 2019 11:32:47 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Hans Verkuil Subject: [PATCHv4 2/6] v4l2-mem2mem: support held capture buffers Date: Fri, 11 Oct 2019 11:32:41 +0200 Message-Id: <20191011093245.33366-3-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfFv/7qNpfmVbnIJ/xmr9gUdIdi33yDJQ9mfG03ySrqcndY7GpTCgB697IjgR55ob4jHfPPDO4c9aONuhGHEiQ7rnoWv/4S4l8pcVM924VrlxEGQ+L3dy qnbf4xhH17soiy8VW29pfyad9zvbR0J3fbb9jtxuoQdRYonjlnmay+mx33L/H3gv5xCmWj5c4h1k41m2Tsigb40BP5Vg78WSoP7419I/qum8VgjJhuKPQjRi jwuq5xlDzGP+BM9B2lFR3cZE6NA2IQtS1GX+QOdViC1TzYluBVjwU6JQa2eY3wumNYT/FhTdc4OFhz1P35hC5TIthxgB0Q047qfUteJJ+dDhIokqxyzKH1uB ndrhrBUeNslspSbo2b6tWOJ0rQbNLLJLDwdgau6m6Ihy6NUZUZFHTUToMscci/8ahnir24A3QRc94tQQvXroC+NA2K4/DyS806T881DbD+r2BVp0vjmHrP3x LaIs/YVYpO95jPGC Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Check for held buffers that are ready to be returned to vb2 in __v4l2_m2m_try_queue(). This avoids drivers having to handle this case. Add v4l2_m2m_buf_done_and_job_finish() to correctly return source and destination buffers and mark the job as finished while taking a held destination buffer into account (i.e. that buffer won't be returned). This has to be done while job_spinlock is held to avoid race conditions. Signed-off-by: Hans Verkuil --- drivers/media/v4l2-core/v4l2-mem2mem.c | 130 ++++++++++++++++++------- include/media/v4l2-mem2mem.h | 33 ++++++- 2 files changed, 128 insertions(+), 35 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 19937dd3c6f6..79c3656f24f7 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -284,7 +284,8 @@ static void v4l2_m2m_try_run(struct v4l2_m2m_dev *m2m_dev) static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, struct v4l2_m2m_ctx *m2m_ctx) { - unsigned long flags_job, flags_out, flags_cap; + unsigned long flags_job; + struct vb2_v4l2_buffer *dst, *src; dprintk("Trying to schedule a job for m2m_ctx: %p\n", m2m_ctx); @@ -307,20 +308,30 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, goto job_unlock; } - spin_lock_irqsave(&m2m_ctx->out_q_ctx.rdy_spinlock, flags_out); - if (list_empty(&m2m_ctx->out_q_ctx.rdy_queue) - && !m2m_ctx->out_q_ctx.buffered) { + src = v4l2_m2m_next_src_buf(m2m_ctx); + dst = v4l2_m2m_next_dst_buf(m2m_ctx); + if (!src && !m2m_ctx->out_q_ctx.buffered) { dprintk("No input buffers available\n"); - goto out_unlock; + goto job_unlock; } - spin_lock_irqsave(&m2m_ctx->cap_q_ctx.rdy_spinlock, flags_cap); - if (list_empty(&m2m_ctx->cap_q_ctx.rdy_queue) - && !m2m_ctx->cap_q_ctx.buffered) { + if (!dst && !m2m_ctx->cap_q_ctx.buffered) { dprintk("No output buffers available\n"); - goto cap_unlock; + goto job_unlock; + } + + if (src && dst && + dst->is_held && dst->vb2_buf.copied_timestamp && + dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { + dst->is_held = false; + v4l2_m2m_dst_buf_remove(m2m_ctx); + v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE); + dst = v4l2_m2m_next_dst_buf(m2m_ctx); + + if (!dst && !m2m_ctx->cap_q_ctx.buffered) { + dprintk("No output buffers available after returning held buffer\n"); + goto job_unlock; + } } - spin_unlock_irqrestore(&m2m_ctx->cap_q_ctx.rdy_spinlock, flags_cap); - spin_unlock_irqrestore(&m2m_ctx->out_q_ctx.rdy_spinlock, flags_out); if (m2m_dev->m2m_ops->job_ready && (!m2m_dev->m2m_ops->job_ready(m2m_ctx->priv))) { @@ -331,13 +342,6 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, list_add_tail(&m2m_ctx->queue, &m2m_dev->job_queue); m2m_ctx->job_flags |= TRANS_QUEUED; - spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags_job); - return; - -cap_unlock: - spin_unlock_irqrestore(&m2m_ctx->cap_q_ctx.rdy_spinlock, flags_cap); -out_unlock: - spin_unlock_irqrestore(&m2m_ctx->out_q_ctx.rdy_spinlock, flags_out); job_unlock: spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags_job); } @@ -412,37 +416,97 @@ static void v4l2_m2m_cancel_job(struct v4l2_m2m_ctx *m2m_ctx) } } -void v4l2_m2m_job_finish(struct v4l2_m2m_dev *m2m_dev, - struct v4l2_m2m_ctx *m2m_ctx) +/* + * Schedule the next job, called from v4l2_m2m_job_finish() or + * v4l2_m2m_buf_done_and_job_finish(). + */ +static void v4l2_m2m_schedule_next_job(struct v4l2_m2m_dev *m2m_dev, + struct v4l2_m2m_ctx *m2m_ctx) { - unsigned long flags; + /* + * This instance might have more buffers ready, but since we do not + * allow more than one job on the job_queue per instance, each has + * to be scheduled separately after the previous one finishes. + */ + __v4l2_m2m_try_queue(m2m_dev, m2m_ctx); - spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + /* + * We might be running in atomic context, + * but the job must be run in non-atomic context. + */ + schedule_work(&m2m_dev->job_work); +} + +/* + * Assumes job_spinlock is held, called from v4l2_m2m_job_finish() or + * v4l2_m2m_buf_done_and_job_finish(). + */ +static bool _v4l2_m2m_job_finish(struct v4l2_m2m_dev *m2m_dev, + struct v4l2_m2m_ctx *m2m_ctx) +{ if (!m2m_dev->curr_ctx || m2m_dev->curr_ctx != m2m_ctx) { - spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); dprintk("Called by an instance not currently running\n"); - return; + return false; } list_del(&m2m_dev->curr_ctx->queue); m2m_dev->curr_ctx->job_flags &= ~(TRANS_QUEUED | TRANS_RUNNING); wake_up(&m2m_dev->curr_ctx->finished); m2m_dev->curr_ctx = NULL; + return true; +} - spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); - - /* This instance might have more buffers ready, but since we do not - * allow more than one job on the job_queue per instance, each has - * to be scheduled separately after the previous one finishes. */ - __v4l2_m2m_try_queue(m2m_dev, m2m_ctx); +void v4l2_m2m_job_finish(struct v4l2_m2m_dev *m2m_dev, + struct v4l2_m2m_ctx *m2m_ctx) +{ + unsigned long flags; + bool schedule_next; - /* We might be running in atomic context, - * but the job must be run in non-atomic context. + /* + * This function should not be used for drivers that support + * holding capture buffers. Those should use + * v4l2_m2m_buf_done_and_job_finish() instead. */ - schedule_work(&m2m_dev->job_work); + WARN_ON(m2m_ctx->cap_q_ctx.q.subsystem_flags & + VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + schedule_next = _v4l2_m2m_job_finish(m2m_dev, m2m_ctx); + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); + + if (schedule_next) + v4l2_m2m_schedule_next_job(m2m_dev, m2m_ctx); } EXPORT_SYMBOL(v4l2_m2m_job_finish); +void v4l2_m2m_buf_done_and_job_finish(struct v4l2_m2m_dev *m2m_dev, + struct v4l2_m2m_ctx *m2m_ctx, + enum vb2_buffer_state state) +{ + struct vb2_v4l2_buffer *src_buf, *dst_buf; + bool schedule_next = false; + unsigned long flags; + + spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + src_buf = v4l2_m2m_src_buf_remove(m2m_ctx); + dst_buf = v4l2_m2m_next_dst_buf(m2m_ctx); + + if (WARN_ON(!src_buf || !dst_buf)) + goto unlock; + v4l2_m2m_buf_done(src_buf, state); + dst_buf->is_held = src_buf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; + if (!dst_buf->is_held) { + v4l2_m2m_dst_buf_remove(m2m_ctx); + v4l2_m2m_buf_done(dst_buf, state); + } + schedule_next = _v4l2_m2m_job_finish(m2m_dev, m2m_ctx); +unlock: + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); + + if (schedule_next) + v4l2_m2m_schedule_next_job(m2m_dev, m2m_ctx); +} +EXPORT_SYMBOL(v4l2_m2m_buf_done_and_job_finish); + int v4l2_m2m_reqbufs(struct file *file, struct v4l2_m2m_ctx *m2m_ctx, struct v4l2_requestbuffers *reqbufs) { diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h index 0b9c3a287061..229d9f5d4370 100644 --- a/include/media/v4l2-mem2mem.h +++ b/include/media/v4l2-mem2mem.h @@ -21,7 +21,8 @@ * callback. * The job does NOT have to end before this callback returns * (and it will be the usual case). When the job finishes, - * v4l2_m2m_job_finish() has to be called. + * v4l2_m2m_job_finish() or v4l2_m2m_buf_done_and_job_finish() + * has to be called. * @job_ready: optional. Should return 0 if the driver does not have a job * fully prepared to run yet (i.e. it will not be able to finish a * transaction without sleeping). If not provided, it will be @@ -33,7 +34,8 @@ * stop the device safely; e.g. in the next interrupt handler), * even if the transaction would not have been finished by then. * After the driver performs the necessary steps, it has to call - * v4l2_m2m_job_finish() (as if the transaction ended normally). + * v4l2_m2m_job_finish() or v4l2_m2m_buf_done_and_job_finish() as + * if the transaction ended normally. * This function does not have to (and will usually not) wait * until the device enters a state when it can be stopped. */ @@ -173,6 +175,33 @@ void v4l2_m2m_try_schedule(struct v4l2_m2m_ctx *m2m_ctx); void v4l2_m2m_job_finish(struct v4l2_m2m_dev *m2m_dev, struct v4l2_m2m_ctx *m2m_ctx); +/** + * v4l2_m2m_buf_done_and_job_finish() - return source/destination buffers with + * state and inform the framework that a job has been finished and have it + * clean up + * + * @m2m_dev: opaque pointer to the internal data to handle M2M context + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx + * @state: vb2 buffer state passed to v4l2_m2m_buf_done(). + * + * Drivers that set V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF must use this + * function instead of job_finish() to take held buffers into account. It is + * optional for other drivers. + * + * This function removes the source buffer from the ready list and returns + * it with the given state. The same is done for the destination buffer, unless + * it is marked 'held'. In that case the buffer is kept on the ready list. + * + * After that the job is finished (see job_finish()). + * + * This allows for multiple output buffers to be used to fill in a single + * capture buffer. This is typically used by stateless decoders where + * multiple e.g. H.264 slices contribute to a single decoded frame. + */ +void v4l2_m2m_buf_done_and_job_finish(struct v4l2_m2m_dev *m2m_dev, + struct v4l2_m2m_ctx *m2m_ctx, + enum vb2_buffer_state state); + static inline void v4l2_m2m_buf_done(struct vb2_v4l2_buffer *buf, enum vb2_buffer_state state) { From patchwork Fri Oct 11 09:32:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59417 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI4-0001pj-FY; Fri, 11 Oct 2019 09:32:52 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727677AbfJKJcu (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:50 -0400 Received: from lb3-smtp-cloud7.xs4all.net ([194.109.24.31]:42383 "EHLO lb3-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727628AbfJKJct (ORCPT ); Fri, 11 Oct 2019 05:32:49 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHziRKjM; Fri, 11 Oct 2019 11:32:47 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Boris Brezillon , Alexandre Courbot , Hans Verkuil Subject: [PATCHv4 3/6] videodev2.h: add V4L2_DEC_CMD_FLUSH Date: Fri, 11 Oct 2019 11:32:42 +0200 Message-Id: <20191011093245.33366-4-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfFv/7qNpfmVbnIJ/xmr9gUdIdi33yDJQ9mfG03ySrqcndY7GpTCgB697IjgR55ob4jHfPPDO4c9aONuhGHEiQ7rnoWv/4S4l8pcVM924VrlxEGQ+L3dy qnbf4xhH17soiy8VW29pfyad9zvbR0J3fbb9jtxuoQdRYonjlnmay+mx33L/H3gv5xCmWj5c4h1k4573ZkekDZQULtoS7VQNtKSBgz1Y8kBFL1TwKGpnhcKp qrD/+tv6l7eGCbcWccPRazY8E0Z1G0fDQGoz8ceqpe+X4PGFzQW5kq+jQENteQ5t8aD+Hh5FS48J8uqhZPn4UZRC39pwhhnRKJD5GZbIZDVpotH09LbNChU1 XozPKJEMUwAgF5o4FFLY8P03axec5Ca1cZMI+y+pAoSASJ8bOZqKiwCmj6E7LwhuGsE0ffageiKKJw03UoYy5LrE8yOLERAtYQKvZc6CRBBOwFsmLmoMm9n0 RyAK3P2F3ejX2cfHM6BiaNDKs0tfsDQPrWEwNzdGVH+uQhEysr344PtoCQ8WX8B0cBYbm21P0mSUTyyzqC1sPwX90+oUXa6eOyUOUQ== Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add this new V4L2_DEC_CMD_FLUSH decoder command and document it. Reviewed-by: Boris Brezillon Reviewed-by: Alexandre Courbot Signed-off-by: Hans Verkuil [Adjusted description] Signed-off-by: Jernej Skrabec --- Documentation/media/uapi/v4l/vidioc-decoder-cmd.rst | 10 +++++++++- Documentation/media/videodev2.h.rst.exceptions | 1 + include/uapi/linux/videodev2.h | 1 + 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/media/uapi/v4l/vidioc-decoder-cmd.rst b/Documentation/media/uapi/v4l/vidioc-decoder-cmd.rst index 57f0066f4cff..f1a504836f31 100644 --- a/Documentation/media/uapi/v4l/vidioc-decoder-cmd.rst +++ b/Documentation/media/uapi/v4l/vidioc-decoder-cmd.rst @@ -208,7 +208,15 @@ introduced in Linux 3.3. They are, however, mandatory for stateful mem2mem decod been started yet, the driver will return an ``EPERM`` error code. When the decoder is already running, this command does nothing. No flags are defined for this command. - + * - ``V4L2_DEC_CMD_FLUSH`` + - 4 + - Flush any held capture buffers. Only valid for stateless decoders. + This command is typically used when the application reached the + end of the stream and the last output buffer had the + ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag set. This would prevent + dequeueing the capture buffer containing the last decoded frame. + So this command can be used to explicitly flush that final decoded + frame. This command does nothing if there are no held capture buffers. Return Value ============ diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions index b58e381bdf7b..c23e5ef30c78 100644 --- a/Documentation/media/videodev2.h.rst.exceptions +++ b/Documentation/media/videodev2.h.rst.exceptions @@ -435,6 +435,7 @@ replace define V4L2_DEC_CMD_START decoder-cmds replace define V4L2_DEC_CMD_STOP decoder-cmds replace define V4L2_DEC_CMD_PAUSE decoder-cmds replace define V4L2_DEC_CMD_RESUME decoder-cmds +replace define V4L2_DEC_CMD_FLUSH decoder-cmds replace define V4L2_DEC_CMD_START_MUTE_AUDIO decoder-cmds replace define V4L2_DEC_CMD_PAUSE_TO_BLACK decoder-cmds diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 9f4e66affac4..d969842bbfe2 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1984,6 +1984,7 @@ struct v4l2_encoder_cmd { #define V4L2_DEC_CMD_STOP (1) #define V4L2_DEC_CMD_PAUSE (2) #define V4L2_DEC_CMD_RESUME (3) +#define V4L2_DEC_CMD_FLUSH (4) /* Flags for V4L2_DEC_CMD_START */ #define V4L2_DEC_CMD_START_MUTE_AUDIO (1 << 0) From patchwork Fri Oct 11 09:32:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59420 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI7-0001pj-BD; Fri, 11 Oct 2019 09:32:55 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727693AbfJKJcv (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:51 -0400 Received: from lb1-smtp-cloud7.xs4all.net ([194.109.24.24]:41959 "EHLO lb1-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727629AbfJKJcu (ORCPT ); Fri, 11 Oct 2019 05:32:50 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHziRKjh; Fri, 11 Oct 2019 11:32:47 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Hans Verkuil Subject: [PATCHv4 4/6] media: v4l2-mem2mem: add stateless_(try_)decoder_cmd ioctl helpers Date: Fri, 11 Oct 2019 11:32:43 +0200 Message-Id: <20191011093245.33366-5-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfFv/7qNpfmVbnIJ/xmr9gUdIdi33yDJQ9mfG03ySrqcndY7GpTCgB697IjgR55ob4jHfPPDO4c9aONuhGHEiQ7rnoWv/4S4l8pcVM924VrlxEGQ+L3dy qnbf4xhH17soiy8VW29pfyad9zvbR0J3fbb9jtxuoQdRYonjlnmay+mx33L/H3gv5xCmWj5c4h1k41m2Tsigb40BP5Vg78WSoP7419I/qum8VgjJhuKPQjRi jwuq5xlDzGP+BM9B2lFR3cZE6NA2IQtS1GX+QOdViC1TzYluBVjwU6JQa2eY3wumNYT/FhTdc4OFhz1P35hC5TIthxgB0Q047qfUteJJ+dDhIokqxyzKH1uB ndrhrBUeNslspSbo2b6tWOJ0rQbNLLJLDwdgau6m6Ihy6NUZUZFHTUToMscci/8ahnir24A3QRc94tQQvXroC+NA2K4/DyS806T881DbD+r2BVp0vjmHrP3x LaIs/YVYpO95jPGC Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jernej Skrabec These helpers are used by stateless codecs when they support multiple slices per frame and hold capture buffer flag is set. It's expected that all such codecs will use this code. Signed-off-by: Jernej Skrabec Co-developed-by: Hans Verkuil Signed-off-by: Hans Verkuil --- drivers/media/v4l2-core/v4l2-mem2mem.c | 53 ++++++++++++++++++++++++++ include/media/v4l2-mem2mem.h | 4 ++ 2 files changed, 57 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 79c3656f24f7..b46d2c388349 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -1218,6 +1218,59 @@ int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh, } EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_try_decoder_cmd); +int v4l2_m2m_ioctl_stateless_try_decoder_cmd(struct file *file, void *fh, + struct v4l2_decoder_cmd *dc) +{ + if (dc->cmd != V4L2_DEC_CMD_FLUSH) + return -EINVAL; + + dc->flags = 0; + + return 0; +} +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_stateless_try_decoder_cmd); + +int v4l2_m2m_ioctl_stateless_decoder_cmd(struct file *file, void *priv, + struct v4l2_decoder_cmd *dc) +{ + struct v4l2_fh *fh = file->private_data; + struct vb2_v4l2_buffer *out_vb, *cap_vb; + struct v4l2_m2m_dev *m2m_dev = fh->m2m_ctx->m2m_dev; + unsigned long flags; + int ret; + + ret = v4l2_m2m_ioctl_stateless_try_decoder_cmd(file, priv, dc); + if (ret < 0) + return ret; + + spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + out_vb = v4l2_m2m_last_src_buf(fh->m2m_ctx); + cap_vb = v4l2_m2m_last_dst_buf(fh->m2m_ctx); + + /* + * If there is an out buffer pending, then clear any HOLD flag. + * + * By clearing this flag we ensure that when this output + * buffer is processed any held capture buffer will be released. + */ + if (out_vb) { + out_vb->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; + } else if (cap_vb && cap_vb->is_held) { + /* + * If there were no output buffers, but there is a + * capture buffer that is held, then release that + * buffer. + */ + cap_vb->is_held = false; + v4l2_m2m_dst_buf_remove(fh->m2m_ctx); + v4l2_m2m_buf_done(cap_vb, VB2_BUF_STATE_DONE); + } + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); + + return 0; +} +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_stateless_decoder_cmd); + /* * v4l2_file_operations helpers. It is assumed here same lock is used * for the output and the capture buffer queue. diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h index 229d9f5d4370..3d9e48ed8817 100644 --- a/include/media/v4l2-mem2mem.h +++ b/include/media/v4l2-mem2mem.h @@ -701,6 +701,10 @@ int v4l2_m2m_ioctl_try_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *ec); int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *dc); +int v4l2_m2m_ioctl_stateless_try_decoder_cmd(struct file *file, void *fh, + struct v4l2_decoder_cmd *dc); +int v4l2_m2m_ioctl_stateless_decoder_cmd(struct file *file, void *priv, + struct v4l2_decoder_cmd *dc); int v4l2_m2m_fop_mmap(struct file *file, struct vm_area_struct *vma); __poll_t v4l2_m2m_fop_poll(struct file *file, poll_table *wait); From patchwork Fri Oct 11 09:32:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59419 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI6-0001pj-CS; Fri, 11 Oct 2019 09:32:54 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727690AbfJKJcu (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:50 -0400 Received: from lb1-smtp-cloud7.xs4all.net ([194.109.24.24]:42969 "EHLO lb1-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727646AbfJKJct (ORCPT ); Fri, 11 Oct 2019 05:32:49 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHziRKk3; Fri, 11 Oct 2019 11:32:47 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Hans Verkuil Subject: [PATCHv4 5/6] v4l2-mem2mem: add new_frame detection Date: Fri, 11 Oct 2019 11:32:44 +0200 Message-Id: <20191011093245.33366-6-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfFv/7qNpfmVbnIJ/xmr9gUdIdi33yDJQ9mfG03ySrqcndY7GpTCgB697IjgR55ob4jHfPPDO4c9aONuhGHEiQ7rnoWv/4S4l8pcVM924VrlxEGQ+L3dy qnbf4xhH17soiy8VW29pfyad9zvbR0J3fbb9jtxuoQdRYonjlnmay+mx33L/H3gv5xCmWj5c4h1k41m2Tsigb40BP5Vg78WSoP7419I/qum8VgjJhuKPQjRi jwuq5xlDzGP+BM9B2lFR3cZE6NA2IQtS1GX+QOdViC1TzYluBVjwU6JQa2eY3wumNYT/FhTdc4OFhz1P35hC5TIthxgB0Q047qfUteJJ+dDhIokqxyzKH1uB ndrhrBUeNslspSbo2b6tWOJ0rQbNLLJLDwdgau6m6Ihy6NUZUZFHTUToMscci/8ahnir24A3QRc94tQQvXroC+NA2K4/DyS806T881DbD+r2BVp0vjmHrP3x LaIs/YVYpO95jPGC Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Drivers that support VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF typically want to know if a new frame is started (i.e. the first slice is about to be processed). Add a new_frame bool to v4l2_m2m_ctx and set it accordingly. Signed-off-by: Hans Verkuil --- drivers/media/v4l2-core/v4l2-mem2mem.c | 11 +++++++++-- include/media/v4l2-mem2mem.h | 7 +++++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index b46d2c388349..db07ef3bf3d0 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -319,8 +319,10 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, goto job_unlock; } - if (src && dst && - dst->is_held && dst->vb2_buf.copied_timestamp && + m2m_ctx->new_frame = true; + + if (src && dst && dst->is_held && + dst->vb2_buf.copied_timestamp && dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { dst->is_held = false; v4l2_m2m_dst_buf_remove(m2m_ctx); @@ -333,6 +335,11 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, } } + if (src && dst && (m2m_ctx->cap_q_ctx.q.subsystem_flags & + VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) + m2m_ctx->new_frame = !dst->vb2_buf.copied_timestamp || + dst->vb2_buf.timestamp != src->vb2_buf.timestamp; + if (m2m_dev->m2m_ops->job_ready && (!m2m_dev->m2m_ops->job_ready(m2m_ctx->priv))) { dprintk("Driver not ready\n"); diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h index 3d9e48ed8817..1d85e24791e4 100644 --- a/include/media/v4l2-mem2mem.h +++ b/include/media/v4l2-mem2mem.h @@ -75,6 +75,11 @@ struct v4l2_m2m_queue_ctx { * struct v4l2_m2m_ctx - Memory to memory context structure * * @q_lock: struct &mutex lock + * @new_frame: valid in the device_run callback: if true, then this + * starts a new frame; if false, then this is a new slice + * for an existing frame. This is always true unless + * V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF is set, which + * indicates slicing support. * @m2m_dev: opaque pointer to the internal data to handle M2M context * @cap_q_ctx: Capture (output to memory) queue context * @out_q_ctx: Output (input from memory) queue context @@ -91,6 +96,8 @@ struct v4l2_m2m_ctx { /* optional cap/out vb2 queues lock */ struct mutex *q_lock; + bool new_frame; + /* internal use only */ struct v4l2_m2m_dev *m2m_dev; From patchwork Fri Oct 11 09:32:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans Verkuil X-Patchwork-Id: 59418 X-Patchwork-Delegate: hverkuil@xs4all.nl Received: from vger.kernel.org ([209.132.180.67]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iIrI5-0001pj-E4; Fri, 11 Oct 2019 09:32:53 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727678AbfJKJcu (ORCPT + 1 other); Fri, 11 Oct 2019 05:32:50 -0400 Received: from lb1-smtp-cloud7.xs4all.net ([194.109.24.24]:43157 "EHLO lb1-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727649AbfJKJct (ORCPT ); Fri, 11 Oct 2019 05:32:49 -0400 Received: from marune.fritz.box ([IPv6:2001:983:e9a7:1:611e:26b3:263e:a49f]) by smtp-cloud7.xs4all.net with ESMTPA id IrHxixmWhjZ8vIrHziRKkO; Fri, 11 Oct 2019 11:32:48 +0200 From: Hans Verkuil To: linux-media@vger.kernel.org Cc: =?utf-8?q?Jernej_=C5=A0krabec?= , Paul Kocialkowski , mripard@kernel.org, tfiga@chromium.org, jonas@kwiboo.se, Ezequiel Garcia , Hans Verkuil Subject: [PATCHv4 6/6] media: cedrus: h264: Support multiple slices per frame Date: Fri, 11 Oct 2019 11:32:45 +0200 Message-Id: <20191011093245.33366-7-hverkuil-cisco@xs4all.nl> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> References: <20191011093245.33366-1-hverkuil-cisco@xs4all.nl> MIME-Version: 1.0 X-CMAE-Envelope: MS4wfBzmwk2wZ2r7Pj+6rXYHDLmxi/ZQLcXEtL9ge0rZoZsrUq1qjsnbssu/lhUUm5/EeUAhs9fKzJkr1MtFfE2YjG/dw+69ohPp89pjpDNh7EC58zf2PEz4 oijuRk881nVL0wS0Ij98MedtpypoHtDwi+j6D/Dj7df9LN51dUbSBWksP9NbBLl4y70idbxEeATlyLBMhV4pTUnaz60qrZPWJNzPe0XUBSQCOGvFguejwg6Z dVk34iA/gPTYKrKp2nA+DpT1b5/D5nkAKF1LHOXmj21sBFAFeo7kVGTUdBjZBYIyssvqI95tsG7ief50KkBV76Y7lZzgfRq8K9KqymcG9SgGIPYASDv2Cnm9 7MrPaIZoGluQLmqZeQL+rr8hSV797xIOAOmRh3wTdpbY3tGn5gKl+C+wnxqXMbycH8M6FWCCg71A19cBbtFH+qJk+txgDXS6eBUWgSIXrtWpdhAIESGdDQuh caoHGjPr6dTGRMxh Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jernej Skrabec With recent changes, support for decoding multi-slice frames can be easily added now. Signal VPU if current slice is first in frame or not and add information about first macroblock coordinates. When frame contains multiple slices and driver works in slice mode, it's more efficient to hold capture buffer in queue until all slices of a same frame are decoded. Add support for that to Cedrus driver by exposing and implementing V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF capability. Signed-off-by: Jernej Skrabec [hverkuil-cisco@xs4all.nl: rewritten to use v4l2_m2m_buf_done_and_job_finish] [hverkuil-cisco@xs4all.nl: removed unnecessary (u32) cast] [hverkuil-cisco@xs4all.nl: use new_frame v4l2_m2m_ctx bool] Signed-off-by: Hans Verkuil --- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 12 +++++++++++- drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 16 ++-------------- .../staging/media/sunxi/cedrus/cedrus_video.c | 14 ++++++++++++++ 3 files changed, 27 insertions(+), 15 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index d6a782703c9b..cd85668f9c80 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -301,6 +301,8 @@ static void cedrus_set_params(struct cedrus_ctx *ctx, dma_addr_t src_buf_addr; u32 offset = slice->header_bit_size; u32 len = (slice->size * 8) - offset; + unsigned int pic_width_in_mbs; + bool mbaff_pic; u32 reg; cedrus_write(dev, VE_H264_VLD_LEN, len); @@ -370,12 +372,20 @@ static void cedrus_set_params(struct cedrus_ctx *ctx, reg |= VE_H264_SPS_DIRECT_8X8_INFERENCE; cedrus_write(dev, VE_H264_SPS, reg); + mbaff_pic = !(slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) && + (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD); + pic_width_in_mbs = sps->pic_width_in_mbs_minus1 + 1; + // slice parameters reg = 0; + reg |= ((slice->first_mb_in_slice % pic_width_in_mbs) & 0xff) << 24; + reg |= (((slice->first_mb_in_slice / pic_width_in_mbs) * + (mbaff_pic + 1)) & 0xff) << 16; reg |= decode->nal_ref_idc ? BIT(12) : 0; reg |= (slice->slice_type & 0xf) << 8; reg |= slice->cabac_init_idc & 0x3; - reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC; + if (ctx->fh.m2m_ctx->new_frame) + reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC; if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) reg |= VE_H264_SHS_FIELD_PIC; if (slice->flags & V4L2_H264_SLICE_FLAG_BOTTOM_FIELD) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c index a942cd9bed57..e7e18424bab1 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c @@ -103,7 +103,6 @@ static irqreturn_t cedrus_irq(int irq, void *data) { struct cedrus_dev *dev = data; struct cedrus_ctx *ctx; - struct vb2_v4l2_buffer *src_buf, *dst_buf; enum vb2_buffer_state state; enum cedrus_irq_status status; @@ -121,24 +120,13 @@ static irqreturn_t cedrus_irq(int irq, void *data) dev->dec_ops[ctx->current_codec]->irq_disable(ctx); dev->dec_ops[ctx->current_codec]->irq_clear(ctx); - src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); - dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); - - if (!src_buf || !dst_buf) { - v4l2_err(&dev->v4l2_dev, - "Missing source and/or destination buffers\n"); - return IRQ_HANDLED; - } - if (status == CEDRUS_IRQ_ERROR) state = VB2_BUF_STATE_ERROR; else state = VB2_BUF_STATE_DONE; - v4l2_m2m_buf_done(src_buf, state); - v4l2_m2m_buf_done(dst_buf, state); - - v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx); + v4l2_m2m_buf_done_and_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx, + state); return IRQ_HANDLED; } diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index 3ec3a2db790c..f745f66c4440 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -303,6 +303,17 @@ static int cedrus_s_fmt_vid_out(struct file *file, void *priv, ctx->src_fmt = f->fmt.pix; + switch (ctx->src_fmt.pixelformat) { + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= + VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + vq->subsystem_flags &= + ~VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + } + /* Propagate colorspace information to capture. */ ctx->dst_fmt.colorspace = f->fmt.pix.colorspace; ctx->dst_fmt.xfer_func = f->fmt.pix.xfer_func; @@ -336,6 +347,9 @@ const struct v4l2_ioctl_ops cedrus_ioctl_ops = { .vidioc_streamon = v4l2_m2m_ioctl_streamon, .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_stateless_try_decoder_cmd, + .vidioc_decoder_cmd = v4l2_m2m_ioctl_stateless_decoder_cmd, + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, .vidioc_unsubscribe_event = v4l2_event_unsubscribe, };