From patchwork Thu Jul 17 10:45:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116014 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM7a-00059d-32 for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:45:52 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 24A2B560B5F for ; Thu, 17 Jul 2025 10:45:50 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E5AC928B516; Thu, 17 Jul 2025 10:45:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e6/zFHNN" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C704127EFEF; Thu, 17 Jul 2025 10:45:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749146; cv=none; b=QFG+S3rJ6OMPXc1BZcJCQl09VYHMnTcUER3tNgS78NXrCbycuJXKCy9DZ+thJXBm0JhYK7vaTX/auYQOevU0+tmd1LvpkaEwknccQj1YVtyldehFTEg6V10hM7fofzZ/cTv0EWIyViamujgEtEKVOPJ00CgFZI9KWbXbWlIDjxM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749146; c=relaxed/simple; bh=kncEHGUHBsOPEbj18mUjhjLaFdDTloHadGxnvcBRhXM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qN9PCXjzSk+Ei8FUzzNNAxBWBnqyvfKrI4M3sUWKlgBPVe8xsUgXCANzrM5cBIl+W6KO+Py49GfDCcGbvtzFoTMxQOfvZYySrp+yXiE1A73kp5ezVoO3eoOUnrzdvRxaV8DQV3BHGM1KvZBRU6SC74W8eh4huyndVEldDOSnlm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=e6/zFHNN; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3928A1FA4; Thu, 17 Jul 2025 12:45:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749107; bh=kncEHGUHBsOPEbj18mUjhjLaFdDTloHadGxnvcBRhXM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=e6/zFHNNGGMuH9U9zuz8Y32+ZXvmPUdFLcgoaUGTzzI7yudysa2ihXzwZ0ROlRTlg VtzvEKB/c3xWVUziYbinIGVaZztT69TdZmXTe6Hpb6GzwdOerHBlL+sZJJK0Tmibs/ uzKT1zKYrc61H11U42cD4rnoSEyJIsULmvDVrhf8= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:27 +0200 Subject: [PATCH 01/26] media: mc: Add per-file-handle data support Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-1-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi , Laurent Pinchart , Hans Verkuil X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7105; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=iZLPf77dc0oKef/FoIrNYxuY8wrefkX+9mbiaN7w1+U=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRObO3WXVK5YuwGNfryMHoO0eT2HGz8KOkkO MoxoOP2EUWJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTgAKCRByNAaPFqFW PB3qD/sHu7AvE8eTIOkPKtlkTHw4fCK4Ra3kALeaaL6nydbMc5ghfVmAgfX8uSjA1Sii5UBXIr1 WW12rKwYIpjl4VM1OSCQ/aA2pKCksaJReYzHumMCR5E47ocMHEsH1S/07zXiMpy85pCMUn/JnBP vsjzArG/VzsGi+hgcG0KHxFpYKx0dncGf9ZR+skWqO+OctpySVJwPfzMkzGXhixn45qal+xzLBX dI56Mr67KoUr1Grw6UH8S11LZ5KcvfZZ1qydxPeVV72dY6IkcbaSto6w72pKW7m2n3h0r+q6UMb jnLV0RUimUfEkaMOEPsmpgKIDvCazpP4VahoLZ0Vd/e+1WkygfkSbS7Jdja6szY3UNfLxFNRjpy qt6B6FdUjyUnanVZ4LWJJDzfIChBHLoojJXO50eEHTuCXcH3ybNOxxFpvqu1u6AAwpb2b+iuUc3 w4P+ZZRcZ8N2VlcuwtbyumX7x1yIlTEQqks+H5F3pTL3jXLgXO6WEH1Y5xmrldia70zceJOBNAW XFGPcTraBODKmoCO3aH7IKamJ5Q8wJhvhVFur5pE+3eINwSTILE90fdLdw510CtbpdYkslq1d2h ekSePo711T/X5zejFK9dzGnjBBmMItdhIKirYJrb3TR1la23SkplncwTJ6B+FJFI3/OOVy6V6Jc uh1xtgaem0c0Oyw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no From: Laurent Pinchart The media devnode core associates devnodes with files by storing the devnode pointer in the file structure private_data field. In order to allow tracking of per-file-handle data introduce a new media devnode file handle structure that stores the devnode pointer, and store a pointer to that structure in the file private_data field. Users of the media devnode code (the only existing user being media_device) are responsible for managing their own subclass of the media_devnode_fh structure. Signed-off-by: Laurent Pinchart Prepare struct media_device_fh to be used for maintaining file handle list to avoid shuffling things here and there right after. Signed-off-by: Sakari Ailus Acked-by: Hans Verkuil Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 14 +++++++++++++- drivers/media/mc/mc-devnode.c | 20 +++++++++----------- include/media/media-device.h | 7 +++++++ include/media/media-devnode.h | 18 +++++++++++++++++- include/media/media-fh.h | 32 ++++++++++++++++++++++++++++++++ 5 files changed, 78 insertions(+), 13 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index c0dd4ae5722725f1744bc6fd6282d5c765438059..f298f8f67b8a84194d66126461e84228c0b20394 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #ifdef CONFIG_MEDIA_CONTROLLER @@ -35,7 +36,6 @@ #define MEDIA_ENT_SUBTYPE_MASK 0x0000ffff #define MEDIA_ENT_T_DEVNODE_UNKNOWN (MEDIA_ENT_F_OLD_BASE | \ MEDIA_ENT_SUBTYPE_MASK) - /* ----------------------------------------------------------------------------- * Userspace API */ @@ -47,11 +47,23 @@ static inline void __user *media_get_uptr(__u64 arg) static int media_device_open(struct file *filp) { + struct media_device_fh *fh; + + fh = kzalloc(sizeof(*fh), GFP_KERNEL); + if (!fh) + return -ENOMEM; + + filp->private_data = &fh->fh; + return 0; } static int media_device_close(struct file *filp) { + struct media_device_fh *fh = media_device_fh(filp); + + kfree(fh); + return 0; } diff --git a/drivers/media/mc/mc-devnode.c b/drivers/media/mc/mc-devnode.c index 56444edaf13651874331e7c04e86b0a585067d38..312eb48ffc2f2a0c013e4744204995df0ff5b12c 100644 --- a/drivers/media/mc/mc-devnode.c +++ b/drivers/media/mc/mc-devnode.c @@ -142,6 +142,7 @@ static long media_compat_ioctl(struct file *filp, unsigned int cmd, static int media_open(struct inode *inode, struct file *filp) { struct media_devnode *devnode; + struct media_devnode_fh *fh; int ret; /* Check if the media device is available. This needs to be done with @@ -162,17 +163,15 @@ static int media_open(struct inode *inode, struct file *filp) get_device(&devnode->dev); mutex_unlock(&media_devnode_lock); - filp->private_data = devnode; - - if (devnode->fops->open) { - ret = devnode->fops->open(filp); - if (ret) { - put_device(&devnode->dev); - filp->private_data = NULL; - return ret; - } + ret = devnode->fops->open(filp); + if (ret) { + put_device(&devnode->dev); + return ret; } + fh = filp->private_data; + fh->devnode = devnode; + return 0; } @@ -181,8 +180,7 @@ static int media_release(struct inode *inode, struct file *filp) { struct media_devnode *devnode = media_devnode_data(filp); - if (devnode->fops->release) - devnode->fops->release(filp); + devnode->fops->release(filp); filp->private_data = NULL; diff --git a/include/media/media-device.h b/include/media/media-device.h index 53d2a16a70b0d9d6e5cc28fe1fc5d5ef384410d5..2fc750efef7c43814f019f12078e9c96c1bd6bf9 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -108,6 +108,10 @@ struct media_device_ops { * other operations that stop or start streaming. * @request_id: Used to generate unique request IDs * + * @fh_list: List of file handles in the media device + * (struct media_device_fh.mdev_list). + * @fh_list_lock: Serialise access to fh_list list. + * * This structure represents an abstract high-level media device. It allows easy * access to entities and provides basic media device-level support. The * structure can be allocated directly or embedded in a larger structure. @@ -180,6 +184,9 @@ struct media_device { struct mutex req_queue_mutex; atomic_t request_id; + + struct list_head fh_list; + spinlock_t fh_list_lock; }; /* We don't need to include usb.h here */ diff --git a/include/media/media-devnode.h b/include/media/media-devnode.h index d27c1c646c2805171be3997d72210dd4d1a38e32..6c2e253dde498779dffd103dc5d00e50e14a0249 100644 --- a/include/media/media-devnode.h +++ b/include/media/media-devnode.h @@ -55,6 +55,20 @@ struct media_file_operations { int (*release) (struct file *); }; +/** + * struct media_devnode_fh - Media device node file handle + * @devnode: pointer to the media device node + * + * This structure serves as a base for per-file-handle data storage. Media + * device node users embed media_devnode_fh in their custom file handle data + * structures and store the media_devnode_fh in the file private_data in order + * to let the media device node core locate the media_devnode corresponding to a + * file handle. + */ +struct media_devnode_fh { + struct media_devnode *devnode; +}; + /** * struct media_devnode - Media device node * @media_dev: pointer to struct &media_device @@ -146,7 +160,9 @@ void media_devnode_unregister(struct media_devnode *devnode); */ static inline struct media_devnode *media_devnode_data(struct file *filp) { - return filp->private_data; + struct media_devnode_fh *fh = filp->private_data; + + return fh->devnode; } /** diff --git a/include/media/media-fh.h b/include/media/media-fh.h new file mode 100644 index 0000000000000000000000000000000000000000..6f00744b81d6000a4b0c503fe6968dd7adcbb1c3 --- /dev/null +++ b/include/media/media-fh.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Media device file handle + * + * Copyright (C) 2019--2023 Intel Corporation + */ + +#ifndef MEDIA_FH_H +#define MEDIA_FH_H + +#include +#include + +#include + +/** + * struct media_device_fh - File handle specific information on MC + * + * @fh: The media device file handle + * @mdev_list: This file handle in media device's list of file handles + */ +struct media_device_fh { + struct media_devnode_fh fh; + struct list_head mdev_list; +}; + +static inline struct media_device_fh *media_device_fh(struct file *filp) +{ + return container_of(filp->private_data, struct media_device_fh, fh); +} + +#endif /* MEDIA_FH_H */ From patchwork Thu Jul 17 10:45:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116017 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM84-0005BD-2l for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:46:22 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7E349560E98 for ; Thu, 17 Jul 2025 10:46:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2C1341F9F73; Thu, 17 Jul 2025 10:45:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N3JWhyUJ" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 571B429ACC0; Thu, 17 Jul 2025 10:45:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749149; cv=none; b=qnNpvs9Xw5pTPavLZTne3E6oU+Qs3y+CshvGWiNVge4MQnPKzeRxMyYvuHALQxQ5DWxrhcG1aRqEQr7teCDh2xy969VGX8ZXHdYd7sXwiaipaBiMSgARGQXB6OaCb7Mvxw994HPbL/P945lgDZiJMaz2wEQUkiExC7sF8RMRYK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749149; c=relaxed/simple; bh=zDLiKpcarRebLqHSrXV+LHH8PWDYb0+SfoPVzO8Rjbg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CIOjCXfgi103HBNMtYG+9phKyDeGm6elncwLTMZrlGVRHoWQxFbtrsHMivTbbSGAG9rO8ZwiIKIirXLeilX8t3yJAOLNRcMRi76vbAog1lZ4wa6RL2Jb7Om782NuLhzGxNztTmVNAHVxQ/9bFTE2RswE4j4hn5QnZlO6tysdgnU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=N3JWhyUJ; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A729221F2; Thu, 17 Jul 2025 12:45:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749108; bh=zDLiKpcarRebLqHSrXV+LHH8PWDYb0+SfoPVzO8Rjbg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=N3JWhyUJPHkJFR6fywQTP3lfTnJe2j3ASCC7Qf+2GRr7QmUmuLjDgFQH4HRtmXf8r zdoRQ4BwUz/8YodGiayWyswz35rIdCagMvd2YsXqJadFd1BelvJULilRvLywc3CJ5j V0LQzqEdi7FmnDbAmDQaSOOW1dncTZ+an6/BS7w8= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:28 +0200 Subject: [PATCH 02/26] media: mc: Maintain a list of open file handles in a media device Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-2-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3849; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=/rqN2PeYLVGrC1gEE4q9OmKFqZ01RUXwuvcLLsZvWlg=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRODvjJd3XMj2tWRmbA5TTdq+0kdqmzKby+D FpwmIcXgoeJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTgAKCRByNAaPFqFW PN9kEADHAaupDlJBw4zoO05gDJC4g+2dUdVBtaKJ2IjklJUtSyewhQLVvyCIeIZ06l6FiAGmRxt vNWuVUDBpzFcvC3mM0AnlCf0CLyuOM7VOS5Vor0if88GN/Hj0TGxfHplOsFx7itbYITAsFliNJD U/dOb/2y1+Kgq8cn+zVIoI6uExDm66LOtEkCSotlBV/M0VKuLYIDMaA0Gqt/dd/odef91JtAHkN M5SFhvCGByUDqyf23dVkVHhI8nz0S9fvPuwhMcJlY6C9qrSRJFBSynISdRtf34mhdHk1Xh/CoYt nCISqlU6WAIwf+EXs7LQeTJe8pNaRw2dwsGNxgtdI6r/DmqcvBm0xczMQpXmAZ9uHe8ryOym+Fh yRHhdlywq//+IwXTUwTD/IcMe9k6qnKn+uIb1ofjkz1czqKWD57zAR2mSLkVb0zhdXYO/L75zmI ol8VsoiVKdA3FfvanFGSsLhCOmkpA8ikYbOgJk9TB625InFA9nL7JmFW+p0eYXFt+bop2odETIQ jOXFxDBowgbL3mpI+Ro3NNUSQXV2oeScvhEzN2UksG2NU4EY4aZ17bAfWg1gVayRt1QEBUjOaCR rvr6PcZbAAMQc0/82aZTUZEtQ/S8E7b+nhkzJyVnjCJuK+fmF120eet7jkQSqhLeTdwJfeTKvNN 8rQjm6Voijtk1Fw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no From: Sakari Ailus The list of file handles is needed to deliver media events as well as for other purposes in the future. Signed-off-by: Sakari Ailus [Access media_device with devnode->media_dev] Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 19 ++++++++++++++++++- drivers/media/mc/mc-devnode.c | 2 +- include/media/media-devnode.h | 3 ++- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index f298f8f67b8a84194d66126461e84228c0b20394..e0cad87087d3863bf14207049a54e5e4dea1cdd4 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -45,8 +45,9 @@ static inline void __user *media_get_uptr(__u64 arg) return (void __user *)(uintptr_t)arg; } -static int media_device_open(struct file *filp) +static int media_device_open(struct media_devnode *devnode, struct file *filp) { + struct media_device *mdev = devnode->media_dev; struct media_device_fh *fh; fh = kzalloc(sizeof(*fh), GFP_KERNEL); @@ -55,13 +56,23 @@ static int media_device_open(struct file *filp) filp->private_data = &fh->fh; + spin_lock_irq(&mdev->fh_list_lock); + list_add(&fh->mdev_list, &mdev->fh_list); + spin_unlock_irq(&mdev->fh_list_lock); + return 0; } static int media_device_close(struct file *filp) { + struct media_devnode *devnode = media_devnode_data(filp); + struct media_device *mdev = devnode->media_dev; struct media_device_fh *fh = media_device_fh(filp); + spin_lock_irq(&mdev->fh_list_lock); + list_del(&fh->mdev_list); + spin_unlock_irq(&mdev->fh_list_lock); + kfree(fh); return 0; @@ -698,10 +709,12 @@ void media_device_init(struct media_device *mdev) INIT_LIST_HEAD(&mdev->pads); INIT_LIST_HEAD(&mdev->links); INIT_LIST_HEAD(&mdev->entity_notify); + INIT_LIST_HEAD(&mdev->fh_list); mutex_init(&mdev->req_queue_mutex); mutex_init(&mdev->graph_mutex); ida_init(&mdev->entity_internal_idx); + spin_lock_init(&mdev->fh_list_lock); atomic_set(&mdev->request_id, 0); @@ -809,6 +822,10 @@ void media_device_unregister(struct media_device *mdev) return; } + spin_lock_irq(&mdev->fh_list_lock); + list_del_init(&mdev->fh_list); + spin_unlock_irq(&mdev->fh_list_lock); + /* Clear the devnode register bit to avoid races with media dev open */ media_devnode_unregister_prepare(mdev->devnode); diff --git a/drivers/media/mc/mc-devnode.c b/drivers/media/mc/mc-devnode.c index 312eb48ffc2f2a0c013e4744204995df0ff5b12c..50435f102aa7cd6a7cde759eeef73a99b9b80239 100644 --- a/drivers/media/mc/mc-devnode.c +++ b/drivers/media/mc/mc-devnode.c @@ -163,7 +163,7 @@ static int media_open(struct inode *inode, struct file *filp) get_device(&devnode->dev); mutex_unlock(&media_devnode_lock); - ret = devnode->fops->open(filp); + ret = devnode->fops->open(devnode, filp); if (ret) { put_device(&devnode->dev); return ret; diff --git a/include/media/media-devnode.h b/include/media/media-devnode.h index 6c2e253dde498779dffd103dc5d00e50e14a0249..26b19373c6646bfd11cfded220c9e61c81130580 100644 --- a/include/media/media-devnode.h +++ b/include/media/media-devnode.h @@ -22,6 +22,7 @@ #include struct media_device; +struct media_devnode; /* * Flag to mark the media_devnode struct as registered. Drivers must not touch @@ -51,7 +52,7 @@ struct media_file_operations { __poll_t (*poll) (struct file *, struct poll_table_struct *); long (*ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); - int (*open) (struct file *); + int (*open) (struct media_devnode *, struct file *); int (*release) (struct file *); }; From patchwork Thu Jul 17 10:45:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116020 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM86-0005BK-2J for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:46:23 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 0D0BBA486F3 for ; Thu, 17 Jul 2025 10:45:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 503FA29B228; Thu, 17 Jul 2025 10:45:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UmMTSjqp" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69C5C29ACC8; Thu, 17 Jul 2025 10:45:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749150; cv=none; b=ezvzy+jx/VLopkaeqdIkQ52ZQnCZeixiFpSpXS0rgNsm8KaK8yAFE85rJDTNpBK2pTZC/hmaw/1BUoCHYT4Pc6rrQf2zdpPyJIFAlwx88HAR/ZY6epzFnNwnh0asGF9CUHS8nCNf38AtXBSsetsVjTn065mnK68fYkdUI+hjwUU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749150; c=relaxed/simple; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aiK7AIRMfBeF8vx6181uc+9PTBfPTbWxCfgdlkUefYKXrllrJO80C08fLXjxAPGgeYhOL4Rlzt0JqbyuvCyWJxMf7dT623rcPNCmtfnu/zpKFVyBMzg0f2WBYcsXwqj5BPJzJ8vGQfp2Q1kcNJqYHt2FPXuz/C4QR8wpggngvsk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=UmMTSjqp; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2334920EE; Thu, 17 Jul 2025 12:45:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749110; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=UmMTSjqpvuVK3Q9KhJ/6vo6FB2X7TmnS9CtKSLhWRQWZkt9vsA1wVHbZOThrW7dhz IhhdheGJKW8JWVr1A2GaERsmxTUZv78WloBiw4NxFgKEclyMQYv5i+sdFiRtexJAZH lziStg4Yo7dms0xsYVljmvVKFq0eT2rBh4QE/c9U= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:29 +0200 Subject: [PATCH 03/26] media: media-entity: Introduce media_entity_context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-3-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7515; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=MQRwA7fqy++hrn9wDtM90iaXfefqzRiYRYRIsmH0eBo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRP61XYyB838338Dlg46RgQigjlhUEOJLdzh pII+hfKMCSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PI2dEACya3+bVHv1j5Q2ClKKwUIQDW7LYn3klEs1T+ShPGyyx4nq8E5dcUCsVwIxWxozjM7CUoD smQnWh4p2o9Spj5d0CBlpbwE3Lbwb3MgCdy714aRGAVEBiZ21gdBPX0oGsqZsj0ypzH3Zv/ShA0 aEhfY+C2MYh3DELVyD7jTOqXH5qxv+tJ1FgWzHeow2PfbBwgDKA7XmWptovTY7Z9snAg3bEIb8Z m76uybcPBLDfk/wT0oawiBV8sqP1Ime9lccZPSZ5tcZ+XXnlahPesOCAvTrd2ynoMcZd8wFSJwj SxBQGy0y1O90+ZlZF3KHS6q7ACip6TWG+9JWbAseveDK6ErmPrvLZtKZK9uBnsu5F+vMDMDZIq8 U8iuwMrfglLTNOl+3REOkp0VWAULKQB12uhf6LBjOs1ytGK/l0V7+7V2pvAauXsjB2BVEKsgYDm OtizhgDp40rxeDL1Kx2Q343YbH5JjvwakLgC3Z5BplfsBvQvlNc0ysRz42heI+5ZIhwEgP1CBJ1 GOr0O9+qEljK1wqSCi6uVGWD5P5vtlwPqxAoKPcRzzGIciWb9yXayQryVB4AM7UCtHh0Mly+oY0 SW1PMxXxt8rL4Kdf5wbgwRgpamluIh1JMZ6DfMMrR5nCHHNkiWJh/AuRRcjyRce5129+T81TZWS UHFcHlZr5cbHwWQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce the 'struct media_entity_context' type, which serves for reference counting and introduce two new media entity operations to allow drivers to allocate and free a media entity context. The newly introduced type will be used as a base type for the device context types (video_device_context and v4l2_subdevice_context) that will be introduced in the next patches. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 46 ++++++++++++++++++++ include/media/media-entity.h | 101 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 045590905582054c46656e20463271b1f93fa6b4..b4a9f0a0aa7353d7a3333f20903980956b3df4a7 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1673,3 +1673,49 @@ struct media_link *__media_entity_next_link(struct media_entity *entity, return NULL; } EXPORT_SYMBOL_GPL(__media_entity_next_link); + +static void media_entity_release_context(struct kref *refcount) +{ + struct media_entity_context *ctx = + container_of(refcount, struct media_entity_context, refcount); + + ctx->entity->ops->destroy_context(ctx); +} + +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_entity_context_get); + +void media_entity_context_put(struct media_entity_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_entity_release_context); +} +EXPORT_SYMBOL_GPL(media_entity_context_put); + +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx) +{ + if (!ctx) + return; + + ctx->entity = entity; + kref_init(&ctx->refcount); + INIT_LIST_HEAD(&ctx->list); +} +EXPORT_SYMBOL_GPL(media_entity_init_context); + +void media_entity_cleanup_context(struct media_entity_context *ctx) +{ +} +EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 64cf590b11343f68a456c5870ca2f32917c122f9..32298fe8a18c6ee3c1dbcff9ef869548904417a7 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -248,6 +249,37 @@ struct media_pad { struct media_pipeline *pipe; }; +/** + * struct media_entity_context - A media entity execution context + * @mdev_context: The media device context this media entity is bound to. + * The field is initialized when the entity is bound to a media + * device context. + * @entity: The media entity this context belongs to + * @refcount: The kref reference counter + * list: The list entry to link the entity context in the media device context + * + * This type represent the 'base class' used to implement execution context for + * video device contexts and subdevice contexts. Those types embedds an instance + * of 'struct media_entity_context' as their first member, allowing the MC core + * to implement type polymorphism and handle video device and subdevice contexts + * transparently. + * + * The main function of this type is to provide reference counting for the + * 'dervived' device context types. The video device and subdevice core + * populates the 'context_release' function pointer that implement specific + * clean-up operations, similar to what a 'virtual destructor' would do in C++. + * + * Drivers are not expected to use this type directly, but only the MC core + * will. + */ +struct media_device_context; +struct media_entity_context { + struct media_device_context *mdev_context; + struct media_entity *entity; + struct kref refcount; + struct list_head list; +}; + /** * struct media_entity_operations - Media entity operations * @get_fwnode_pad: Return the pad number based on a fwnode endpoint or @@ -269,6 +301,15 @@ struct media_pad { * media_entity_has_pad_interdep(). * Optional: If the operation isn't implemented all pads * will be considered as interdependent. + * @alloc_context: Allocate a media entity context. Drivers are allowed to + * sub-class the entity context type by defining a driver + * specific type that embeds an instance of either a + * video_device_context or subdevice_context as first + * member, and allocate the size of a driver-specific type + * in the implementation of this operation. Returns 0 for + * success, or an error code < 0 otherwise. + * @destroy_context: Release a media entity context previously allocated by + * the driver. * * .. note:: * @@ -284,6 +325,9 @@ struct media_entity_operations { int (*link_validate)(struct media_link *link); bool (*has_pad_interdep)(struct media_entity *entity, unsigned int pad0, unsigned int pad1); + int (*alloc_context)(struct media_entity *entity, + struct media_entity_context **context); + void (*destroy_context)(struct media_entity_context *context); }; /** @@ -1448,3 +1492,60 @@ struct media_link *__media_entity_next_link(struct media_entity *entity, MEDIA_LNK_FL_DATA_LINK)) #endif + +/** + * media_entity_context_get - Increase the media entity context reference count + * and return a reference to it + * + * @ctx: the media entity context + * + * Increase the media entity context reference count. The reference count + * is increased by the V4L2 core when: + * + * * a new context is allocated when bounding a media entity to a media device + * context (by kref_init()) + * * the media pipeline the context is part of starts streaming + * + * The entity context gets automatically decreased by the V4L2 core when: + * + * * a context is unbound + * * the pipeline stops streaming + */ +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx); + +/** + * media_entity_context_put - Decrease the media entity context reference count + * + * @ctx: the media entity context + * + * Decrease the media entity context reference count. The reference count + * is decreased by the V4L2 core when: + * + * * the file handle the context is associated with is closed + * * the media pipeline the context is part of is stopped + */ +void media_entity_context_put(struct media_entity_context *ctx); + +/** + * media_entity_init_context - Initialize the media entity context + * + * @entity: the media entity this context belongs to + * @ctx: the media entity context + * + * Initialize the media entity context by initializing the kref reference + * counter. The intended caller of this function are the video device context + * and subdevic context initialize functions. + */ +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx); + +/** + * media_entity_cleanup_context - Cleanup the media entity context + * + * @ctx: the media entity context + * + * Cleanup the media entity context. The intended caller of this function are + * the video device and subdevice context cleanup functions. + */ +void media_entity_cleanup_context(struct media_entity_context *ctx); From patchwork Thu Jul 17 10:45:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116026 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM8m-0005Cw-26 for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:47:06 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id C88DE3AD964 for ; Thu, 17 Jul 2025 10:46:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ED81C29CB41; Thu, 17 Jul 2025 10:45:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F55Et+Mw" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8516D29B8CF; Thu, 17 Jul 2025 10:45:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749153; cv=none; b=D5GyT6u0Xl4F+SHBszUXfGqNES4/pPJprPyJuGCnAuOf1HSwdLTO71Lmthy9kKezzKOVTEK2liCD//sVnomB/+fOM0byfEmmwYxp/NJSCcgCG9zg75IxIJ7YkYhIcv7UOQCaGR/g5Lt/pvQR7zChS/W6LBlpZG8rUpbDWs491A0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749153; c=relaxed/simple; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=trGZi9SbKKo+0vN7gUZlCYwcJgHhv0r9GTEBkhuSElgtL6lWdzQtlDRnGRkgKV3AgtziH+zUjCWmLUoCmNB4Q8pmVVfk5aHbGfDGRsaiISLDwHZa0JQjzFuxSNLdMPDZqCw2dczMHovgtAPynuh2nNzOj8vOuxyHz8CSAOzvb3w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=F55Et+Mw; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DA92121F5; Thu, 17 Jul 2025 12:45:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749112; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=F55Et+MwRnhzFqOY8k0h8lgKwQQ7aywHgjho4MlfBO83KKVX1PyWuoG5cZl4EFL1X E0nwS7v6I8EAmkPn5q2uMoA4jvJkn0WpSeQ36sm/m8JIETANcIb03dyZms1FfyfRzr fV/AW9h0ZD8T0FASbiuXWCr6BE0YOb0zcSTAmoz4= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:30 +0200 Subject: [PATCH 04/26] media: media-device: Introduce media device context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-4-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=18790; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Qd19apeMZFSiMBZkqVWpWcHh8AbZaZrN82RvB+0JkQY=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRP5yneztZZ/v+L83FetOPLVE//qA5a3S5dJ G5X/Kgb7P6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW POVVEACqaEO8SLii1FyJjiX7A/NFP8t2dFMnYBpeb/beBT6N6A3TZ0fjG1drXxednclt01ZT9qr JHu0o9M7J61anG6gLHVF4kzku+r37i9ktQJ4wxiYABUTb3gGxl915kz0z7H3NQb6EeOgoXWqACP Q7+lKQ1BUnrtgj5T18OhPVTUwCBVvpkf0DAXuYV3HJx/zFGSU4bMe5wc/q0TSDfbBTYRDP32xEM C54W411RkNDjlQDVHKyHscJ9oQXU5wzKv1BUflVgmPDK1+hn3CSZnzoLzXR8i3/LfWoEt4BfWtK xWrhDGgkcj2hTpJbu4big7V9Ml82EavNivY3VauVlXqKldC7BswjsbL/K9jbz0a3bOflB97vmhk hzDlEAlCQaNYVSbtY5MTRAkA8TrBhUnW98tEcvz0OoflgWJj4m6l+KCITBppKFyMzheebMbQaGQ phz0QqWKBORe7fWyG3GcTmqJdAUs8XBmG5zBgisfBQvtLl0aQCrSw1hM8dAqfTd6RuGXXLMXtVp EA56tpUARH/rVQb8rIVUiogem8c4NP4BnU62HMX/Jt2U2iv1MJH1QE38TXLTWlWobN7jO3jbEQ+ c3WCvhTc7GN4HoPQsk/Zb7J/eoLevIyK6qSibaAEwnehy8OaiMz3/e0Q2ZtRgGhdvVnW4heUbUn G48EdTgnws32otw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Introduce a new type in the media-fh.h header that represent a media device context. A media device context is allocated when the media device is open and released when the last reference to it is put. A new pair of media_device_ops is added to allow device drivers to allocate and release a media context. The media context groups together the media entity contexts that are associated with it to form an isolated execution context. Provide helpers in mc-device.c for drivers and for the v4l2-core to handle media device contexts and to bind/unbind entity contexts to it. Once an entity context has been bound to a media device context it is possible to retrieve it by using a pointer to the entity the device is represented by. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 168 ++++++++++++++++++++++++++++++++++ drivers/media/mc/mc-entity.c | 1 + include/media/media-device.h | 211 +++++++++++++++++++++++++++++++++++++++++++ include/media/media-fh.h | 5 + 4 files changed, 385 insertions(+) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index e0cad87087d3863bf14207049a54e5e4dea1cdd4..d8f12db933d22ae7466051698d853f4bdc599400 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -49,11 +51,31 @@ static int media_device_open(struct media_devnode *devnode, struct file *filp) { struct media_device *mdev = devnode->media_dev; struct media_device_fh *fh; + int ret; fh = kzalloc(sizeof(*fh), GFP_KERNEL); if (!fh) return -ENOMEM; + if (mdev->ops && mdev->ops->alloc_context) { + if (WARN_ON(!mdev->ops->destroy_context)) { + kfree(fh); + return -EINVAL; + } + + ret = mdev->ops->alloc_context(mdev, &fh->context); + if (ret) { + kfree(fh); + return ret; + } + + /* + * Make sure the driver implementing alloc_context has + * called media_device_init_context() + */ + WARN_ON(!fh->context->initialized); + } + filp->private_data = &fh->fh; spin_lock_irq(&mdev->fh_list_lock); @@ -73,6 +95,8 @@ static int media_device_close(struct file *filp) list_del(&fh->mdev_list); spin_unlock_irq(&mdev->fh_list_lock); + media_device_context_put(fh->context); + kfree(fh); return 0; @@ -860,6 +884,150 @@ void media_device_unregister(struct media_device *mdev) } EXPORT_SYMBOL_GPL(media_device_unregister); +/* ----------------------------------------------------------------------------- + * Context handling + */ + +static void media_device_release_context(struct kref *refcount) +{ + struct media_device_context *context = + container_of(refcount, struct media_device_context, refcount); + + /* + * All the associated entity contexts should have been released if we + * get here. + */ + WARN_ON(!list_empty(&context->contexts)); + + context->mdev->ops->destroy_context(context); +} + +struct media_device_context * +media_device_context_get(struct media_device_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get); + +void media_device_context_put(struct media_device_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_device_release_context); +} +EXPORT_SYMBOL_GPL(media_device_context_put); + +struct media_device_context *media_device_context_get_from_fd(unsigned int fd) +{ + struct media_device_context *ctx; + struct file *filp = fget(fd); + struct media_device_fh *fh; + + if (!filp) + return NULL; + + fh = media_device_fh(filp); + ctx = media_device_context_get(fh->context); + fput(filp); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get_from_fd); + +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx) +{ + ctx->mdev = mdev; + INIT_LIST_HEAD(&ctx->contexts); + mutex_init(&ctx->lock); + kref_init(&ctx->refcount); + + ctx->initialized = true; + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_init_context); + +void media_device_cleanup_context(struct media_device_context *ctx) +{ + mutex_destroy(&ctx->lock); + list_del_init(&ctx->contexts); +} +EXPORT_SYMBOL_GPL(media_device_cleanup_context); + +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context) +{ + struct media_entity_context *entry; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + + /* Make sure the entity has not been bound already. */ + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry == context) + return -EINVAL; + } + + list_add_tail(&context->list, &mdev_context->contexts); + context->mdev_context = media_device_context_get(mdev_context); + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_bind_context); + +int media_device_unbind_context(struct media_entity_context *context) +{ + struct media_device_context *mdev_context = context->mdev_context; + struct media_entity_context *entry; + struct media_entity_context *tmp; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + list_for_each_entry_safe(entry, tmp, &mdev_context->contexts, list) { + if (entry != context) + continue; + + list_del(&entry->list); + media_device_context_put(mdev_context); + entry->mdev_context = NULL; + + return 0; + } + + WARN(true, "Media entity context is not bound to any media context\n"); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(media_device_unbind_context); + +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity) +{ + struct media_entity_context *entry; + + guard(mutex)(&mdev_context->lock); + + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry->entity == entity) + return media_entity_context_get(entry); + } + + return ERR_PTR(-EINVAL); +} +EXPORT_SYMBOL(media_device_get_entity_context); + #if IS_ENABLED(CONFIG_PCI) void media_device_pci_init(struct media_device *mdev, struct pci_dev *pci_dev, diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index b4a9f0a0aa7353d7a3333f20903980956b3df4a7..7bc276c725f974539ea06e3882d004b81be1de68 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1717,5 +1717,6 @@ EXPORT_SYMBOL_GPL(media_entity_init_context); void media_entity_cleanup_context(struct media_entity_context *ctx) { + media_device_unbind_context(ctx); } EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-device.h b/include/media/media-device.h index 2fc750efef7c43814f019f12078e9c96c1bd6bf9..2ea8fce9ba75700286961f1622584372a954cb8a 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -18,10 +18,73 @@ #include #include +#include struct ida; struct media_device; +/** + * struct media_device_context - Media device context + * @mdev: The media device this context is associated with + * @refcount: The kref reference counter + * @lock: Protects the entities contexts list + * @contexts: List of entity contexts associated with this media device context + * @initialized: Flag set to true by media_device_init_context() + * + * A media device context is created every time the media device gets opened by + * userspace. It is then uniquely identified for applications by the numerical + * file descriptor returned by a successful call to open() and is associated + * with an instance of :c:type:`media_device_fh`. + * + * Media device contexts are ref-counted and thus freed once the last reference + * to them is released. + * + * A media device context groups together the media entity contexts registered + * on a video device or v4l2 subdevice that has been associated with a media + * device context. The association between a media entity context and media + * device context is called 'bounding', and the result of bounding is to create + * an 'execution context' independent from other execution contexts. + * + * An entity context is bound to a media device context by a call to the + * VIDIOC_BIND_CONTEXT ioctl on video devices and by a call to + * VIDIOC_SUBDEV_BIND_CONTEXT on subdevices by userspace. The bounding operation + * groups together entity contexts to the same media device context. As video + * devices and v4l2 subdevices devnodes can be opened multiple times, each file + * descriptor resulting from a successful open() call can be bound to a + * different media device context. + * + * Creating execution contexts by bounding video entity contexts to a media + * device context allows userspace to effectively multiplex the usage of a + * media graph and of the device nodes that are part of it. + * + * In order to create an execution context userspace should: + * + * 1) Open the media device to create a media device context identified by the + * file descriptor returned by a successful 'open()' call + * 2) Open the video device or v4l2 subdevice and bind the file descriptors to + * the media device context by calling the VIDIOC_BIND_CONTEXT and + * VIDIOC_SUBDEV_BIND_CONTEXT ioctls + * + * All devices bound to the same media device context are now part of the same + * execution context. From this point on all the operations performed on a file + * descriptor bound to a media device context are independent from operations + * performed on a file descriptor bound to a different execution context. + * + * Binding an entity context to a media device context increases the media + * device context reference count. This guarantees that references to media + * device context are valid as long as there are valid entity contexts that + * refers to it. Symmetrically, unbinding an entity context from a media + * device context decreases the media device context reference count. + */ +struct media_device_context { + struct media_device *mdev; + struct kref refcount; + /* Protects the 'contexts' list */ + struct mutex lock; + struct list_head contexts; + bool initialized; +}; + /** * struct media_entity_notify - Media Entity Notify * @@ -62,6 +125,13 @@ struct media_entity_notify { * request (and thus the buffer) must be available to the driver. * And once a buffer is queued, then the driver can complete * or delete objects from the request before req_queue exits. + * @alloc_context: Allocate a media device context. The operation allows drivers to + * allocate a driver-specific structure that embeds a + * media_device_context instance as first member where to store + * driver-specific information that are global to all device + * contexts part of media device context. Returns 0 on success a + * negative error code otherwise. + * @destroy_context: Release a media device context. */ struct media_device_ops { int (*link_notify)(struct media_link *link, u32 flags, @@ -70,6 +140,9 @@ struct media_device_ops { void (*req_free)(struct media_request *req); int (*req_validate)(struct media_request *req); void (*req_queue)(struct media_request *req); + int (*alloc_context)(struct media_device *mdev, + struct media_device_context **ctx); + void (*destroy_context)(struct media_device_context *ctx); }; /** @@ -298,6 +371,144 @@ int __must_check __media_device_register(struct media_device *mdev, */ void media_device_unregister(struct media_device *mdev); +/* ----------------------------------------------------------------------------- + * media device context handling + */ + +/** + * media_device_context_get - Increase the media device context reference count + * and return a reference to it + * @ctx: The media device context + */ +struct media_device_context * +media_device_context_get(struct media_device_context *ctx); + +/** + * media_device_context_put - Decrease the media device context reference count + * @ctx: The media device context + */ +void media_device_context_put(struct media_device_context *ctx); + +/** + * media_device_context_get_from_fd - Get the media device context associated with a + * numerical file descriptor + * + * @fd: the numerical file descriptor + * + * A media device context is created whenever the media device devnode is opened + * by userspace. It is then associated uniquely with a numerical file descriptor + * which is unique in the userspace process context. + * + * This function allows to retrieve the media device associated with such + * numerical file descriptor and increases the media device context reference + * count to guarantee the returned reference stays valid at least until the + * caller does not call media_device_context_put(). + * + * Caller of this function are required to put the returned media device context + * once they are done with it. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * which need to get the media device contexts associated to a numerical file + * descriptor. + */ +struct media_device_context *media_device_context_get_from_fd(unsigned int fd); + +/** + * media_device_init_context - Initialize the media device context + * + * @mdev: The media device this context belongs to + * @ctx: The media device context to initialize + * + * Initialize the fields of a media device context. Device drivers that support + * multi context operations shall call this function in their implementation of + * media_device_operations.alloc_context() + */ +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx); + +/** + * media_device_cleanup_context - Cleanup the media device context + * + * @ctx: The media device context to clean up + * + * Cleanup a media device context. Device drivers that support multi context + * operations shall call this function in their implementation of + * media_device_operations.destroy_context() before releasing the memory allocated + * by media_device_operations.alloc_context(). + */ +void media_device_cleanup_context(struct media_device_context *ctx); + +/** + * media_device_bind_context - Bind an entity context to a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @context: the entity context to bind + * + * This function creates a mapping entry in the media device context that + * associates an entity context to the media entity it belongs to and stores it + * in a linked list so that they can be retrieved later. + * + * Binding an entity context to a media device context increases the media + * device context refcount. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * that binds a newly created context to a media device context. + */ +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context); + +/** + * media_device_unbind_context - Unbind an entity context from a media device + * context + * + * @context: the entity context to unbind + * + * An entity context is unbound from a media device context when the file handle + * it is associated with gets closed. + * + * Unbinding an entity context from a media device context decreases the media + * device context refcount. + * + * Returns 0 if the context was bound to a media device context, -EINVAL + * otherwise. + */ +int media_device_unbind_context(struct media_entity_context *context); + +/** + * media_device_get_entity_context - Get the entity context associated with + * a media entity in a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @entity: pointer to struct &media_entity that the entity context is + * associated with + * + * An entity context is uniquely associated with a media device context after it + * has been bound to it by a call to the VIDIOC_BIND_CONTEXT ioctl. This helper + * function retrieves the entity context associated with a media device context + * for a specific entity that represents a video device or a v4l2 subdevice. + * + * The reference count of the returned entity context is increased to guarantee + * the returned reference stays valid until the caller does not call + * media_entity_context_put(). + * + * Drivers are not expected to call this function directly but should instead + * use the helpers provided by the video_device and v4l2_subdevice layers, + * video_device_context_get() and v4l2_subdev_get_context() respectively. + * Drivers are always required to decrease the returned context reference count + * by calling video_device_context_put() and v4l2_subdev_put_context(). + * + * If no entity context has been associated with the media device context + * provided as first argument an error pointer is returned. Drivers are + * required to always check the value returned by this function. + */ +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity); + +/*------------------------------------------------------------------------------ + * Media entity handling + */ + /** * media_device_register_entity() - registers a media entity inside a * previously registered media device. diff --git a/include/media/media-fh.h b/include/media/media-fh.h index 6f00744b81d6000a4b0c503fe6968dd7adcbb1c3..48ec266416dd288a008bc5f93db5eb7ec6b8859c 100644 --- a/include/media/media-fh.h +++ b/include/media/media-fh.h @@ -13,15 +13,20 @@ #include +struct media_device_context; + /** * struct media_device_fh - File handle specific information on MC * * @fh: The media device file handle * @mdev_list: This file handle in media device's list of file handles + * @context: The media device context associated with the file handle */ struct media_device_fh { struct media_devnode_fh fh; struct list_head mdev_list; + + struct media_device_context *context; }; static inline struct media_device_fh *media_device_fh(struct file *filp) From patchwork Thu Jul 17 10:45:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116023 Received: from am.mirrors.kernel.org ([147.75.80.249]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM8i-0005CW-26 for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:47:03 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 4038C1AA0571 for ; Thu, 17 Jul 2025 10:47:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B40FA29C355; Thu, 17 Jul 2025 10:45:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="o1Vqy7nD" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EA4D29B8D2; Thu, 17 Jul 2025 10:45:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749153; cv=none; b=iQsTLKXxqKUwN5eWnCrPHMm3uoZTyPQbzYQ2+YgjJeU9B2baJ2QmMs6pORNyKeryD0/uhie4XeCzrm4qqGhAeWyRgO1cGwEGJJU7yoBIPeMWclUkvKyQ0KuhW/Eh1TwOnR0fuTbiY2MYCYhlEXBJR0GGrmTKbUft/hvSf8BRoG0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749153; c=relaxed/simple; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GTYb3ycZycbz8Is9gkT9OAHq1elOm1aV4iu/i8xtM8c6za4NKMYPoT3dx0lq4tggyVDsskkyR4uvjIhAf6GMytthfp68yXF/eg2AA8yj3En3SqHr/TcF1LazS8d3SpQ+G2j0vlAgmJbxAPE3JPVzN8ce1oG4H11IlDCsUxcgLSU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=o1Vqy7nD; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5BAA0245A; Thu, 17 Jul 2025 12:45:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749113; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=o1Vqy7nDEMLAs6ynqCHBKfp8Lu3lydwYUnx+TxmuEXulbM7Wtq7l4bBRIFG0lPx0I Yzi+TEksMelV29wZBq20oxVVvtxKmpTn2ND8lkvqSmFisNiif44qKquhORBwQMmrOc /8Xtw0SyRGpT6CdMWxbXcRLJOWA8xuvLRgHTYClk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:31 +0200 Subject: [PATCH 05/26] media: v4l2-dev: Introduce video device context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-5-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9358; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=DnyByKVCK9e8LIByMHqQ/KRM1C+sAIkgyijuQbTd5Yw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRPmZPNqWcp2o41L8WvXnyZ+RaMzh/4RTuIA hHJ3eTkoZeJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PARFD/9C6s2MBoRPt50ZCEAgm3WoSWGngkhpdNfCDtaSjwcLSwLyXhEWFxYP/SRbdyog8fGM18A xJf0HWYsnviL5uWoG3FiVmp1yp+/wjnGzfeh1dxv+lj58E46W/vKU05SzIWYr16eBxDcN4D++Yy PP8nvUnkV5X+Ir/9JuzEHI7BabCWyklKBC+Es1TcspDpcrBdjOL38br4RRFufVLOlKo+68K3i4C FlctA3L4/4Dew+xsrd8BWkJVUFnbdLL6ytiYzY78xnomvV9ZNJH/W+a2pleczxMlSK8iKam5/pk X4ZDrvSGpIo+blfJJxySFWNMhPVwCbONnCi3nOsYk8vqUZURivGiq5AQKUZgmcK/KviZalhxhSH EsyxEX0f00Jl+pLZeQQiSCnbvH/3uOQhIfLd7zfizJJKe2R9c6P8halPUYZX7qf6DXj8pL4nxsn ot1B11NPYyJCA6T29AGr3d9o1Jplosbvbt/9XS1lLUmh9K6nR7LEyJJYicZolZxiw5nfYLV94Jo F/iD6HZ+eo3rxvj3xUw8U5MBzCf3BEOpyJh0MXL24zZB0BtAD9ngAnzNCOCxthSI3P9sNJdS1IE 8vbn5M1ZGiCsFh9cn7lgaNfjQ2P8ENAw3/UmR21WKganRrAG4rKmIAUQmB66fCPInCnvrRFgoQx uZVjNy8R5ECH7yw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce a new type in video-device that represents a video device context. It extends 'struct media_entity_context' to include video-device specific data and configuration. The new type is intended to be extended by drivers that can store driver-specific information in their video device derived types. The next patch will introduce the VIDIOC_BIND_CONTEXT ioctl that allows to create a video device context and uniquely associate it with a media context. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 43 +++++++++++++ include/media/v4l2-dev.h | 126 +++++++++++++++++++++++++++++++++++++ include/media/v4l2-fh.h | 3 + 3 files changed, 172 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index c369235113d98ae26c30a1aa386e7d60d541a66e..c83c37843c9e7beb899a4b2bd176273c3dec381b 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1200,6 +1200,49 @@ struct media_pipeline *video_device_pipeline(struct video_device *vdev) } EXPORT_SYMBOL_GPL(video_device_pipeline); +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev) +{ + struct media_entity *entity = &vdev->entity; + struct media_entity_context *ctx = + media_device_get_entity_context(mdev_context, entity); + + if (!ctx) + return NULL; + + return container_of(ctx, struct video_device_context, base); +} +EXPORT_SYMBOL_GPL(video_device_context_get); + +void video_device_context_put(struct video_device_context *ctx) +{ + if (!ctx) + return; + + media_entity_context_put(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_context_put); + +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx) +{ + media_entity_init_context(&vdev->entity, &ctx->base); + + ctx->vdev = vdev; + mutex_init(&ctx->queue_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(video_device_init_context); + +void video_device_cleanup_context(struct video_device_context *ctx) +{ + mutex_destroy(&ctx->queue_lock); + media_entity_cleanup_context(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_cleanup_context); + #endif /* CONFIG_MEDIA_CONTROLLER */ /* diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index a69801274800f4f9add723b51fe0a31331e88f97..9276e095fb17414a9eb7845db0aa81572c42ca45 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -18,6 +18,7 @@ #include #include +#include #define VIDEO_MAJOR 81 @@ -660,6 +661,131 @@ __must_check int video_device_pipeline_alloc_start(struct video_device *vdev); */ struct media_pipeline *video_device_pipeline(struct video_device *vdev); +/** + * struct video_device_context - The video device context + * @base: The media entity context base class member + * @vdev: The video device this context belongs to + * @queue_lock: Protects the vb2 queue + * @queue: The vb2 queue + * + * This structure represents an isolated execution context of a video device. + * This type 'derives' the base 'struct media_entity_context' type which + * implements refcounting on our behalf and allows instances of this type to be + * linked in the media_device_context contexts list. + * + * By storing the data and the configuration of a video device in a per-file + * handle context, userspace is allowed to multiplex the usage of a single video + * device devnode by opening it multiple times and by associating it with a + * media device context. This operation is called 'bounding' and is performed + * using the VIDIOC_BIND_CONTEXT ioctl. + * + * A video device context is created and stored in the v4l2-fh file handle + * associated with an open file descriptor when a video device is 'bound' to a + * media device context. The 'bounding' operation realizes a permanent + * association valid until the video device context is released. + * + * A video device can be bound to the same media device context once only. + * Trying to bind the same video device to the same media device context a + * second time, without releasing the already established context by closing the + * bound file descriptor first, will result in an error. + * + * To create a video device context userspace shall use the VIDIOC_BIND_CONTEXT + * ioctl that creates the video device context and uniquely associates it with a + * media device file descriptor. + * + * Once a video device file descriptor has been bound to a media device context, + * all the operations performed on the video device file descriptor will be + * directed on the just created video device context. This means, in example, + * that the video device format and the buffer queue are isolated from the ones + * associated with a different file descriptor obtained by opening again the + * same video device devnode but bound to a different media device context. + * + * Drivers that implement multiplexing support have to provide a valid + * implementation of the context-related operations in the + * media entity operations. + * + * Drivers are allowed to sub-class the video_device_context structure by + * defining a driver-specific type which embeds a struct video_device_context + * instance as first member, and allocate the driver-specific structure size in + * their implementation of the `alloc_context` operation. + * + * Video device contexts are ref-counted by embedding an instance of 'struct + * media_entity_context' and are freed once all the references to it are + * released. + * + * A video device context ref-count is increased when: + * - The context is created by bounding a video device to a media device context + * - The media pipeline starts streaming + * A video device context ref-count is decreased when: + * - The associated file handle is closed + * - The media pipeline stops streaming + * + * The ref-count is increased by a call to video_device_context_get() and is + * reponsibility of the caller to decrease the reference count with a call to + * video_device_context_put(). + */ +struct video_device_context { + struct media_entity_context base; + + struct video_device *vdev; + /* Protects the vb2 queue. */ + struct mutex queue_lock; + struct vb2_queue queue; +}; + +/** + * video_device_context_get - Helper to get a video device context from a + * media device context + * + * @mdev_context: The media device context + * @vdev: The video device the context refers to + * + * Helper function that wraps media_device_get_entity_context() and returns + * the video device context associated with a video device in a media device + * context. + * + * The reference count of the returned video device context is increased. + * Callers of this function are required to decrease the reference count of + * the context reference with a call to video_device_context_put(). + */ +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev); + +/** + * video_device_context_put - Helper to decrease a video context reference + * count + * + * @ctx: The video context to release + */ +void video_device_context_put(struct video_device_context *ctx); + +/** + * video_device_init_context - Initialize the video device context + * + * @vdev: The video device this context belongs to + * @ctx: The context to initialize + * + * Initialize the video device context. The intended callers of this function + * are driver-specific implementations of the media_entity_ops.alloc_context() + * function that allocates their driver specific types that derive from + * struct video_device_context. + */ +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx); + +/** + * video_device_cleanup_context - Cleanup the video device context + * + * @ctx: The context to cleanup. + * + * Cleanup the video device context. The intended callers of this function are + * driver specific implementation of the media_entity_ops.destroy_context() + * function before releasing the memory previously allocated by + * media_entity_ops.alloc_context(). + */ +void video_device_cleanup_context(struct video_device_context *ctx); + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h index b5b3e00c8e6a0b082d9cd8a0c972a5094adcb6f2..a8de8613a026589ede94dc3e70c8a49ae08f4582 100644 --- a/include/media/v4l2-fh.h +++ b/include/media/v4l2-fh.h @@ -20,6 +20,7 @@ struct video_device; struct v4l2_ctrl_handler; +struct video_device_context; /** * struct v4l2_fh - Describes a V4L2 file handler @@ -38,6 +39,7 @@ struct v4l2_ctrl_handler; * @sequence: event sequence number * * @m2m_ctx: pointer to &struct v4l2_m2m_ctx + * @context: The video device context */ struct v4l2_fh { struct list_head list; @@ -54,6 +56,7 @@ struct v4l2_fh { u32 sequence; struct v4l2_m2m_ctx *m2m_ctx; + struct video_device_context *context; }; /** From patchwork Thu Jul 17 10:45:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116029 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM99-0005Dd-2H for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:47:28 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 0EF593B1B13 for ; Thu, 17 Jul 2025 10:46:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 15ACE29E0EA; Thu, 17 Jul 2025 10:45:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SOiHuxq8" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 198FD29DB7F; Thu, 17 Jul 2025 10:45:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749156; cv=none; b=GkHdO99/oRJE+S23W1cDbm1dUFOeUHEEYXkODmZHDsCG53GQaB+Wz9y9Nup+c9kFwwcUQqLkD/MAwm4tw7hqon9qhvTphFOpZFhbBe9I2QCjYFGKRvBsvhuw6sLhjtSwXQa4u8yRjl8CC+aQmm94TgV6JrFmYINFuoSoVXwIxt4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749156; c=relaxed/simple; bh=8R73HPaz5pGFoxscWZIYsH9YWq0Ywb87x5bejY/EFeM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QHyp/xz8M4R+lxygFeyM8kBI0919U3RgeYmDxW+zj3FOEbpiaF1TztxkN0PiLfCITEjufZDXOo8pjQUUE1X+iAYzpty9vbAEWpkGRSYvzuVewEbs4uSGlj2BTImw7LkxI5VbtQI2d1BlkpBHvYYFKCiNczuUMFc2lV9nzDYwIrI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=SOiHuxq8; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C93F3198D; Thu, 17 Jul 2025 12:45:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749115; bh=8R73HPaz5pGFoxscWZIYsH9YWq0Ywb87x5bejY/EFeM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SOiHuxq8bdNt7SUOg01x5EwIA3zPLzESlFTgWYrVGjEHsxAno2JVpff/lYDXyYxvn WCggyMJgoWJlyzBLV1qU7pt3DxqyFpOP9ZJoF8o9RwzG+mcQIuu9WkLjtfNN2wZeLC oIBVD8zOmfZjdx9WqMGJG5648ji+ca0iXpNxNG5k= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:32 +0200 Subject: [PATCH 06/26] media: v4l2-ioctl: Introduce VIDIOC_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-6-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8477; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=8R73HPaz5pGFoxscWZIYsH9YWq0Ywb87x5bejY/EFeM=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRPuxlgrQbLEhBMUC26EnJg5sMetl7FSM+mi 8Q1kKPMoYqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PNIjD/0fEqn9SDhgeerqLoTAs2HwXZnGao1NsXkGYyUmaguuhhDcaMETK1sZeSWBDURKkRGij0X WXrWQpEbohjIBjGAObAiOae4/zPpcHEvZCq1P3knfpKdDleAyOMCRkpXgkN9Zbrm/22PMa+zqv8 jj+g+jdoqDnyOt2f7WqVMiDzlJRTq6KucXNeio3Rwk9QPA+Perj1a5U6niBrZqUPe0Y0SEFzRty zxrmQQCaVJXcRy9pMIyqofajGgIrMrXDhrRmGn5nMc9QAu4+oTQwX6zJxgAxlekt08KD9L6Ioqc 5HkpMu/QtEF1RkZmLbvsV7E7M5qHxGeDd6et4W4wW35Ciw1LNjDWsD/xMHmZ9Kf+thPEZzdIGVV j2S3fIJunNWw9nYffnWXm9Q3Su+++kv19zSWanLxPw0zHG7d/xvYSX2BhSbAwQzD/LJnNptXcny Y4rs5Ls77K3DuVoybPh9FYdak+5m7uU6AjrBPnnH3ugsztxXkRRXwaG73oK3RjY3PHiuJO28L0e ouc9VzJspHZ32CXbyPDzGHYMQvDdmlZCaO8/QS9q+L/cvqu6a9PrJCvRfiFPF6+gB+DkCSkHvc9 6tTshdadZ/WX6c9BF0XLpWfj2SGOV+vLD1AlAjTcbwd8CZ8mpTRzifPz4nJv+KFt9KV0QKN+KPh mZKXBqwSo2SPv2A== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce a new ioctl in V4L2 to allocate a video device context and associate it with a media device context. The ioctl is valid only if support for MEDIA_CONTROLLER is compiled in as it calls into the entity ops to let driver allocate a new context and binds the newly created context with the media context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated video context is then stored in the v4l2-fh that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 10 ++++++ drivers/media/v4l2-core/v4l2-fh.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 64 ++++++++++++++++++++++++++++++++++++ include/media/v4l2-ioctl.h | 7 ++++ include/uapi/linux/videodev2.h | 11 +++++++ 5 files changed, 93 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index c83c37843c9e7beb899a4b2bd176273c3dec381b..bc6502b4ce21cc0ad53136e1637d1c926e31dd89 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -606,6 +606,10 @@ static void determine_valid_ioctls(struct video_device *vdev) __set_bit(_IOC_NR(VIDIOC_ENUM_FREQ_BANDS), valid_ioctls); if (is_vid) { +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif + /* video specific ioctls */ if ((is_rx && (ops->vidioc_enum_fmt_vid_cap || ops->vidioc_enum_fmt_vid_overlay)) || @@ -661,12 +665,18 @@ static void determine_valid_ioctls(struct video_device *vdev) SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_cap); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } else if (is_meta && is_tx) { /* metadata output specific ioctls */ SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_out); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } if (is_vbi) { /* vbi specific ioctls */ diff --git a/drivers/media/v4l2-core/v4l2-fh.c b/drivers/media/v4l2-core/v4l2-fh.c index 90eec79ee995a2d214590beeacc91b9f8f33236d..f7af444d2344541ccae1eae230b39d4cbc47f6bd 100644 --- a/drivers/media/v4l2-core/v4l2-fh.c +++ b/drivers/media/v4l2-core/v4l2-fh.c @@ -93,6 +93,7 @@ int v4l2_fh_release(struct file *filp) struct v4l2_fh *fh = filp->private_data; if (fh) { + video_device_context_put(fh->context); v4l2_fh_del(fh); v4l2_fh_exit(fh); kfree(fh); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 46da373066f4ec786b87ef18b8372abee621332f..bade64cc62b66dd6237ccd5338aa6dd8ab00ef8c 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -9,6 +9,7 @@ */ #include +#include #include #include #include @@ -350,6 +351,13 @@ static void v4l_print_format(const void *arg, bool write_only) } } +static void v4l_print_context(const void *arg, bool write_only) +{ + const struct v4l2_context *c = arg; + + pr_cont("context=%u\n", c->context_fd); +} + static void v4l_print_framebuffer(const void *arg, bool write_only) { const struct v4l2_framebuffer *p = arg; @@ -2151,6 +2159,61 @@ static int v4l_overlay(const struct v4l2_ioctl_ops *ops, return ops->vidioc_overlay(file, fh, *(unsigned int *)arg); } +static int v4l_bind_context(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vdev = video_devdata(file); + struct media_device_context *mdev_context; + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vdev->flags) ? fh : NULL; + struct v4l2_context *c = arg; + int ret; + + /* + * TODO: do not __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls) + * if V4L2_FL_USES_V4L2_FH isn't set or the driver does not implement + * alloc_context and destroy_context. + */ + if (!vfh) + return -ENOTTY; + + if (!vdev->entity.ops || !vdev->entity.ops->alloc_context || + !vdev->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context = media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + /* Let the driver allocate the per-file handle context. */ + ret = vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vfh->context); + if (ret) + goto err_put_mdev_context; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret = media_device_bind_context(mdev_context, + (struct media_entity_context *) + vfh->context); + if (ret) + goto err_put_context; + + media_device_context_put(mdev_context); + + return 0; + +err_put_context: + video_device_context_put(vfh->context); +err_put_mdev_context: + media_device_context_put(mdev_context); + + return ret; +} + static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -2998,6 +3061,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), IOCTL_INFO(VIDIOC_REMOVE_BUFS, v4l_remove_bufs, v4l_print_remove_buffers, INFO_FL_PRIO | INFO_FL_QUEUE | INFO_FL_CLEAR(v4l2_remove_buffers, type)), + IOCTL_INFO(VIDIOC_BIND_CONTEXT, v4l_bind_context, v4l_print_context, 0), }; #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index 82695c3a300a73219f262fb556ed61a8f09d273e..6d9edfd9ca912972ad15acdc07014dee1ed36ab6 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -18,6 +18,7 @@ #include struct v4l2_fh; +struct video_device_context; /** * struct v4l2_ioctl_ops - describe operations for each V4L2 ioctl @@ -149,6 +150,8 @@ struct v4l2_fh; * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata capture * @vidioc_try_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata output + * @vidioc_bind_context: pointer to the function that implements + * :ref:`VIDIOC_BIND_CONTEXT ` ioctl * @vidioc_reqbufs: pointer to the function that implements * :ref:`VIDIOC_REQBUFS ` ioctl * @vidioc_querybuf: pointer to the function that implements @@ -402,6 +405,10 @@ struct v4l2_ioctl_ops { int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + /* Context handlers */ + int (*vidioc_bind_context)(struct file *file, void *fh, + struct video_device_context *c); + /* Buffer handlers */ int (*vidioc_reqbufs)(struct file *file, void *fh, struct v4l2_requestbuffers *b); diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..0b9aa89e2479620dbbaa54f1aadff7aaa7a3d0f7 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1057,6 +1057,14 @@ struct v4l2_jpegcompression { * always use APP0 */ }; +/* + * V I D E O D E V I C E C O N T E X T + */ + +struct v4l2_context { + __u32 context_fd; +}; + /* * M E M O R Y - M A P P I N G B U F F E R S */ @@ -2818,6 +2826,9 @@ struct v4l2_remove_buffers { #define VIDIOC_REMOVE_BUFS _IOWR('V', 104, struct v4l2_remove_buffers) +/* Context handling */ +#define VIDIOC_BIND_CONTEXT _IOW('V', 105, struct v4l2_context) + /* Reminder: when adding new ioctls please add support for them to drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ From patchwork Thu Jul 17 10:45:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116032 Received: from am.mirrors.kernel.org ([147.75.80.249]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM9D-0005Dt-2v for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:47:33 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C19951AA2F3D for ; Thu, 17 Jul 2025 10:47:47 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3995129E0F5; Thu, 17 Jul 2025 10:45:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="daPpCCeU" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65AA229DB8F; Thu, 17 Jul 2025 10:45:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749157; cv=none; b=A41eSsm76l5uc9eqylo5zcpf2C6x4KlvK3ocy+jkmdA615Vl0bnIOkbONuJ6BNnI7+yzYJVu3CRRgY6nDVAgyHmgE5iSmVXnjxynW8Pgoy0dabcXX5PptxSM7fUa3Z2mg76rToi1wzAskYstrb+vZCUuu5Ore0f6Sr8oJEO7khk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749157; c=relaxed/simple; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LBRQdwRm1SBABd2eMHgrStzUmKNgc+Jgz+4v2Hs35Xnmqj9Jd8KkSaGgP08QUE7W4nYhcUTbH4g4muijGaZwMwBjYtltPLkP3ouLxxzfiw8rSxeyV/n8gKlVLaCNSLL+F09Ke3HtOEcp791hc5poNqO4xnUfIzOvuTQOmJGJPlg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=daPpCCeU; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4CBDA1AE2; Thu, 17 Jul 2025 12:45:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749116; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=daPpCCeUW43d24KCryQ2pdLtbpWDh0u0qHqEytOQnQlF0qWHGt/4IkmWWRVRszmH6 27EHu2j1Ny/2xPvIBpyQkcxN/9fmBz+l6YXd9Iu0kM/MJ7ZScL8K5WPl5xOWZ7ievn kRdtsuh1udrJAOqP16K+GfFV/75LDAUGhXuUXWrc= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:33 +0200 Subject: [PATCH 07/26] media: Documentation: Add VIDIOC_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-7-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4469; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V29a6qX8KA0ELmVzRolMfVBPPzH85sU33AYkFGGSdRI=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRP4kD2I23sWOplrDqy5y3c+7A+RTx3u8rN9 jYoHzA0q+6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PHo6EACcywtLQFchr4d0k6G4vWMujDGLd+QdUrBmIoK1W9TQKB97vJKK0bBo2m4z0VdVzN7/iQf xBnqB9Kffzup/5jvPbijDrdud6CsXCU9DO3JXpdA04Me2ZDl7wa+cGUtQ8EcF8+/SMGaPvo4q7t SllO+1g5Y2Vu6RCg1AqQR6LyxrZuxp3+kq03dlLPX1RLy8F9yAMGhwKcYLt3XU+/wlttt14ULM+ IAsMavvf2Rk7FZHi9RzCJHBeFDXWBroLPpgpBn+qf8oTNHk5FKvpCwfn0ukYV1TbxuOrluNnhKO WEZnRODoJLK9EO4bCEeRh3RL/ZCILC6ynpeLyV95HzQwdKRDPb0DobzqPIk31qsk9tz0XuiZ1rl g+Qx7qwGwqvyYx5tgheiiZDJ5f6hBraeQDEPfANHJxYuSQ1jA7cziTS4SJa6mEKJxicFdXPUpcr APzKQvJSgCob8Ce2RcJpvTBlRcJzPsfuNUYqKBp9jXV/obI2PeQmO2AaCHZhgUMuL+BZ0oMD9Yq 0HygKlidcXBm0z1FAlOgduNnpAvANlRN5Nw6716e/p8aOYAlha6FLv2vna3AXlBFJ7x+lO/YFHI 5U1zMW0aSFbalRzvQY9eL8PYty8lx9rvd/QnVNPqQmfkbttddmIqPKQ4flmBJJaYB8yjBr8ZoTv g/y4JE7cV972vnA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Document the newly introduced VIDIOC_BIND_CONTEXT ioctl. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-bind-context.rst | 80 ++++++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst index 6f661138801cde2810997740ee8305085fe73a43..0d9aff56ab653b2a4f6afe4828f88bc5637addf1 100644 --- a/Documentation/userspace-api/media/v4l/user-func.rst +++ b/Documentation/userspace-api/media/v4l/user-func.rst @@ -12,6 +12,7 @@ Function Reference func-close func-ioctl + vidioc-bind-context vidioc-create-bufs vidioc-cropcap vidioc-dbg-g-chip-info diff --git a/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst b/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst new file mode 100644 index 0000000000000000000000000000000000000000..74c885169e22c495056a9364e722e2140d8f21c5 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/vidioc-bind-context.rst @@ -0,0 +1,80 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: V4L + +.. _vidioc_bind_context: + +************************* +ioctl VIDIOC_BIND_CONTEXT +************************* + +Name +==== + +VIDIOC_BIND_CONTEXT - Bind a video device file handle to a media device context + +Synopsis +======== + +.. c:macro:: VIDIOC_BIND_CONTEXT + +``int ioctl(int fd, VIDIOC_BIND_CONTEXT, struct v4l2_context *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`v4l2_context`. + +Description +=========== + +Applications call the ``VIDIOC_BIND_CONTEXT`` ioctl to bind a video device file +handle to a media device context. Binding a video device file handle to a media +device context creates an isolated execution context which allows to multiplex +the usage of a video device. This means, in practice, that the video device +configuration (format, sizes etc) applied on a file handle bound to a media +device context won't be visible on file handles bound to a different media +device context (or not bound at all). + +By opening a media device applications create a media device context to which +video devices and subdevices file handles can be bound to. The file descriptor +returned by a call to :c:func:`open()` on the media device identifies uniquely +the media device context. Application populates the ``context_fd`` field of +:c:type:`v4l2_context` with the file descriptor of an open media device to +identify the media context to which they want to bind a video device to. + +Applications can open a video device node multiple times, and call +``VIDIOC_BIND_CONTEXT`` on each file handle returned by a successful call to +:c:func:`open()` to isolate the operations performed on that file handle from +any operation performed on other file handles bound to different contexts. This +means, in example, that the video device format, sizes and the buffers are +isolated from the ones associated with a file descriptor, obtained by opening +the same video device but bound to a different media device context (or not +bound at all). + +The bounding operation realizes a permanent association valid until the video +device context is released by closing the file handle. + +A video device file handle can be bound to the same media device context once +only. Trying to bind the same file handle to the same media device context a +second time, without releasing the already established context by closing the +bound file descriptor first, will result in an error. + +Bounding is an opt-in feature that applications are free to ignore. Any +operation directed to a non bound file handle will continue to work as it used +to, and the video device configuration (formats, sizes etc) will be visible +across all the other non-bound file handles. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes ` chapter. + +EINVAL + The media device context file handle ``context_fd`` is not valid or the + video device file handle is already bound to a context. From patchwork Thu Jul 17 10:45:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116038 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM9s-0005Es-2q for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:48:15 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 392A23BD3EC for ; Thu, 17 Jul 2025 10:47:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7C6282BD032; Thu, 17 Jul 2025 10:46:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qhwKVawd" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 827F92BCF51; Thu, 17 Jul 2025 10:45:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749160; cv=none; b=roWeFLzOVVWsd/bFFzQwIn8ftOr71JHiFlRjqBbx94M3g8aMvINhVXjo/2pB4oo+WT/2Ga5r36XlWOC3/JMSL2UjxHUAyV/mBTgkUWDRdUSQwoVAkF+mCM2p0kAMCOxQ9nqpTje2oNyQuRRNtj0iDnLv60yS1CXeCSJG/lQ33LY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749160; c=relaxed/simple; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dRUVfcjs4QbyHsM7SlCCzvvW6qgwn8fN8T/ShyXou6lOf0f5CkcvfV2AVp4NsxN0vDyvQjQ0vt9TFzA6rg1oIg7KAE4JKIcpG/lOQq3bBTA8ryRQqmCZrNeOJPuE9m5+GiJl/UKTS+HcNTLjLr5bHKk5PLLizDQhQ4oNLMiEx1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=qhwKVawd; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CBBE61FA4; Thu, 17 Jul 2025 12:45:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749118; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qhwKVawdjRs89dZMeIMZ9874dTZKV8VYv2Qpkd4StVdxc5TCzqnfEXETURqSSTk78 K3pyKkexxunKZAvdRvgGuT2S2M4giLXyotrm+IuAAO2wMv9lcEKw7cm6inc2Te2a5X aCFSo42/fcppnGDoFM7ip9skMFZnzbRYEZEuBCVk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:34 +0200 Subject: [PATCH 08/26] media: v4l2-dev: Introduce default contexts Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-8-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6550; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=EonmGcn1CUXqn+CIokljWT6wYvOQxvsBhcP+HctWnYE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRPhWQgMR/YWcKAffqyCaN5tFPhmYw9o3CJD wRQLkVuNd2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PEycD/47kDMLqGLWmoTXR4z/+6Xe7Kf4IxN8CNBGoao1QvXSk4UlQ0ZXq9RKqX/WT7l9jecuICL 0Y/IEf8y0ekf7HMdgs4ewv1znQ/6xFMWoA4oyqF80bZ9dNtO9k0p5JqMLcyRAJeHkXWogWZD6lD pbHbbtRLWpZX+V8Ku7jdp3ByLSl2syCmRC9j0r0dAFo0As59ZLQwER8fpwBH3qk2NWwPR/E8x/j JlDD4VI6S9UtZSuIq0qfV2yhtI8TQ+Dz8LL81Kq5EYPxprzI6IkvajE4qBrdSDg3Qr6AAydfVJi jgKFK/sK+HJJBC8MmBFiX5STEdxEu0McOkru3CQSUasKYWmNgbsL5kdgOLMtH0Ldcfbi1XiQPYS UR72PL19sZEVQ+ni6EDcRHqOXfpwAXjQQoGZoA31u1SxXRjsZQCnFQD1GPCPKB6n5yPvjoUlENW 1xQjVeS8pDjqn3oBf55pQs1frrF9+6/tVKrPgeLF7tw2brib/+iZz1f4COgxiogRwb+JwDZXsWo AvdaO7xCrQL1JVCx/Q6WOhS4cjpa3/wF3w+JoCmBTWOgOwW1KFOyOTjNXm+k7quV5ECH/VUv4bb Z2hXNXqXn9ZnA+CC7I0ehsIGQTzNL62YmVRbODzCFACKZVmDYZosUugxzKYbCeRM9hS2MjFdkFw ue4Nr8q5pDFRj8w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce the media device and video device default contexts. Drivers ported to use multi-context support that used to work with a non-context aware userspace (which doesn't call VIDIOC_BIND_CONTEXT) shall continue to work even if they are context aware. Provide a default context in the media device and in the video device structures and let drivers allocate and release them with the newly introduced operations. Bind the video device default context to the default context of the media device associated with the v4l2_dev. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 11 +++++++++ drivers/media/v4l2-core/v4l2-dev.c | 46 +++++++++++++++++++++++++++++++++++--- include/media/media-device.h | 5 +++++ include/media/v4l2-dev.h | 3 +++ 4 files changed, 62 insertions(+), 3 deletions(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index d8f12db933d22ae7466051698d853f4bdc599400..e1f34f884fee2e8c3750f9c1e85142ff2f6b7bf0 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -746,12 +746,23 @@ void media_device_init(struct media_device *mdev) media_set_bus_info(mdev->bus_info, sizeof(mdev->bus_info), mdev->dev); + mdev->default_context = NULL; + if (mdev->ops && + mdev->ops->alloc_context && mdev->ops->destroy_context) { + if (mdev->ops->alloc_context(mdev, &mdev->default_context)) { + dev_err(mdev->dev, + "Failed to initialize media device default context\n"); + return; + } + } + dev_dbg(mdev->dev, "Media device initialized\n"); } EXPORT_SYMBOL_GPL(media_device_init); void media_device_cleanup(struct media_device *mdev) { + media_device_context_put(mdev->default_context); ida_destroy(&mdev->entity_internal_idx); mdev->entity_internal_idx_max = 0; media_graph_walk_cleanup(&mdev->pm_count_walk); diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index bc6502b4ce21cc0ad53136e1637d1c926e31dd89..2d78096931efd88215bc847b105e198a54035546 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -31,6 +31,8 @@ #include #include +#include + #define VIDEO_NUM_DEVICES 256 #define VIDEO_NAME "video4linux" @@ -220,8 +222,13 @@ static void v4l2_device_release(struct device *cd) if (v4l2_dev->release == NULL) v4l2_dev = NULL; - /* Release video_device and perform other - cleanups as needed. */ + /* Release default context. */ +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context = NULL; + + /* Release video_device and perform other cleanups as needed. */ vdev->release(vdev); /* Decrease v4l2_device refcount */ @@ -1086,7 +1093,36 @@ int __video_register_device(struct video_device *vdev, /* Part 5: Register the entity. */ ret = video_register_media_controller(vdev); - /* Part 6: Activate this minor. The char device can now be used. */ + /* + * Part 6: Complete the video device registration by initializing the + * default context. The defaul context serves for context-aware driver + * to operate with a non-context-aware userspace that never creates + * new contexts. If the video device driver is not context aware, it + * will never implement 'context_ops' and will never use the default + * context. + */ + vdev->default_context = NULL; +#ifdef CONFIG_MEDIA_CONTROLLER + if (vdev->entity.ops && vdev->entity.ops->alloc_context && + vdev->entity.ops->destroy_context) { + ret = vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vdev->default_context); + if (ret) { + pr_err("%s: default context alloc failed\n", __func__); + goto cleanup; + } + + ret = media_device_bind_context(vdev->v4l2_dev->mdev->default_context, + &vdev->default_context->base); + if (ret) { + pr_err("%s: default context bind failed\n", __func__); + goto cleanup; + } + } +#endif + + /* Part 7: Activate this minor. The char device can now be used. */ set_bit(V4L2_FL_REGISTERED, &vdev->flags); mutex_unlock(&videodev_lock); @@ -1094,6 +1130,10 @@ int __video_register_device(struct video_device *vdev, cleanup: mutex_lock(&videodev_lock); +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context = NULL; if (vdev->cdev) cdev_del(vdev->cdev); video_devices[vdev->minor] = NULL; diff --git a/include/media/media-device.h b/include/media/media-device.h index 2ea8fce9ba75700286961f1622584372a954cb8a..b3cc6793a8b5eff4c26e57b01e1a62ab71e8195b 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -184,6 +184,9 @@ struct media_device_ops { * @fh_list: List of file handles in the media device * (struct media_device_fh.mdev_list). * @fh_list_lock: Serialise access to fh_list list. + * @default_context: The default video device context. Used by drivers that + * support multi-context operation when operated by a + * non-context aware userspace. * * This structure represents an abstract high-level media device. It allows easy * access to entities and provides basic media device-level support. The @@ -260,6 +263,8 @@ struct media_device { struct list_head fh_list; spinlock_t fh_list_lock; + + struct media_device_context *default_context; }; /* We don't need to include usb.h here */ diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 9276e095fb17414a9eb7845db0aa81572c42ca45..9e1cf58623acc4ab5f503a9663fd999b38130226 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -237,6 +237,8 @@ struct v4l2_file_operations { * @ctrl_handler: Control handler associated with this device node. * May be NULL. * @queue: &struct vb2_queue associated with this device node. May be NULL. + * @default_context: &struct video_device_context associated with this device + * node * @prio: pointer to &struct v4l2_prio_state with device's Priority state. * If NULL, then v4l2_dev->prio will be used. * @name: video device name @@ -283,6 +285,7 @@ struct video_device { struct vb2_queue *queue; + struct video_device_context *default_context; struct v4l2_prio_state *prio; /* device info */ From patchwork Thu Jul 17 10:45:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116035 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucM9k-0005Ec-2u for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:48:06 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9082656267F for ; Thu, 17 Jul 2025 10:48:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3F1B32BD012; Thu, 17 Jul 2025 10:46:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ja8iQTKh" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C8ED2BCF4D; Thu, 17 Jul 2025 10:45:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749160; cv=none; b=WZFBy7NBKn2B8tNRaA/467TQ6cJP2ud7+h5oGD/YCombm3kK9qs/rm7GuoJ7ay0eMi4EFzbB8UUaiPE1tbCzhZotAdHxbC5G7xZp4uQ0+0TffIQeYfYljohVmNrZmuwKp5W/2fITVAwVJQRr7Vmb+ebODsZrgQL+87HJmB4QyP0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749160; c=relaxed/simple; bh=197W8u1gSO7XU4oga3u7Ge9U/VY49m37BOEZ2Y4hDcg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=E6qHaVaUsz0ESgijSQctox83TeJEnCEs3TmoWcELl2F9oNEVI9qoQfI18vg9onR6aAq6cOBlcb+THAJT05QDCLDq3ZIl7+g59jb/oA4RohsNcV4ugzL22ND2u1oNjinNjs5YDa98VKWJ8ICIVHh20TZ2rPlnhl5l5Zlkz7g59W4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ja8iQTKh; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8609C21F2; Thu, 17 Jul 2025 12:45:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749119; bh=197W8u1gSO7XU4oga3u7Ge9U/VY49m37BOEZ2Y4hDcg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ja8iQTKheZrLtCb1fEdWhqHs4ashqfpbS6Qc77UCXxliQQ1HArA2OrSDb6YapkbMO estn3R9optM/LLPZjT0X4kvpP7mKvO/HJCyQPNn5bHRm0429McUbs0cRPx1JX/A6x3 A4nUgW+bNdItefXcwOzffLuMqkgTZN38VR2tBQt4= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:35 +0200 Subject: [PATCH 09/26] media: v4l2-dev: Add video_device_context_from_file() Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-9-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3667; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=197W8u1gSO7XU4oga3u7Ge9U/VY49m37BOEZ2Y4hDcg=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRPaIxen4Bw+KjcuuI/aC5KhOiFFpyZ9JkIi eu2RNObyE2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PMX6D/4380V0vWPd3gLiLX2RuDp2c2NJYxbw0KLgTkZAmggpTnVeZHIBeZ1lgcUOQM34GAhqQBl Skv2pazfVaQZio+jfqnSTW5526fyb56tq3618wm4Wanq2R1MKN7lsBkqNtmgMWR6Uod2lsHKHhd bnmhhyvNFUKkpKAcEmMcWctv+9cRZaBhpRjtkaFFI7jGL8i/Cj8q99Qp0VFx85u9wBnbsxHeV91 qcUEZV2t7IfqOacFeoNhCMg95wB31iyQHVDMir8Mnc0lqd/VSc6j4MN0glKjaz+EWGFeLYeUD9i 1v/wJdyZiC+6ECKFTgL5yn2bu2vuqhqAKcckj9/I0gIB0EfToDlNdHfK2bqQ43+Dre5l2J5VVNb +1b+qZ9rt7VNuOdfVNLDoC6PWGYFOR31U2syyfW2KsJhc+SjfK+njqVBuZ+QggRHyRJsP9/Z7K/ QKmdk2lExSHvmlVR4ZiuzJxf22lxDPYfnHl3ZfwBqfqlP+o0UWY4MeHZOijMezr/dOOZc3QRyAd aLrOHG1tYEPHoynqFzqmlzLkBnJzNJa7IMFkkJUiq9eMGkmD9f1YQKhtYH8P/vR/ozlki/XLZht DPANRuh9Z9U+kckKjZ6tuWZHZabo/whsWs8+VQIlS0lvM5mJdiS1ITOf025nkHF1UmRanm36ikM cdpq3tAQIE/ogQQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce an helper function to get a video_device_context from file. If no context has been bound to the file, return the video device default context. As a video device context lifetime is bound to the one of the file handle it is associated with, and the intended user of this helper are the v4l2_ioctl_ops handlers that operates on an open file handle, the context returned by this function is guaranteed to be valid for the whole function scope of the caller. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 15 +++++++++++++++ include/media/v4l2-dev.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 2d78096931efd88215bc847b105e198a54035546..96696959314abfb1864ea5d96742e579b5a41f6f 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1180,6 +1180,21 @@ struct dentry *v4l2_debugfs_root(void) EXPORT_SYMBOL_GPL(v4l2_debugfs_root); #endif +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vfd) +{ + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vfd->flags) ? filp->private_data + : NULL; + + /* If the file handle has been bound to a context return it. */ + if (vfh && vfh->context) + return vfh->context; + + return vfd->default_context; +} +EXPORT_SYMBOL_GPL(video_device_context_from_file); + #if defined(CONFIG_MEDIA_CONTROLLER) __must_check int video_device_pipeline_start(struct video_device *vdev, diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 9e1cf58623acc4ab5f503a9663fd999b38130226..a41afb81663bc9cb3bfc06dcf9b11ceeaf7c3415 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -789,6 +789,36 @@ int video_device_init_context(struct video_device *vdev, */ void video_device_cleanup_context(struct video_device_context *ctx); +/** + * video_device_context_from_file - Get the video device context associated with + * an open file handle + * @filp: The file handle + * @vdev: The video device + * + * If a video device context has been bound to an open file handle by a call + * to the VIDIOC_BIND_CONTEXT ioctl this function returns it, otherwise returns + * the default video device context. + * + * The intended callers of this helper are the driver-specific v4l2_ioctl_ops + * handlers, which receive as first argument a file pointer. By using this + * helper drivers can get the context associated with such file, if any. + * Otherwise, if the video device has not been bound to any media device + * context, the default video device context is returned. + * + * As video device contexts are reference counted and their lifetime is + * guaranteed to be at least the one of the file handle they are associated + * with, callers of this function are guaranteed to always receive a valid + * context reference by this function. The reference will remain valid for the + * whole function scope of the caller that has received the open file handle as + * argument. Likewise, accessing the media context from the video device context + * returned by this function is always safe within the same function scope. + * + * This function does not increase the reference count of the returned video + * device context. + */ +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vdev); + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ From patchwork Thu Jul 17 10:45:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116041 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMAL-0005FT-0I for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:48:47 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id EA6B35823F2 for ; Thu, 17 Jul 2025 10:48:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 548CF2BDC17; Thu, 17 Jul 2025 10:46:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YYiuPOep" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9800F2BD5B3; Thu, 17 Jul 2025 10:46:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749163; cv=none; b=ZGTI5Vsay7LY40t5X+05xIrQd75219MM+YwjogZzVayceLa0F01dP2grqsjgl90M+7G8Gf2JMlpoTx+Q49npzuGH6EAHIUZc+jL7glmXxUAlSVLKR6xLc6YmvUPR/Fm3IjgPkfGTGRHrxj/92XC5EmdHuJvL313SVLtMQEi03P4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749163; c=relaxed/simple; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bzxz8pLQFUQ0HhVZfH+EdgjDWTkuzqYDO2Rw8yHlvxEHyKn7IgkyK98g/LULBgFnW/MrHUgmzaEdbpeIYNwcOBrgW1FfrnAhJ9h9KGjv74Anz8HuT4cZVg44FxSJRnEAzvlPGgN7J8YjrM2aS3Hwsun7AYl9aAUU4UxaBY0ba0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=YYiuPOep; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1E872245A; Thu, 17 Jul 2025 12:45:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749121; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=YYiuPOeprmFLaNJmzOXHBcYHGT8fBlsFW4+sj1s+N64Tw+55Rw/PTCnB94yCebWYN VEZOlOSWEcgJJ2pJG5nm3fPkBnUUYETbkiemAh9Qkj9CAOwlk8dwwbsLCuqBjo/484 R9u+juYMLuvIoUHww25r+AOojgmci9b8bpoHw2+k= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:36 +0200 Subject: [PATCH 10/26] media: v4l2-dev: Add video_device_context_from_queue() Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-10-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3081; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=mLUIKlRtGcg7V5Vy0wqmsYRGL64MqtvU55q1oBc3ZZs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRPd2EOVo7NzC+9Tgxe4bjIIOEosUfZWrH3d jZq5MccBbCJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUTwAKCRByNAaPFqFW PPLwD/9GLnCoGIVQv4Jy8/8f0VN/n8yLhbkFFqZOcYTdZThOGrK5bXosdH7Ef9dwIY9kNLXxth8 3/OR92mYXXtLciXPDGfd4GfPBWFO0XT1Af6bbIbT8fJCnntcJiiaPZhDPa6MKa4nCRi37F7579D IXJCAWZABJ9eeUdkaXtOTRcNt4s4qfB1erA/O/fveLJhp4P2NAVkFG9RE2Bc3Q/H35w7lM70H2f ckFJlH3LsiclEi+alz9FcT+wjCoXuIDB+fewxdTCD7HXun9nQExxapYiSmz7ZqZVVb/lZ8FP9L9 PY1gagHJfGbGFv7iF/huAHRF7UMC8TSWUQAjOno6ZZOswchTKH0Z4OASVrQkMW0DJgWTVx7V+rx Yw42g2rTfYzPRLkq/Os0aZl8dss1ma6j2lhh0gC9l7Ths/nsgblQhylZfHOcapKiQXCeiFsDZYr Ds+9dFlLoxrG5+rFS9EIqr5lE7gTAX8irvmUhB43r8I5ayc7kr7RGZ/CpwPg/bWxh4aECyLNwqQ kdFR8PLFg6+nrIamxPHqKhR2If85q6GvPDyjoV7Dy14HLGQVmtmCUrTGIXfFcmY98rw8j751K+G TmpjgzkS/f8oP6STb+jqmhRfuQ8MNU39QkPDqC+Nqa+/KzhqoqfrAu0uM1VfFjHT7ElY9MoA7Ms Tr2NADj0jvjubow== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Introduce an helper function to get a video_device_context from a videobuf2 queue. Device drivers that support multi-context operations will receive in their vb2_ops callbacks implementation a vb2_queue argument that is embedded in a video device context, either a context created by binding the video device to a media device context or the video device default context. This helper allows those drivers to retrieve the context the vb2_queue is embedded with. As the intended callers of this helpers are vb2_ops callbacks implementation, called by the videobuf2 core in response to an operation on a file handle, the returned context is guaranteed to be valid for the whole duration of the callback. Signed-off-by: Jacopo Mondi --- include/media/v4l2-dev.h | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index a41afb81663bc9cb3bfc06dcf9b11ceeaf7c3415..bab4b13b109362bec84d8d16440b6ea895206b60 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -819,6 +819,42 @@ void video_device_cleanup_context(struct video_device_context *ctx); struct video_device_context * video_device_context_from_file(struct file *filp, struct video_device *vdev); +/** + * video_device_context_from_queue - Get a video device context associated with + * a vb2 queue + * + * @q: the videobuf2 queue + * + * Return the video device context this videobuf2 queue belongs to. + * + * Device drivers that support multi-context operations will always be provided + * with a device context to work with, either a context created by userspace + * by binding the video device to a media device context or the video device + * default context. The videobuf2 queue that context-aware drivers operates will + * always be contained in either one of those contexts. + * + * This function should be used by driver-specific callbacks implementation of + * the vb2_ops which receive a videobuf2 as argument. Being the vb2_ops callback + * called by the videobuf2 core in response to a userspace operation on an open + * file handle, callers of this function are guaranteed to always receive a + * valid context reference by this function. The reference will remain valid for + * the whole function scope of the vb2_ops callback that has received the + * videobuf2 queue as argument. Likewise, accessing the media device context + * from the video device context returned by this function is always safe within + * the same function scope. + * + * Drivers that do not support multi-context operations shall never use + * this function. + * + * This function does not increase the reference count of the returned video + * device context. + */ +static inline struct video_device_context * +video_device_context_from_queue(struct vb2_queue *q) +{ + return container_of(q, struct video_device_context, queue); +} + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ From patchwork Thu Jul 17 10:45:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116044 Received: from am.mirrors.kernel.org ([147.75.80.249]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMAV-0005G5-1Z for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:48:55 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id BE20E1AA3BDE for ; Thu, 17 Jul 2025 10:49:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 06C512BDC30; Thu, 17 Jul 2025 10:46:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rH23bMK9" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF81F2BD001; Thu, 17 Jul 2025 10:46:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749163; cv=none; b=bJzyYMUNM+nspPCzuRwceZnfKi1B97XPEKovDxFk+M6vartmJruYGC3UZsADCKoy26tmYOzVJ10hvHadIwquT+pJ67xTEGlHxPU1oVVJMmsaTeDORc7ZQGuXU4aGERj31V9P5Z7ljF55M94JvkhWpQrEglP7yrBMvoCer3tQ62Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749163; c=relaxed/simple; bh=LUe4O1ANeWng4EOxHijMT12GIxXH8Gc6BKlXOjBsYj4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KrFke6rr2mwfBxZLnpvKsxRnwUqBuE8sq7HvRyyYr/rOhfMxNneUfNCblW6OYmH9QXl06h7f+hKGlPNZjzkmR40zcq5WGJpiV+AD0qvr4DwfbK/VnFINZR8HIyWvtEuNvMejxQlVDRvimSCgqdUsikAQbr9Ri8zY5rVBQZ85rcI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=rH23bMK9; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 827FF1E74; Thu, 17 Jul 2025 12:45:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749122; bh=LUe4O1ANeWng4EOxHijMT12GIxXH8Gc6BKlXOjBsYj4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rH23bMK9nY7OM/iczF3S4nT0Nba0tCMv1g9R5gDw9wWpU8rxJtY+zE0IsU+ro1+ET O8Fhs2ZWm8mMb51HJfusHUC9MzsBAOV51UYuiRnFy2IEfCDg0SymK2UxRFjcc7nzMP Yq4Yr9tXIevsrM1H2yoibvRlPhEWOXt0bCRwSLXk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:37 +0200 Subject: [PATCH 11/26] media: videobuf2-v4l2: Support vb2_queue embedded in a context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-11-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12110; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=LUe4O1ANeWng4EOxHijMT12GIxXH8Gc6BKlXOjBsYj4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQCFrxdssrnhkAyPfNGVcWPDjHgW3/Vz0Tm 4uK3GTfrcOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PCgTD/9Jswu3HUzFc2cgTgBtrlChHcpc9Jg5xvNDYAq8jLmjLi/05jW8jUqe+u3B/8Wly4sCKuA +k3bsZChMSsBSxK2FLAdG9El31f72fYmW7nF9lXLM02NkEIKvk46c4LbZ7+/oJSNBhxULxqtxCS Qsn0tlmQ3W4ZTnwkSTknPO9GZG8k3CGqI8TZWcUoVae7WXlzW21WZzUDVgiCSU8OtXr7eZbrXvS qLQA2pigZUDP0SjbXxm+xNQWAOln0WliT5A1h5Qbc0DSld1+2iXsPKt6kdJ8Pk+Fgv4Pu7A2TrO L2RA8Py8hFi4YPiOgWOXRzN7nqi0ruqmqncx8FA4LKoOgb4IYBUcxyLGgSVkWoAHddlGlUEHTeX uhYtj1IggdyQn18BHpt/VL50/RxxboD/K64gPmDD/3misMI8hPKwsJiUTCI0NsM9MF6pMlr2b8Y +HAH3di5EyAqpuVZyyd2y5NNQGMINqHuOxbIgdacidmQ3QX8I+kjsShGOATN7ESU/5HHjZqaOCv o4voQL4M0sJ/h2JWS1fRy8ds4JUxWZhFjUby1ukwOgzN6V9W+6/d/eHEX8PdQAnOzOc9WVcXE7O M18Okdi6tOAc7gfG2bAgx/ml4a0zqX5tWe6vThVn9h7FUhY9jz+XERxr5Pztz5nhA9AkEUoD4Gl 9muJ0jjbwaBNEPw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Support device drivers that implement multi-context operations in the videobuf2 core by providing an helper to retrieve the vb2_queue from the context associated with an open file handle. If no context is associated with a file handle, retrieve it from the video device default context, created by the core for multi-context aware drivers. Fall-back to use the vb2_queue from the video_device to support existing drivers which are not context aware. Signed-off-by: Jacopo Mondi --- drivers/media/common/videobuf2/videobuf2-v4l2.c | 135 +++++++++++++++--------- 1 file changed, 84 insertions(+), 51 deletions(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index 1cd26faee50338aefeb670c6865da7c2d43f44d3..01269e03102f9ed7731732cc5d3444664eaa7bec 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -998,23 +998,42 @@ EXPORT_SYMBOL_GPL(vb2_poll); * and so they simplify the driver code. */ +/* + * Helper to get the vb2 queue either from: + * 1) The video context bound to the open file handle + * 2) The default context for context-aware drivers if userspace has not bound + * a context to the file handle + * 3) From the video device for non-context aware drivers + */ +static struct vb2_queue *get_vb2_queue(struct file *file, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_from_file(file, vdev); + + return ctx ? &ctx->queue + : vdev->default_context ? &vdev->default_context->queue + : vdev->queue; +} + /* vb2 ioctl helpers */ int vb2_ioctl_remove_bufs(struct file *file, void *priv, struct v4l2_remove_buffers *d) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vdev->queue->type != d->type) + if (q->type != d->type) return -EINVAL; if (d->count == 0) return 0; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_core_remove_bufs(vdev->queue, d->index, d->count); + return vb2_core_remove_bufs(q, d->index, d->count); } EXPORT_SYMBOL_GPL(vb2_ioctl_remove_bufs); @@ -1022,21 +1041,21 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv, struct v4l2_requestbuffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type); + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->type); u32 flags = p->flags; - vb2_set_flags_and_caps(vdev->queue, p->memory, &flags, - &p->capabilities, NULL); + vb2_set_flags_and_caps(q, p->memory, &flags, &p->capabilities, NULL); p->flags = flags; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_core_reqbufs(vdev->queue, p->memory, p->flags, &p->count); + res = vb2_core_reqbufs(q, p->memory, p->flags, &p->count); /* If count == 0, then the owner has released all buffers and he is no longer owner of the queue. Otherwise we have a new owner. */ if (res == 0) - vdev->queue->owner = p->count ? file->private_data : NULL; + q->owner = p->count ? file->private_data : NULL; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_reqbufs); @@ -1045,11 +1064,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, struct v4l2_create_buffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, p->format.type); + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->format.type); - p->index = vb2_get_num_buffers(vdev->queue); - vb2_set_flags_and_caps(vdev->queue, p->memory, &p->flags, - &p->capabilities, &p->max_num_buffers); + p->index = vb2_get_num_buffers(q); + vb2_set_flags_and_caps(q, p->memory, &p->flags, &p->capabilities, + &p->max_num_buffers); /* * If count == 0, then just check if memory and type are valid. * Any -EBUSY result from vb2_verify_memory_type can be mapped to 0. @@ -1058,12 +1078,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, return res != -EBUSY ? res : 0; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_create_bufs(vdev->queue, p); + res = vb2_create_bufs(q, p); if (res == 0) - vdev->queue->owner = file->private_data; + q->owner = file->private_data; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_create_bufs); @@ -1072,69 +1092,76 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_prepare_buf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf); int vb2_ioctl_querybuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); /* No need to call vb2_queue_is_busy(), anyone can query buffers. */ - return vb2_querybuf(vdev->queue, p); + return vb2_querybuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_querybuf); int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_qbuf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf); int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_dqbuf(vdev->queue, p, file->f_flags & O_NONBLOCK); + return vb2_dqbuf(q, p, file->f_flags & O_NONBLOCK); } EXPORT_SYMBOL_GPL(vb2_ioctl_dqbuf); int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamon(vdev->queue, i); + return vb2_streamon(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamon); int vb2_ioctl_streamoff(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamoff(vdev->queue, i); + return vb2_streamoff(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamoff); int vb2_ioctl_expbuf(struct file *file, void *priv, struct v4l2_exportbuffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_expbuf(vdev->queue, p); + return vb2_expbuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); @@ -1143,20 +1170,22 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_mmap(vdev->queue, vma); + return vb2_mmap(q, vma); } EXPORT_SYMBOL_GPL(vb2_fop_mmap); int _vb2_fop_release(struct file *file, struct mutex *lock) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); if (lock) mutex_lock(lock); - if (!vdev->queue->owner || file->private_data == vdev->queue->owner) { - vb2_queue_release(vdev->queue); - vdev->queue->owner = NULL; + if (!q->owner || file->private_data == q->owner) { + vb2_queue_release(q); + q->owner = NULL; } if (lock) mutex_unlock(lock); @@ -1167,7 +1196,8 @@ EXPORT_SYMBOL_GPL(_vb2_fop_release); int vb2_fop_release(struct file *file) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; return _vb2_fop_release(file, lock); } @@ -1177,19 +1207,20 @@ ssize_t vb2_fop_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_WRITE)) + if (!(q->io_modes & VB2_WRITE)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - err = vb2_write(vdev->queue, buf, count, ppos, - file->f_flags & O_NONBLOCK); - if (vdev->queue->fileio) - vdev->queue->owner = file->private_data; + err = vb2_write(q, buf, count, ppos, + file->f_flags & O_NONBLOCK); + if (q->fileio) + q->owner = file->private_data; exit: if (lock) mutex_unlock(lock); @@ -1201,20 +1232,21 @@ ssize_t vb2_fop_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_READ)) + if (!(q->io_modes & VB2_READ)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - vdev->queue->owner = file->private_data; - err = vb2_read(vdev->queue, buf, count, ppos, + q->owner = file->private_data; + err = vb2_read(q, buf, count, ppos, file->f_flags & O_NONBLOCK); - if (!vdev->queue->fileio) - vdev->queue->owner = NULL; + if (!q->fileio) + q->owner = NULL; exit: if (lock) mutex_unlock(lock); @@ -1225,7 +1257,7 @@ EXPORT_SYMBOL_GPL(vb2_fop_read); __poll_t vb2_fop_poll(struct file *file, poll_table *wait) { struct video_device *vdev = video_devdata(file); - struct vb2_queue *q = vdev->queue; + struct vb2_queue *q = get_vb2_queue(file, vdev); struct mutex *lock = q->lock ? q->lock : vdev->lock; __poll_t res; void *fileio; @@ -1241,7 +1273,7 @@ __poll_t vb2_fop_poll(struct file *file, poll_table *wait) fileio = q->fileio; - res = vb2_poll(vdev->queue, file, wait); + res = vb2_poll(q, file, wait); /* If fileio was started, then we have a new queue owner. */ if (!fileio && q->fileio) @@ -1257,8 +1289,9 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags); + return vb2_get_unmapped_area(q, addr, len, pgoff, flags); } EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area); #endif From patchwork Thu Jul 17 10:45:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116047 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMAq-0005Gw-17 for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:49:13 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id C648716A09E for ; Thu, 17 Jul 2025 10:49:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6D68B2BE046; Thu, 17 Jul 2025 10:46:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t/XlmP8/" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BF712BE024; Thu, 17 Jul 2025 10:46:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749166; cv=none; b=k+vKwy1FDaCODFnat8kuO/N9GK5MPLgx5hsqPqPON6ag4LfUcoypjcn1vqplQFOCkNJg2NQY+UB5jZ50LTDtRe0DsMOlzanI4EJbHnrKDYrW493i/9riVC49+TMRxeymNV8rhtYK+qFwpk3YJd9sApDVbRB6WbxjZB8JeRfXbxU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749166; c=relaxed/simple; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AHEvOQsvxDjVXUkKRzZ0NV1FXpAkKiMd1XYv6M0z4yC3mF7qV9KIqISjBtN1Hsko2oim1rdhn/xilf11/Qvw4uR284suX7ReL60gHPwZ7oQLrXOC96b3Gf53YenM/G4HzKt/4+SzCWx5nmK/vveuksluAvwOD8ORmbFKA22sPsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=t/XlmP8/; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 17A6721F5; Thu, 17 Jul 2025 12:45:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749125; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=t/XlmP8/X5RXCxwFsmf1/nSxWNBNsq8e9euPrc+fVLWEhiORcQD/1udMaug+br6ez AWiOcssX+z06OmeERZaiRbuLzEzPtkDjQZUZ88ab6/xG8XMy3rt68DT9My74IXvloF +Hl7NNPlSBsgFmbh3Y72+q/H4AMXjEFhVKZDfO4s= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:38 +0200 Subject: [PATCH 12/26] media: v4l2-subdev: Introduce v4l2 subdev context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-12-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8677; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=7I13wywrt99Sx2g/tHAlEPxkYo6PoehZ9YwsTffq1K4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQ8UB8+mF7gMnThRVJKgz7B8w/il6wiPY6O AMgv2ZUrh6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PDFcD/0etgZd6NlXIO9TufXayrKZfYsSFzxTtO9JnydgaM3udlm/IOAIggj9+M9nI5Y2XyxDhvs TUwyYiKCy9c1tGeZWWQBVqRrFZzFPaFzvxQhd0RCId4J+SHkBpcYXFKA5a/1hci37cMI3aGJSWf UvAGs+ilwBNcohu3IhSXCRgyr/JEOyrRdJuNhcy6XPeXxKaYDnXLw6FTxPUhLWFzXitNjDtrLA3 cxzFpLtMgeaQvpzjlJP0YCULQ7SKZMXutvrUTr8vReQabcOog6niSXgsRr14+cdxll+O6nGfpTw jp3ZSScspLTzn5bOBgPoXrroEsBnFv2hrZ7y3l+rYaHhHBUlT22dUuRAjHpnD/b5Ddg0lD+dXkP /T5aEqiFLM8fnFIUwW/9ppE4UtrTD+T3exjtGNH8wveQ7iN4DBz6u5FgnbmlTURPpawT7AGxi7e 8pklFf62glLv/JsKm1OzQyvDqZ5yMvrzufbNXAXBFHmCNatwW1doSsbDBcQ1HDjoKxBjEVevnvn WCDxVnEW/lS0feUxF3pJ2A9JTr9WxmONdHETs/k9aSTOTEtB2YEXednSpIGKKhvD9kmixGYFvvX nJ1lpNx/jWItJz3SXrgyAX9RRs7D74n23310B/ryVjqp+PeD4FqK4i8NGRjMd9H3Le4YRJUXgVa PGNLSbkxcdGzUUA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Introduce a new type in v4l2 subdev that represents a v4l2 subdevice contex. It extends 'struct media_entity_context' and is intended to be extended by drivers that can store driver-specific information in their derived types. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 39 +++++++++++ include/media/v4l2-subdev.h | 126 ++++++++++++++++++++++++++++++++++ 2 files changed, 165 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 4fd25fea3b58477056729665706ddbacc436379c..7307f57439499c8d5360c89f492944828ac23973 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1577,6 +1577,45 @@ bool v4l2_subdev_has_pad_interdep(struct media_entity *entity, } EXPORT_SYMBOL_GPL(v4l2_subdev_has_pad_interdep); +struct v4l2_subdev_context * +v4l2_subdev_context_get(struct media_device_context *mdev_context, + struct v4l2_subdev *sd) +{ + struct media_entity *entity = &sd->entity; + struct media_entity_context *ctx = + media_device_get_entity_context(mdev_context, entity); + + if (!ctx) + return NULL; + + return container_of(ctx, struct v4l2_subdev_context, base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_context_get); + +void v4l2_subdev_context_put(struct v4l2_subdev_context *ctx) +{ + if (!ctx) + return; + + media_entity_context_put(&ctx->base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_context_put); + +int v4l2_subdev_init_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context *context) +{ + media_entity_init_context(&sd->entity, &context->base); + + return 0; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_init_context); + +void v4l2_subdev_cleanup_context(struct v4l2_subdev_context *context) +{ + media_entity_cleanup_context(&context->base); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup_context); + struct v4l2_subdev_state * __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name, struct lock_class_key *lock_key) diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 5dcf4065708f32e7d3b5da003771810d5f7973b8..9d257b859acafb11cfe6976e906e7baabd0206f6 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -757,6 +757,78 @@ struct v4l2_subdev_state { struct v4l2_subdev_stream_configs stream_configs; }; +/** + * struct v4l2_subdev_context - The v4l2 subdevice context + * @base: The media entity context base class member + * @state: The subdevice state associated with this context + * + * This structure represents an isolated execution context of a subdevice. + * This type 'derives' the base 'struct media_entity_context' type which + * implements refcounting on our behalf and allows instances of this type to be + * linked in the media_device_context contexts list. + * + * The subdevice context stores the subdev state in a per-file handle context, + * userspace is allowed to multiplex the usage of a subdevice devnode by opening + * it multiple times and by associating it with a media device context. This + * operation is called 'bounding' and is performed using the + * VIDIOC_SUBDEV_BIND_CONTEXT ioctl. + * + * A subdevice context is created and stored in the v4l2_fh file handle + * associated with an open file descriptor when a subdevice is 'bound' to a + * media device context. The 'bounding' operation realizes a permanent + * association valid until the subdevice context is released. + * + * A subdevice can be bound to the same media device context once only. + * Trying to bind the same subdevice to the same media device context a + * second time, without releasing the already established context by closing the + * bound file descriptor first, will result in an error. + * + * To create a subdevice context userspace shall use the + * VIDIOC_SUBDEV_BIND_CONTEXT ioctl that creates the subdevice context and + * uniquely associates it with a media device file descriptor. + * + * Once a subdevice file descriptor has been bound to a media device context, + * all the operations performed on the subdevice file descriptor will be + * directed on the just created subdevice context. This means, in example, that + * the subdevice state and configuration is isolated from the ones associated + * with a different file descriptor obtained by opening again the same subdevice + * devnode but bound to a different media device context. + * + * Drivers that implement multiplexing support have to provide a valid + * implementation of the context-related operations in the media entity + * operations. + * + * Drivers are allowed to sub-class the v4l2_subdevice_context structure by + * defining a driver-specific type which embeds a struct v4l2_subdevice_context + * instance as first member, and allocate the driver-specific structure size in + * their implementation of the `alloc_context` operation. + * + * Subdevice contexts are ref-counted by embedding an instance of 'struct + * media_entity_context' and are freed once all the references to it are + * released. + * + * A subdevice context ref-count is increased when: + * - The context is created by bounding a video device to a media device context + * - The media pipeline it is part of starts streaming + * A subdevice context ref-count is decreased when: + * - The associated file handle is closed + * - The media pipeline it is part of stops streaming + * + * The ref-count is increased by a call to v4l2_subdev_context_get() and is + * reponsibility of the caller to decrease the reference count with a call to + * v4l2_subdev_context_put(). + */ +struct v4l2_subdev_context { + struct media_entity_context base; + /* + * TODO: active_state should most likely be changed from a pointer to an + * embedded field. For the time being it's kept as a pointer to more + * easily catch uses of active_state in the cases where the driver + * doesn't support it. + */ + struct v4l2_subdev_state *state; +}; + /** * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations * @@ -1152,6 +1224,7 @@ struct v4l2_subdev_fh { struct module *owner; #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) struct v4l2_subdev_state *state; + struct v4l2_subdev_context *context; u64 client_caps; #endif }; @@ -1285,6 +1358,59 @@ int v4l2_subdev_link_validate(struct media_link *link); bool v4l2_subdev_has_pad_interdep(struct media_entity *entity, unsigned int pad0, unsigned int pad1); +/** + * v4l2_subdev_context_get - Helper to get a v4l2 subdev context from a + * media device context + * + * @mdev_context: The media device context + * @sd: The V4L2 subdevice the context refers to + * + * Helper function that wraps media_device_get_entity_context() and returns + * the v4l2 subdevice context associated with a subdevice in a media device + * context. + * + * The reference count of the returned v4l2 subdevice context is increased. + * Callers of this function are required to decrease the reference count of + * the context reference with a call to v4l2_subdev_context_put(). + */ +struct v4l2_subdev_context * +v4l2_subdev_context_get(struct media_device_context *mdev_context, + struct v4l2_subdev *sd); + +/** + * v4l2_subdev_context_put - Helper to decrease a v4l2 subdevice context + * reference count + * + * @ctx: The v4l2 subdevice context to put + */ +void v4l2_subdev_context_put(struct v4l2_subdev_context *ctx); + +/** + * v4l2_subdev_init_context - Initialize the v4l2 subdevice context + * + * @sd: The subdevice the context belongs to + * @ctx: The context to initialize + * + * Initialize the v4l2 subdevice context. The intended callers of this function + * are driver-specific implementations of the media_entity_ops.alloc_context() + * function that allocates their driver specific types that derive from + * struct v4l2_subdev_context. + */ +int v4l2_subdev_init_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context *ctx); + +/** + * v4l2_subdev_cleanup_context - Cleanup the v4l2 subdevice context + * + * @ctx: The context to cleanup. + * + * Cleanup the v4l2 subdevice context. The intended callers of this function are + * driver specific implementation of the media_entity_ops.destroy_context() + * function before releasing the memory previously allocated by + * media_entity_ops.alloc_context(). + */ +void v4l2_subdev_cleanup_context(struct v4l2_subdev_context *ctx); + /** * __v4l2_subdev_state_alloc - allocate v4l2_subdev_state * From patchwork Thu Jul 17 10:45:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116059 Received: from sy.mirrors.kernel.org ([147.75.48.161]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMCd-0005LN-2N for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:51:04 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 395D67B8998 for ; Thu, 17 Jul 2025 10:47:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 18D562BE059; Thu, 17 Jul 2025 10:46:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="je+6kEei" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CFBD2BDC14; Thu, 17 Jul 2025 10:46:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749166; cv=none; b=GLWsl+4lnRoYb+t0YG3aTNFwwKMUEh/FH4hXRPVirVieSMAGrQ+6XaZXN6Ps9nygOX0WkvTq12DLh8ZQtiznKjjIVgqZLx/FewLbV4sXMLO3tl8JkZ6oKblFkLc21LIp7vWLwRF6b2flQ53KYxBITj074fPAgyu+6bPbxAMrhLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749166; c=relaxed/simple; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ehCWH8JyvqrNcfEY+0RNZJ9Ctd46XNJQ0lilteYt3BSaWJ7Xqw/n7yx7x1ZE9VHLtXuOL3Jlkgi2RSq+p6OK1OxfN5PBCPZF8p7cYZnCioYJ0wYdiQdbHfGjUT/Cpopax/4yCRaDMtT/zZIGqr3BoTqW782Dh5o1uBIi1W6/fH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=je+6kEei; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4F1C8198D; Thu, 17 Jul 2025 12:45:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749126; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=je+6kEeinJv8XEGNRkmbD17Kk5mbjzKPp0lVm1E+10+pjnpBe8/YW0a0djHMgCN7a F7uXduJsjt2pwC42fT7utji23CB+VzcwWnHi02yAfzlzmIaka8rrqz81Urj2pOORIE 1tryJq4y01oXnCivQQr68c4NJCsZ8CQ7E9K4RmoQ= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:39 +0200 Subject: [PATCH 13/26] media: v4l2-subdev: Introduce VIDIOC_SUBDEV_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-13-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4492; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V4Bd2SEp0ALBPlJWG8EsVuj29YkOY/732hyVjUTvCTo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQ3HpHt0cfK1Rw4QO76tsSk1mTlN3701wEW iXhvqtbpiuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PFj2D/4lp8scJ2lbnr5pqbCOmJlJsoRtCUiKZjKW3aMvNUle89R342VeULHVHgfVXxzRGAWWxOK FsKCQz7RHzHAhKam0BiMPQWzEOSD6ZwwEPl8O5yAEANgxDJ2JVX/ino8+Jpol/frwRhPqshCx45 X3isORwRd/+DwK9qf9MwlUa3A0k8YDP2wgMibMf6imvD2Xnr7KNQxlAh+mbWeg0JO2GvNpU4Enm V6DfKRgXyuCUL/JXuzDXQ6RpsW6E3PhhK1D4NFdFUYVVNxV7oR7dG2CWpHGbA4bvQrzHNKZR/yw rD4oLiwkQ0ueLlE4NOc3QjCQvzh/FUW1ts70dGDnH8bzX8tFqMaEjtltcEVhJdUiAL4UtXn+BjX /Xk0PUgM/IgYs4bhOCIpkMj2WgvQncpmfG0f+05r6bIhKm3Czunzl2cYHiCM8BWw9AMVnsi7gU1 OkvJJa3y2Y9Xj7YpOahY5WZJdY7j6tSGGtA1q1YgxPGIk1/lkRzfvKSrjJXh6JUfe0XAVYuLzht vjz/uePhW1ID+SCx9VujYlD25AtS8H2B5ema9vG/4d493Mlfa2eEKRoBlYreQZ2yIaLhkBCim0H DL5x/PuzNmvKAWtulT0v3dX15MxOf4WL53AjSdbxLwTY0BhL7qzUAuJfTmwU8pOVzfZXoA8Oicf HQaJu+s7wBbsGWw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Introduce a new ioctl in V4L2 subdev to allocate a subdevice context and associate it with a media device context. The ioctl handler calls the entity ops to let the driver allocate a new context and initialize the subdev state there contained. The new subdevice context is bound to the media device context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated context is stored in the v4l2_subdev_fh handle that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 59 +++++++++++++++++++++++++++++++++++ include/uapi/linux/v4l2-subdev.h | 11 +++++++ 2 files changed, 70 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 7307f57439499c8d5360c89f492944828ac23973..300f84317623dd082a4cd2caec97057f972e82a3 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -607,6 +607,46 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh, v4l2_subdev_get_unlocked_active_state(sd); } +static int subdev_do_bind_context(struct v4l2_subdev *sd, + struct v4l2_subdev_context **context, + struct media_device_context *mdev_context) +{ + static struct lock_class_key key; + struct v4l2_subdev_state *state; + int ret; + + ret = sd->entity.ops->alloc_context(&sd->entity, + (struct media_entity_context **) + context); + if (ret) + return ret; + + state = __v4l2_subdev_state_alloc(sd, "context->state->lock", &key); + if (IS_ERR(state)) { + ret = PTR_ERR(state); + goto err_put_context; + } + (*context)->state = state; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret = media_device_bind_context(mdev_context, + (struct media_entity_context *) + *context); + if (ret) + goto err_free_state; + + return 0; + +err_free_state: + __v4l2_subdev_state_free((*context)->state); +err_put_context: + v4l2_subdev_context_put(*context); + return ret; +} + static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, struct v4l2_subdev_state *state) { @@ -1089,6 +1129,25 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, return 0; } + case VIDIOC_SUBDEV_BIND_CONTEXT: { + struct v4l2_subdev_bind_context *c = arg; + struct media_device_context *mdev_context; + int ret; + + if (!sd->entity.ops || !sd->entity.ops->alloc_context || + !sd->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context = media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + ret = subdev_do_bind_context(sd, &subdev_fh->context, + mdev_context); + media_device_context_put(mdev_context); + return ret; + } + case VIDIOC_SUBDEV_G_CLIENT_CAP: { struct v4l2_subdev_client_capability *client_cap = arg; diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h index 2347e266cf7516b4073c1edd43b97a3ddddb183b..6184cc0153a9dd9fbfa6729e1b6127ba4a961395 100644 --- a/include/uapi/linux/v4l2-subdev.h +++ b/include/uapi/linux/v4l2-subdev.h @@ -243,6 +243,16 @@ struct v4l2_subdev_routing { __u32 reserved[11]; }; +/** + * struct v4l2_subdev_bind_context - Subdev context information + * + * @context_fd: The file descriptor of the media_device instance the subdevice + * has to be bound to + */ +struct v4l2_subdev_bind_context { + __u32 context_fd; +}; + /* * The client is aware of streams. Setting this flag enables the use of 'stream' * fields (referring to the stream number) with various ioctls. If this is not @@ -285,6 +295,7 @@ struct v4l2_subdev_client_capability { #define VIDIOC_SUBDEV_S_SELECTION _IOWR('V', 62, struct v4l2_subdev_selection) #define VIDIOC_SUBDEV_G_ROUTING _IOWR('V', 38, struct v4l2_subdev_routing) #define VIDIOC_SUBDEV_S_ROUTING _IOWR('V', 39, struct v4l2_subdev_routing) +#define VIDIOC_SUBDEV_BIND_CONTEXT _IOWR('V', 50, struct v4l2_subdev_bind_context) #define VIDIOC_SUBDEV_G_CLIENT_CAP _IOR('V', 101, struct v4l2_subdev_client_capability) #define VIDIOC_SUBDEV_S_CLIENT_CAP _IOWR('V', 102, struct v4l2_subdev_client_capability) From patchwork Thu Jul 17 10:45:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116050 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMBP-0005Hd-0A for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:49:51 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EED2E3ADF12 for ; Thu, 17 Jul 2025 10:49:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CEF0A2BEC44; Thu, 17 Jul 2025 10:46:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gHqF9lIe" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD6092BE65E; Thu, 17 Jul 2025 10:46:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749169; cv=none; b=NPrUeCicM5DRTsrBf9G5D1l3b1iGe9n93Peq4Tz5fsFzsbrsFWF7xNrKcV0PzfaUyzVss4TcIm8KorJLkPkggOLrgl6StCWzgjIDVy31dV0EdHSBGK4OG8BqKxuVD6JTSj+0e2Ef5bB7phdwCJWtwSXSNtN/Vqy/oJG8w8Rss38= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749169; c=relaxed/simple; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AdHg7Ajwd6p+JXgm1p8yfax751YBwqdd0TsZmS3y5CaIIGXDVAwlgUMw16IfQLqs+TRZFt+I9w3I7VllTrzP1DRweaa43HZvC2fmiQDoYBYc9F8/OkQqXxriVDB0DcHfxr+ZJZopamqNDKviCKLiTvK7xCsmlE6Cw4q7qwGbQis= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=gHqF9lIe; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC5611FA4; Thu, 17 Jul 2025 12:45:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749128; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=gHqF9lIeEXEIlFgd4UBuefsJQTggK9svNRtnb81kGndV4XvtOpBQyKp7HpVqC33jG j923ZKbxt9FcNa7VCxVs+115V+eCoM7q9ODKCQFzYKJJOv44qFvHPClzNy0P4mY21w ZZ+YdHCMatFtoyEzYNU10sMMWNI3IUqB3MlwD5xw= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:40 +0200 Subject: [PATCH 14/26] media: Documentation: Add VIDIOC_SUBDEV_BIND_CONTEXT Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-14-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4626; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=u88FSqcsc0N2ZFvrgby11vKsM4ytR+rXokZ729cwq9o=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQk1yOJoEDek6iiL2eleDdyKYZutPxtgquz FOroGfbEWaJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PN69D/46DkQRh3CTHBkRTCWVoMjubo1kkCjNP4B+/WR73mLjVBKAX3MWevQbqPeGbnitV12MmZd rn+gvsWaDHprI755i31DtB8rs+aluAGdE4my3Rva2Zoa3IqOBkPdfDQVFiJo1HEHRaQAGfY6dWb 6OHxfFyvEpeT1itHoH/4CkNVdQQKSiqriX4XDoTLH/1nQFFOCJITTCnV4N6jpiw1NGo+hsrepmA Ui505YamB8RGRLI3ZBG9Ia0EsBpROXP8p5VDDqmvWNqqDIpPKRjJB+iQOw5D3XSOUwpJas4UJfI +U/GxaZPXmn7y6VoYhkQqWQ2I1weayGFzUATx2f7C6zEvOr9nCgcQC3WjE8oqaXs8SJwG5lQ5cB KMxjeLke9cbbLGu9C7q/Q9A2ThNPjvmbXaENHXRauKCkqiuupVBgvH0SWaAZ3LJua03FMc++oIb ZzMUUGK046RkDj8fRmStkYcj9yZokZMbNo+i85RWUZZs9Sw29mtowlJu+HbUNO9Och/Kt3kaik3 JQq2uyPfSshzKHv7QGee1MxW1IwaFn+DUutm0GrLK6o3jkmLg7JX17uVItROVX5VrVkAGb3XWpq lIwQoK64UIMarGD72EQVX/c3+cQQpsnRjq4qIKnje1JEDWM2sTvE1Hl9ixMnYYa+UueHoC8lgzR mrpKSKoraoHdJrg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Document the newly introduced VIDIOC_SUBDEV_BIND_CONTEXT ioctl. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-subdev-bind-context.rst | 81 ++++++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst index 0d9aff56ab653b2a4f6afe4828f88bc5637addf1..236847a1d6cb2a266bea30ecf7583979099b0343 100644 --- a/Documentation/userspace-api/media/v4l/user-func.rst +++ b/Documentation/userspace-api/media/v4l/user-func.rst @@ -66,6 +66,7 @@ Function Reference vidioc-remove-bufs vidioc-s-hw-freq-seek vidioc-streamon + vidioc-subdev-bind-context vidioc-subdev-enum-frame-interval vidioc-subdev-enum-frame-size vidioc-subdev-enum-mbus-code diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-context.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-context.rst new file mode 100644 index 0000000000000000000000000000000000000000..5cba529a3cdcb63c7257f871d667fa792c0ca382 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-bind-context.rst @@ -0,0 +1,81 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: V4L + +.. _vidioc_subdev_bind_context: + +******************************** +ioctl VIDIOC_SUBDEV_BIND_CONTEXT +******************************** + +Name +==== + +VIDIOC_SUBDEV_BIND_CONTEXT - Bind a subdevice file handle to a media device +context + +Synopsis +======== + +.. c:macro:: VIDIOC_SUBDEV_BIND_CONTEXT + +``int ioctl(int fd, VIDIOC_SUBDEV_BIND_CONTEXT, struct v4l2_subdev_bind_context *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`v4l2_subdev_bind_context`. + +Description +=========== + +Applications call the ``VIDIOC_SUBDEV_BIND_CONTEXT`` ioctl to bind a subdevice +file handle to a media device context. Binding a subdevice file handle to a +media device context creates an isolated execution context which allows to +multiplex the usage of a video device. This means, in practice, that the +subdevice configuration (format, sizes etc) applied on a file handle bound to a +media device context won't be visible on file handles bound to a different media +device context (or not bound at all). + +By opening a media device applications create a media device context to which +video devices and subdevices file handles can be bound to. The file descriptor +returned by a call to :c:func:`open()` on the media device identifies uniquely +the media device context. Application populates the ``context_fd`` field of +:c:type:`v4l2_subdev_bind_context` with the file descriptor of an open media +device to identify the media context to which they want to bind a subdevice +to. + +Applications can open a subdevice node multiple times, and call +``VIDIOC_BIND_CONTEXT`` on each file handle returned by a successful call to +:c:func:`open()` to isolate the operations performed on that file handle from +any operation performed on other file handles bound to different contexts. This +means, in example, that the subdevice format and sizes are isolated from the +ones associated with a file descriptor, obtained by opening the same subdevice +but bound to a different media device context (or not bound at all). + +The bounding operation realizes a permanent association valid until the +subdevice context is released by closing the file handle. + +A subdevice file handle can be bound to the same media device context once +only. Trying to bind the same file handle to the same media device context a +second time, without releasing the already established context by closing the +bound file descriptor first, will result in an error. + +Bounding is an opt-in feature that applications are free to ignore. Any +operation directed to a non bound file handle will continue to work as it used +to, and the video device configuration (formats, sizes etc) will be visible +across all the other non-bound file handles. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes ` chapter. + +EINVAL + The media device context file handle ``context_fd`` is not valid or the + subdevice file handle is already bound to a context. From patchwork Thu Jul 17 10:45:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116068 Received: from sy.mirrors.kernel.org ([147.75.48.161]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMDe-0005NC-0s for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:52:07 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 73DB37B6017 for ; Thu, 17 Jul 2025 10:48:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 34E742BEFF7; Thu, 17 Jul 2025 10:46:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mNMMVCC3" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B2C92BE044; Thu, 17 Jul 2025 10:46:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749170; cv=none; b=A6FVX2Gc/KJ1H/UVYFEz0f3b1EWPKAVJpYx1lT31x9RSsOAfxNWsxWcpuCXkJG6+HNGuiO+Jnme+GNrVpju0bOodo08h1PnBOpYZ2W5vCSdI5ZtrmKbkHeN9D9Bd8h9BuN2eam5hYXejqW+7rfTXbPUv+k1pxjes5P7JTmOaxMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749170; c=relaxed/simple; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D5KGYgepfJNyIixaDyLaQHqmx0hBy6CAYvGHNq9aKKfefv+c0/4kY563HUbwfLoefGqruK+tK6sDuHf6OHMAJuYHQ8MMZNEXQI/xml1V/u60F7E03PJUuJXq3l9U0T+Wfa0Psz8DXNRRNdrrdDaqn3sgznhRb0ihvNNbWZAD/98= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=mNMMVCC3; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 501251AE2; Thu, 17 Jul 2025 12:45:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749129; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mNMMVCC3G4ShiWZaEq6mzKdKIIvbhdEhL7lYffpkMdsnQ8rw855e7y4uYYRfcWRhu MDeoCR7C5xJ3069Xg2L5eDFf/fkDtZY6DWvkWVs0dC5CTpZlOi4+ophz1HfJjGUH9C B3wRa0M3acbQn4AYBwnnZLC98kKE37EQT6466KZk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:41 +0200 Subject: [PATCH 15/26] media: v4l2_subdev: Introduce default context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-15-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5859; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=RMEJm/E/usNBixZcp+GBwmSg7HkpwQhhGNile6Mrpfg=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQARFaawY5Ymb5LCqC6Mrgnq+bBHXeD5/Vw bms7lMdbF2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PBGkD/4uAZ7atbk/AkFfdYiHt0SfIDUqI6aViU0Mtz0FbtztrBockfTaMn+bueBtFJoYyIzcpx4 XO+LGYBxU9aaEh0+LL/6T62Bey/7IjbwVemMjGn0wK4UZGiusTTQD2PvLAMA/KwO2bWs+GpZTg4 Knk11XQu6ppEBLrTntpaJQVvOiW3Cu3Z8bRMRYHf1D7mUyjT1fymUflQ4FstF4DEFnjjg48AP0s V+5trxEaV4C5irKmPRPM3swpDPDIH7lKbLTqhoLkY9mlwv1Adr6xGg3UkuGEMVx/lLtih04+iHx khz1JBWiL6Sd1+xBByDonmHY3XsbOp4SoNo95KaRGg6GPScH8xmuxmDO7dFFanSwHHkNDbkf7yj 6IqRJ1J7i4QvJuWmzEgQjpkJTQ1s39Q57AVwRRyW0fSBeNo69MTBjYUotTw09yP4TdUkmrhrl4p ngdPZkIUoUKKNPffqRW0fiOtuxZKlpqNh3AOldf9mnI4jxAMSexk8gRk5EMJekXlUbVpHJKlTdA yvmTSpexf5Vb1IqZqCVHSXawZtzw77omwsZbHuddY7bORnaEEARz9cdaz1SdFxGB9UiXQVxud+T Fp64lSgAK1EBd22vq3dXzctq4RGapKoWRMeZ2OWkzU2G4ot0bVpR77YndVTo7KX+TJutcW0IeLs a0G453vqAkf1Ftg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Introduce a default context for v4l2 subdvice. Drivers ported to use multi-context support that used to work with a non-context aware userspace (which doesn't call VIDIOC_SUBDEV_BIND_CONTEXT) shall continue to work even if they are context aware. Provide a default context in the v4l2 subdev and bind it to the media device default context when the subdevice is fully registered by providing a v4l2_subdev_registered() function. Release the context when the subdevice gets unregistered by the core, providing a v4l2_subdev_unregistered() helper. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-device.c | 11 +++----- drivers/media/v4l2-core/v4l2-subdev.c | 50 +++++++++++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 29 ++++++++++++++++++++ 3 files changed, 83 insertions(+), 7 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-core/v4l2-device.c index 5e537454f5cd71b3c50a2a2864642f7d5548047b..bf3ebd77b7bb8b13c849a89d01e6d889e8a2e4fd 100644 --- a/drivers/media/v4l2-core/v4l2-device.c +++ b/drivers/media/v4l2-core/v4l2-device.c @@ -146,11 +146,9 @@ int __v4l2_device_register_subdev(struct v4l2_device *v4l2_dev, } #endif - if (sd->internal_ops && sd->internal_ops->registered) { - err = sd->internal_ops->registered(sd); - if (err) - goto error_unregister; - } + err = v4l2_subdev_registered(sd); + if (err) + goto error_unregister; sd->owner = module; @@ -274,8 +272,7 @@ void v4l2_device_unregister_subdev(struct v4l2_subdev *sd) list_del(&sd->list); spin_unlock(&v4l2_dev->lock); - if (sd->internal_ops && sd->internal_ops->unregistered) - sd->internal_ops->unregistered(sd); + v4l2_subdev_unregistered(sd); sd->v4l2_dev = NULL; #if defined(CONFIG_MEDIA_CONTROLLER) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 300f84317623dd082a4cd2caec97057f972e82a3..438f51980e5ac0f092ba6b0a979a376133968ddf 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1275,6 +1275,56 @@ const struct v4l2_file_operations v4l2_subdev_fops = { .poll = subdev_poll, }; +#ifdef CONFIG_MEDIA_CONTROLLER +static int v4l2_subdev_register_default_context(struct v4l2_subdev *sd) +{ + struct media_device_context *mdev_context; + + /* If the driver does not support contexts, return here. */ + if (!sd->entity.ops || !sd->entity.ops->alloc_context || + !sd->entity.ops->destroy_context) + return 0; + + mdev_context = sd->entity.graph_obj.mdev->default_context; + return subdev_do_bind_context(sd, &sd->default_context, mdev_context); +} +#endif /* CONFIG_MEDIA_CONTROLLER */ + +int v4l2_subdev_registered(struct v4l2_subdev *sd) +{ + int ret; + +#ifdef CONFIG_MEDIA_CONTROLLER + ret = v4l2_subdev_register_default_context(sd); + if (ret) + return ret; +#endif /* CONFIG_MEDIA_CONTROLLER */ + + if (sd->internal_ops && sd->internal_ops->registered) { + ret = sd->internal_ops->registered(sd); + if (ret) + goto err_registered; + } + + return 0; + +err_registered: + if (sd->default_context) + v4l2_subdev_context_put(sd->default_context); + return ret; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_registered); + +void v4l2_subdev_unregistered(struct v4l2_subdev *sd) +{ + if (sd->default_context) + v4l2_subdev_context_put(sd->default_context); + + if (sd->internal_ops && sd->internal_ops->unregistered) + sd->internal_ops->unregistered(sd); +} +EXPORT_SYMBOL_GPL(v4l2_subdev_unregistered); + #ifdef CONFIG_MEDIA_CONTROLLER int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity, diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 9d257b859acafb11cfe6976e906e7baabd0206f6..1fa42a9f322be0be44fc9308744f4f4ae0cf1606 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1130,6 +1130,10 @@ struct v4l2_subdev_platform_data { * @active_state: Active state for the subdev (NULL for subdevs tracking the * state internally). Initialized by calling * v4l2_subdev_init_finalize(). + * @default_context: Default context for the subdev, allows to operate + * context-aware drivers with a context-unaware userspace. + * It is initialized when the subdev is registered in + * v4l2_subdev_registered(). * @enabled_pads: Bitmask of enabled pads used by v4l2_subdev_enable_streams() * and v4l2_subdev_disable_streams() helper functions for * fallback cases. @@ -1182,6 +1186,7 @@ struct v4l2_subdev { * doesn't support it. */ struct v4l2_subdev_state *active_state; + struct v4l2_subdev_context *default_context; u64 enabled_pads; bool s_stream_enabled; }; @@ -1286,6 +1291,30 @@ static inline void *v4l2_get_subdev_hostdata(const struct v4l2_subdev *sd) return sd->host_priv; } +/** + * v4l2_subdev_registered - Subdevice registered notification + * + * @sd: The subdevice that has been registered + * + * Notify that a subdevice has been registered by the core. This function wraps + * a call to sd->internal_ops->registered (if available) and instantiates the + * default v4l2 subdevice context. + * + * Returns 0 on success, a negative error code otherwise. + */ +int v4l2_subdev_registered(struct v4l2_subdev *sd); + +/** + * v4l2_subdev_unregistered - Subdevice unregistered notification + * + * @sd: The subdevice that has been unregistered + * + * Notify that a subdevice has been unregistered by the core. This function + * wraps a call to sd->internal_ops->unregistered (if available) and deletes + * the default v4l2 subdevice context. + */ +void v4l2_subdev_unregistered(struct v4l2_subdev *sd); + #ifdef CONFIG_MEDIA_CONTROLLER /** From patchwork Thu Jul 17 10:45:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116083 Received: from sy.mirrors.kernel.org ([147.75.48.161]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMFF-0005Qd-0U for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:53:46 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 735CE7BACAC for ; Thu, 17 Jul 2025 10:48:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DD17229ACE6; Thu, 17 Jul 2025 10:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ExdYgcp0" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F32812C08B0; Thu, 17 Jul 2025 10:46:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749172; cv=none; b=LeswXAI7OjJlCGixJS9lAeKIG4s0PVo5m6A0NzZNF4f64JYa/KMWvNHMNLVFS7vv8gD+AeEavSzeK7i7V7HQv6UdOR5nyNaBSANtIwIKi2dUNKr812Yv5f1kC3J8PRga14BespRR1j95HyW2Ss4Ai7bcqbIczeLbrIT+aMWTN2c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749172; c=relaxed/simple; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=J/9/Mh96cJXajb0b4c++Vcs94SI9QIJ89shb0SXhxeNWR+z4JRYAn+Bmzyh9jOxbkbaJlIZ0sMzzdPp8A9XAdrR085HeA3XloZdIo7rI6uhyVM3ze6XYhaj2pKe03wycCtH/Inbo958fhLVnran6snoyH1GRexHDsleWSva7ev0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ExdYgcp0; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C4A0521F2; Thu, 17 Jul 2025 12:45:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749131; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ExdYgcp0jOZrgrTJ3NTwlKNEE7iLSHOFyghIB051ctiBngx1VTdYq6OnU4YYkUpt7 sR5VEDheliciCq/T4+axPVRHx8TNwyfplpEV7FvV4QGuJwOECuZRpZEw0qRuuYC9v3 qrTwqgN7iuAzpiRMC52H+B576ZzT/8etsj6M/sUk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:42 +0200 Subject: [PATCH 16/26] media: v4l2-subdev: Add subdev state accessor helpers Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-16-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=10156; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=LTlCiL6fmmVVuL8bKDmzSBOA/E25dmRY67EoUr9sHTM=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQppY7U0ZAkK4bd8Un0L22hEFZVYylXs1Q/ 2N51ZSjJyiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PFCMEAC7Vpxyn3TlgIXtjyd3nHZhEEbHYqCMJIr+IccAXw5XtgqCfAqX7NCSn7VTy9B6i2eVTp/ CKYWjGpxUlygChhQMuQl0dMF/T2hI9VokTUymwEepxwIF081U2JaFJv9l2qCDSZ0s/2zX0nfo7X +zeHTMzQZNkmcpCUem26Fi9n+EO1Fm01Btbjp0+bahp/ZfvXV7LUqQ786mByz0EFcJPBKosGq2u U/10n65+FjV8VWYo7ppOXlRF4YDRzQtpKKkaK9y521Niak+iTycxbB3oF8nRDZ2439X1f8zs34p 31gm2h5HyVZIABFjwkaC5o4279SaMIhFRxNGnPhgmZ4BBvDw/dCPCcGLUR5RsWGSffR5oYQ2FIc gh3I58zpqjhxQIws1UfzS+rZZDrxyIAazguf3JGjfJdhUvs4pX+FEkzeYTTDc3jaVOGZyyJvdzS RFdG6LnwCQz4HK4Mgu87YDHTsGG9zDv66muAdASeMT0arw46ZLwLuqY7dnnIoRpySPfwy6qEAwG SKyNlxDiZn/pb0cVLGDrNuCdHEtgakwQhHruzwHjc1t/D8Hv5SV/bIDraLJFekTbPrPUD074xrK Ip56FEMo6iq57hbG2w0+1553eBfuqkuuL6MQKS57q4vtDmA0c38eVi51avRcq8knvKXq5qdJDIb ZXBVegHPTuM8MWg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no The v4l2-subdev.c file offers three helpers to access the subdevice active state from a v4l2_subdev pointer: - v4l2_subdev_get_unlocked_active_state(sd) - v4l2_subdev_get_locked_active_state(sd) - v4l2_subdev_lock_and_get_active_state(sd) With the introduction of struct v4l2_subdev_context which contains a subdev_state as well, the actual "active" state is stored in three possible places: - A context associated with a v4l2_subdev_fh for context aware drivers operated by context aware userspace - The default subdevice context for context aware drivers operated by non-context aware userspace - The subdevice active state for non-context aware drivers Provide helpers similar in spirit the existing ones but accept as argument either a subdevice the context itself and retrieve the active state from the correct place. Helpers will be used in following patches for link validation. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 101 +++++++++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 104 +++++++++++++++++++--------------- 2 files changed, 159 insertions(+), 46 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 438f51980e5ac0f092ba6b0a979a376133968ddf..7372f61127c871cec44a3d1900e2b8bef34632b9 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -189,6 +190,106 @@ static inline int check_pad(struct v4l2_subdev *sd, u32 pad) return 0; } +/* subdev state accessor helpers */ + +/* + * Access the state from the subdevice. + * + * If the driver is context-aware use the state stored in the default context + * otherwise use the active state stored in the subdevice. + */ + +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + lockdep_assert_not_held(sd->default_context->state->lock); + + return sd->default_context->state; + } + + if (sd->active_state) + lockdep_assert_not_held(sd->active_state->lock); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_unlocked_active_state_from_sd); + +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + lockdep_assert_held(sd->default_context->state->lock); + + return sd->default_context->state; + } + + if (sd->active_state) + lockdep_assert_held(sd->active_state->lock); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_locked_active_state_from_sd); + +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_sd(struct v4l2_subdev *sd) +{ + if (!sd) + return NULL; + + if (sd->default_context) { + v4l2_subdev_lock_state(sd->default_context->state); + + return sd->default_context->state; + } + + if (sd->active_state) + v4l2_subdev_lock_state(sd->active_state); + return sd->active_state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_lock_and_get_active_state_from_sd); + +/* Access the subdevice state from a subdvice context. */ +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_ctx(struct v4l2_subdev_context *ctx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + lockdep_assert_not_held(ctx->state->lock); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_unlocked_active_state_from_ctx); + +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_ctx(struct v4l2_subdev_context *ctx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + lockdep_assert_held(ctx->state->lock); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_locked_active_state_from_ctx); + +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_ctx(struct v4l2_subdev_context *ctx) +{ + if (!ctx) + return NULL; + + if (ctx->state) + v4l2_subdev_lock_state(ctx->state); + return ctx->state; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_lock_and_get_active_state_from_ctx); + static int check_state(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, u32 which, u32 pad, u32 stream) { diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 1fa42a9f322be0be44fc9308744f4f4ae0cf1606..8087c0ae3bc0a0a95512b4b0ff5257522a104ca0 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -29,16 +29,17 @@ #define V4L2_DEVICE_NOTIFY_EVENT _IOW('v', 2, struct v4l2_event) -struct v4l2_device; +struct led_classdev; +struct media_device_context; +struct tuner_setup; struct v4l2_ctrl_handler; +struct v4l2_device; struct v4l2_event; struct v4l2_event_subscription; struct v4l2_fh; +struct v4l2_mbus_frame_desc; struct v4l2_subdev; struct v4l2_subdev_fh; -struct tuner_setup; -struct v4l2_mbus_frame_desc; -struct led_classdev; /** * struct v4l2_decode_vbi_line - used to decode_vbi_line @@ -1968,64 +1969,75 @@ static inline void v4l2_subdev_unlock_states(struct v4l2_subdev_state *state1, mutex_unlock(state2->lock); } +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_sd(struct v4l2_subdev *sd); +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_sd(struct v4l2_subdev *sd); +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_sd(struct v4l2_subdev *sd); + +struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_active_state_from_ctx(struct v4l2_subdev_context *ctx); +struct v4l2_subdev_state * +v4l2_subdev_get_locked_active_state_from_ctx(struct v4l2_subdev_context *ctx); +struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_active_state_from_ctx(struct v4l2_subdev_context *ctx); + /** - * v4l2_subdev_get_unlocked_active_state() - Checks that the active subdev state - * is unlocked and returns it - * @sd: The subdevice + * v4l2_subdev_get_unlocked_active_state() - Checks that the subdev state is + * unlocked and returns it + * @sdctx: The subdevice, or the subdevice context * - * Returns the active state for the subdevice, or NULL if the subdev does not - * support active state. If the state is not NULL, calls - * lockdep_assert_not_held() to issue a warning if the state is locked. + * Returns the subdevice state, or NULL if it is not valid. If the state is + * not NULL, calls lockdep_assert_not_held() to issue a warning if the state + * is locked. * - * This function is to be used e.g. when getting the active state for the sole - * purpose of passing it forward, without accessing the state fields. + * This function is to be used e.g. when getting the state for the sole purpose + * of passing it forward, without accessing the state fields. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_get_unlocked_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - lockdep_assert_not_held(sd->active_state->lock); - return sd->active_state; -} +#define v4l2_subdev_get_unlocked_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_get_unlocked_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_unlocked_active_state_from_ctx) \ + (sdctx) /** - * v4l2_subdev_get_locked_active_state() - Checks that the active subdev state - * is locked and returns it - * - * @sd: The subdevice + * v4l2_subdev_get_locked_active_state() - Checks that the subdev state is + * locked and returns it + * @sdctx: The subdevice, or the subdevice context * - * Returns the active state for the subdevice, or NULL if the subdev does not - * support active state. If the state is not NULL, calls lockdep_assert_held() - * to issue a warning if the state is not locked. + * Returns the subdevice state, or NULL is not valid. If the state is not NULL, + * calls lockdep_assert_held() to issue a warning if the state is not locked. * - * This function is to be used when the caller knows that the active state is + * This function is to be used when the caller knows that the context state is * already locked. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_get_locked_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - lockdep_assert_held(sd->active_state->lock); - return sd->active_state; -} +#define v4l2_subdev_get_locked_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_get_locked_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_locked_active_state_from_ctx) \ + (sdctx) /** - * v4l2_subdev_lock_and_get_active_state() - Locks and returns the active subdev - * state for the subdevice - * @sd: The subdevice + * v4l2_subdev_lock_and_get_active_state_from_ctx() - Locks and returns the + * subdevice state + * @sdctx: The subdevice, or the subdevice context * - * Returns the locked active state for the subdevice, or NULL if the subdev - * does not support active state. + * Returns the locked subdevice state, or NULL if it is not valid. * * The state must be unlocked with v4l2_subdev_unlock_state() after use. */ -static inline struct v4l2_subdev_state * -v4l2_subdev_lock_and_get_active_state(struct v4l2_subdev *sd) -{ - if (sd->active_state) - v4l2_subdev_lock_state(sd->active_state); - return sd->active_state; -} +#define v4l2_subdev_lock_and_get_active_state(sdctx) \ + _Generic((sdctx), \ + struct v4l2_subdev *: \ + v4l2_subdev_lock_and_get_active_state_from_sd, \ + struct v4l2_subdev_context *: \ + v4l2_subdev_get_locked_active_state_from_ctx) \ + (sdctx) /** * v4l2_subdev_init - initializes the sub-device struct From patchwork Thu Jul 17 10:45:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116053 Received: from am.mirrors.kernel.org ([147.75.80.249]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMC4-0005J4-0U for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:50:39 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id CF3581894F42 for ; Thu, 17 Jul 2025 10:50:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 096AA2C325C; Thu, 17 Jul 2025 10:46:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OOff5Etz" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DDC32C08D6; Thu, 17 Jul 2025 10:46:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749172; cv=none; b=IaOMFS14XmFtDndeFJ58m9aIO/w99OBvxn1eTLLzk1OCcmnjqXfasOYllBC6SJXKWWzA6TszjrEm7ZsQafZLjGyM995/1LEiWYHSWPiUFIIDUcktDr9pWKcUy6piPOxF+19SEfT5XY2yFhFkwnOKKmKEmWGCSXUDpUY6CN5aNYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749172; c=relaxed/simple; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=d+EFg7YJpOzuD3KuVIhdL5JGzJknGjo7p4cpj3nSNhFImh9ShW8/2wJeniZPd8nOETWiDzzZpGraj04YL63QU/l2p1lEqHbQsQMr9hxJuU/b54NCCUlRhC/KDOfNd5Eao6H2vVZVmoR3CVEJ6mcMhkwrYQXfdioBdy9CV5UmcE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=OOff5Etz; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 67BB31E74; Thu, 17 Jul 2025 12:45:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749134; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OOff5Etzj1BASviY7XteSLrji0sQ2MGJumgJuzBT90EpkcnHlYXDMiQwybMFvWJm0 Nj6o8RgXitaS2Zv2e2IMoWrNfqaVlZnvYqkbIvznkwhVTKWzvywG3NdwBxnq1sA+yx uRitJwr0g1r4mSwKWwHNvnv+W4MD2cHGZhTWRD+M= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:43 +0200 Subject: [PATCH 17/26] media: v4l2-subdev: Get state from context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-17-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1875; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Av+qDIxci6/rNYahjLOq/w3ykEkjt8C35QIlJVVOQlA=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQd90DYybLMFBMGXEIc5XbdYrJwrR2tprN2 gyuKk9WcI+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PBoCD/9SwqPLTbNwWfPpNqq9bJosyxb8HDVvloOV15iO71vhg1hePnB46b86CzelfoY+NP9Uq0v ANEIbTXjPUQMVqQ4fpg6pNRTrkOf+hcLZXlKURymNy7O0CcYFtB//rdeIhyQFOGaDSElURD0V2t ar3ixtgA1UsMDN+cwcUNqDkXBWP3jyPPLoKG5V0i3G4Mnyg6JlzqJ3p9oY4yyvyYc/fVHuNAJTe Z61yhJTEeMLTT/plbkrn3RURNh8CrRpwrkR1CG5h0/LSVG/rKeHrWl0MmHy48Q3yRP+AvOuO4f9 sttWFV9NZdS+gXU26MAoXO1LI5/UEZVb+q/waqbQWRgqPfNt8SaBfYmGRtEu69tUxYvNrVdTYtw z9lxN0EsVWMf5BqAV3HpScpYnhuIl2fYv3ApP/4ikWaOiKJdoAYu4gZF2rPFvrhxOK/qyw8jXT9 snmr1jrT4W5xcgaUHZZa85Om18QKEnFEEbxL753ZJuolLI9KkHxhWeSTudyrKlnoy5QIGCZvYQx UPFyx8k2PI+ELqyzLc2jAXl9STxb1MsLRMEZxxMSct9YYL9qlJfa+4R1ft6dEJxd5leFZNwptIq AvExURcpv2bxcRFCAOu6CW+nFgWuKV/05XPH1PgksNHGO3hDSznSW7aRfvqYZEli/sAQeSrvkjI QH6qPtpuaiyZG0w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no The V4L2 subdev ioctl handler retrieves the correct subdev state inspecting the 'which' field of the ioctl argument. So far the subdev state either come from the file handle in case of V4L2_SUBDEV_FORMAT_TRY or from the active state stored in the subdev directly in case of V4L2_SUBDEV_FORMAT_ACTIVE. With the introduction of multi-contexts support, there will be a subdev state associated to each bound context. If we have a valid context, use the state from there in case of V4L2_SUBDEV_FORMAT_ACTIVE, and default to the subdev active state in case the context is not valid. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 7372f61127c871cec44a3d1900e2b8bef34632b9..66c539d880127844893620d325a2b05ac4aa9e96 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -703,9 +703,20 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh, break; } - return which == V4L2_SUBDEV_FORMAT_TRY ? - subdev_fh->state : - v4l2_subdev_get_unlocked_active_state(sd); + /* + * If which is FORMAT_TRY return the state stored in the file handle. + * If a context has been allocated because the subdev has been bound + * then return the state stored in the context. Otherwise default to the + * subdevice active state. + */ + + if (which == V4L2_SUBDEV_FORMAT_TRY) + return subdev_fh->state; + + if (subdev_fh->context) + return v4l2_subdev_get_unlocked_active_state(subdev_fh->context); + + return v4l2_subdev_get_unlocked_active_state(sd); } static int subdev_do_bind_context(struct v4l2_subdev *sd, From patchwork Thu Jul 17 10:45:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116056 Received: from ny.mirrors.kernel.org ([147.75.199.223]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMCW-0005KI-1S for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:50:58 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 46EE356236E for ; Thu, 17 Jul 2025 10:50:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 91C762D63E0; Thu, 17 Jul 2025 10:46:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eIP2rUtv" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B7192D23BF; Thu, 17 Jul 2025 10:46:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749176; cv=none; b=lgsmeNhhRLWgxFvL3UTrlPQhI+827QPKwNCHn5nbG2COUdedJBEY2LF4fKPyZGLhNqDmcysfTYkzA39fhzXTr9vjeAoAUpFs7SbtaI+3800n9nbxUn22DOAEszETspT9kiICqGsu3A5XY3RiacUBbdjxCaL86wUxAXQhVm5sYlc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749176; c=relaxed/simple; bh=9EOdO/COHzQLny/RumAreucWfszOgExSRSTj6l4Lnxw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=O166iu89XG6RDBqiGYL0HNQOk/FA/k9mGsqHsFPnOm5PnazrVCwgDEHt4EP0pkMslRpRfgml3i5YyFZdQ04+fOIylTXd//yuc25LiaI2BG6YhkZlXxpYQMNsjcxQJdxX4ffhHnxp/ij0VvvvxLyGTuf2dGzXphpj3p1saOnzOWg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=eIP2rUtv; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B4F98198D; Thu, 17 Jul 2025 12:45:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749136; bh=9EOdO/COHzQLny/RumAreucWfszOgExSRSTj6l4Lnxw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=eIP2rUtvj4XVChnhxO73AoC9nZDd8YVYghDb9I11bQi5i95qlcYhJwn09IoxSZoH7 n9wN3A6FsY+9TbBSo02dCDhVMeOyItOOEtb6BG5z6DNPGv8WlW/jydJS8C6ab236O3 I167KPbt7KRL5ZKZiOTwpyBk03NGceiv5EyCs/YY= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:44 +0200 Subject: [PATCH 18/26] media: media-entity: Support context in pipeline_start Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-18-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=17561; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=9EOdO/COHzQLny/RumAreucWfszOgExSRSTj6l4Lnxw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRQb5XJQ8WaFkazJJHpibRlzjXNaQWnm4wIr JvoB8u/knqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUAAKCRByNAaPFqFW PJCgD/9CjPOsI8BZsR0hoYcEI6pKmdRZt4S+A16gifirJw30MELbX2sLr8usRI9VE/OeBqyUOz9 VqfqrPS13hkkHVkyrdo5T6WYuaRcmmTgTdiy3J6N61g0Wt8E0OY84fyLKbzRYxsce01paixEXvU A+sI/41MS8xW2xLoEAsX7W+ihOLU5ZRw/H3TuU+Mu4F/awBn3Vanb+BdpIp9Sb6XxRH2K2WNUAO W+Z8f8db9DLeURXWebXtRhpuoVVjJIHzv1101zhjyHZd7GYL+KgYqszQcoKZ+cX5N4ZBotADm8z wRYRHrc/tLZ0vbHdm3Sk8Ii2P8DyC3ADvH2kVjujk4sU1ntxBVKTQwRVXKR7Zl6WxH40rugG3k0 lqWnWJQJtevHr7q77k70ZXNDUpLBCughWX8fKuJGH45u0ZqNBXXFJy0KgZShS2Y3i6KXU7c6Kkr 4TpSZCqsNVCTHXd0WFbNnsH8TbT0jXxS45FyvgT9fxowOSCWa1v+x6fuNDRsMFZbVC5VDZWkMB/ 9KUlbzhDmC5wPh2s9B6k8vskNCHyb9hb0nbYQYyw+B2bWSh35cl0x/H6TypRZZr0JrEPu+HVCk9 Hw9kPZ//6qpcv+y8xnZ1UU9vc5+yuFHzkCY0JpwcZ4TsGPOsPLxJXqjjICj8QQNxMx9FciPuh1Q tFlYeUw56tNWxIA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Add helpers to propagate an execution context when starting the media pipeline. Starting a pipeline with an execution context implies that when creating the streaming pipeline all the entities part of the pipeline shall have a valid context associated with the media context in use. When creating the media pipeline increase the device context use count to guarantee that during the streaming phase drivers will always operate with valid contexts. Add 'overrides' for all the video_device_pipeline_start() and media_pipeline_start() variants to support contexts. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 150 ++++++++++++++++++++++++++++++++----- drivers/media/v4l2-core/v4l2-dev.c | 44 +++++++++++ include/media/media-entity.h | 56 +++++++++++++- include/media/v4l2-dev.h | 57 ++++++++++++++ 4 files changed, 287 insertions(+), 20 deletions(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 7bc276c725f974539ea06e3882d004b81be1de68..f421d6d74630bb96400d39d805c5db5d3d1ff913 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -768,8 +768,50 @@ static int media_pipeline_populate(struct media_pipeline *pipe, return ret; } -__must_check int __media_pipeline_start(struct media_pad *origin, - struct media_pipeline *pipe) +static int +media_pipeline_validate_context(struct media_device_context *mdev_context, + struct media_entity *entity, + struct media_pipeline_pad *ppad) +{ + struct media_entity_context *context; + + if (!mdev_context) + return 0; + + /* + * It's not mandatory for all entities in the pipeline to support + * contexts. + */ + if (!entity->ops || !entity->ops->alloc_context || + !entity->ops->destroy_context) + return 0; + + /* + * But if they do they should be bound to the same media device context + * as all other entities. + * + * media_device_get_entity_context increases the ref-counting of the + * context. Store a reference in the ppad for later decreasing the + * reference count when the pipeline is stopped. + * + * Fail validation if no context is associated with this media context + * and be loud about that as userspace should be informed it has to + * bind all entities of the pipeline in the same context. + */ + context = media_device_get_entity_context(mdev_context, entity); + if (WARN_ON(IS_ERR(context))) + return -EPIPE; + + media_entity_context_get(context); + ppad->context = context; + + return 0; +} + +__must_check int +__media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *mdev_context) { struct media_device *mdev = origin->graph_obj.mdev; struct media_pipeline_pad *err_ppad; @@ -829,7 +871,15 @@ __must_check int __media_pipeline_start(struct media_pad *origin, } /* - * 2. Validate all active links whose sink is the current pad. + * 2. If we have a media context, ensure the entity has a device + * context associated with it. + */ + ret = media_pipeline_validate_context(mdev_context, entity, ppad); + if (ret) + goto error; + + /* + * 3. Validate all active links whose sink is the current pad. * Validation of the source pads is performed in the context of * the connected sink pad to avoid duplicating checks. */ @@ -875,7 +925,7 @@ __must_check int __media_pipeline_start(struct media_pad *origin, } /* - * 3. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, + * 4. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, * ensure that it has either no link or an enabled link. */ if ((pad->flags & MEDIA_PAD_FL_MUST_CONNECT) && @@ -905,6 +955,9 @@ __must_check int __media_pipeline_start(struct media_pad *origin, if (err_ppad == ppad) break; + if (err_ppad->context) + media_entity_context_put(err_ppad->context); + err_ppad->pad->pipe = NULL; } @@ -912,19 +965,35 @@ __must_check int __media_pipeline_start(struct media_pad *origin, return ret; } +EXPORT_SYMBOL_GPL(__media_pipeline_start_context); + +__must_check int __media_pipeline_start(struct media_pad *origin, + struct media_pipeline *pipe) +{ + return __media_pipeline_start_context(origin, pipe, NULL); +} EXPORT_SYMBOL_GPL(__media_pipeline_start); -__must_check int media_pipeline_start(struct media_pad *origin, - struct media_pipeline *pipe) +__must_check int +media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context) { struct media_device *mdev = origin->graph_obj.mdev; int ret; mutex_lock(&mdev->graph_mutex); - ret = __media_pipeline_start(origin, pipe); + ret = __media_pipeline_start_context(origin, pipe, context); mutex_unlock(&mdev->graph_mutex); return ret; } +EXPORT_SYMBOL_GPL(media_pipeline_start_context); + +__must_check int media_pipeline_start(struct media_pad *origin, + struct media_pipeline *pipe) +{ + return media_pipeline_start_context(origin, pipe, NULL); +} EXPORT_SYMBOL_GPL(media_pipeline_start); void __media_pipeline_stop(struct media_pad *pad) @@ -942,8 +1011,11 @@ void __media_pipeline_stop(struct media_pad *pad) if (--pipe->start_count) return; - list_for_each_entry(ppad, &pipe->pads, list) + list_for_each_entry(ppad, &pipe->pads, list) { + if (ppad->context) + media_entity_context_put(ppad->context); ppad->pad->pipe = NULL; + } media_pipeline_cleanup(pipe); @@ -962,14 +1034,13 @@ void media_pipeline_stop(struct media_pad *pad) } EXPORT_SYMBOL_GPL(media_pipeline_stop); -__must_check int media_pipeline_alloc_start(struct media_pad *pad) +static struct media_pipeline * +media_pipeline_alloc(struct media_pad *pad) { struct media_device *mdev = pad->graph_obj.mdev; - struct media_pipeline *new_pipe = NULL; struct media_pipeline *pipe; - int ret; - mutex_lock(&mdev->graph_mutex); + lockdep_assert_held(&mdev->graph_mutex); /* * Is the pad already part of a pipeline? If not, we need to allocate @@ -977,19 +1048,33 @@ __must_check int media_pipeline_alloc_start(struct media_pad *pad) */ pipe = media_pad_pipeline(pad); if (!pipe) { - new_pipe = kzalloc(sizeof(*new_pipe), GFP_KERNEL); - if (!new_pipe) { - ret = -ENOMEM; - goto out; - } + pipe = kzalloc(sizeof(*pipe), GFP_KERNEL); + if (!pipe) + return ERR_PTR(-ENOMEM); - pipe = new_pipe; pipe->allocated = true; } + return pipe; +} + +__must_check int media_pipeline_alloc_start(struct media_pad *pad) +{ + struct media_device *mdev = pad->graph_obj.mdev; + struct media_pipeline *pipe; + int ret; + + mutex_lock(&mdev->graph_mutex); + + pipe = media_pipeline_alloc(pad); + if (IS_ERR(pipe)) { + ret = PTR_ERR(pipe); + goto out; + } + ret = __media_pipeline_start(pad, pipe); if (ret) - kfree(new_pipe); + kfree(pipe); out: mutex_unlock(&mdev->graph_mutex); @@ -998,6 +1083,33 @@ __must_check int media_pipeline_alloc_start(struct media_pad *pad) } EXPORT_SYMBOL_GPL(media_pipeline_alloc_start); +__must_check +int media_pipeline_alloc_start_context(struct media_pad *pad, + struct media_device_context *mdev_context) +{ + struct media_device *mdev = pad->graph_obj.mdev; + struct media_pipeline *pipe; + int ret; + + mutex_lock(&mdev->graph_mutex); + + pipe = media_pipeline_alloc(pad); + if (IS_ERR(pipe)) { + ret = PTR_ERR(pipe); + goto out; + } + + ret = __media_pipeline_start_context(pad, pipe, mdev_context); + if (ret) + kfree(pipe); + +out: + mutex_unlock(&mdev->graph_mutex); + + return ret; +} +EXPORT_SYMBOL_GPL(media_pipeline_alloc_start_context); + struct media_pad * __media_pipeline_pad_iter_next(struct media_pipeline *pipe, struct media_pipeline_pad_iter *iter, diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 96696959314abfb1864ea5d96742e579b5a41f6f..d2136f5a84b165cf1f8f7b37cef981d74c3e5ac2 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1221,6 +1221,36 @@ __must_check int __video_device_pipeline_start(struct video_device *vdev, } EXPORT_SYMBOL_GPL(__video_device_pipeline_start); +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev = context->vdev; + struct media_entity *entity = &vdev->entity; + + if (entity->num_pads != 1) + return -ENODEV; + + return media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(video_device_context_pipeline_start); + +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev = context->vdev; + struct media_entity *entity = &vdev->entity; + + if (entity->num_pads != 1) + return -ENODEV; + + return __media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(__video_device_context_pipeline_start); + void video_device_pipeline_stop(struct video_device *vdev) { struct media_entity *entity = &vdev->entity; @@ -1254,6 +1284,20 @@ __must_check int video_device_pipeline_alloc_start(struct video_device *vdev) } EXPORT_SYMBOL_GPL(video_device_pipeline_alloc_start); +__must_check int +video_device_context_pipeline_alloc_start(struct video_device_context *context) +{ + struct video_device *vdev = context->vdev; + struct media_entity *entity = &vdev->entity; + + if (entity->num_pads != 1) + return -ENODEV; + + return media_pipeline_alloc_start_context(&entity->pads[0], + context->base.mdev_context); +} +EXPORT_SYMBOL_GPL(video_device_context_pipeline_alloc_start); + struct media_pipeline *video_device_pipeline(struct video_device *vdev) { struct media_entity *entity = &vdev->entity; diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 32298fe8a18c6ee3c1dbcff9ef869548904417a7..b60c311ab390beb6931fe8f2bbe8939e11cda452 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -119,16 +119,20 @@ struct media_pipeline { * @list: Entry in the media_pad pads list * @pipe: The media_pipeline that the pad is part of * @pad: The media pad + * @context: Reference to a video device or subdevice context * * This structure associate a pad with a media pipeline. Instances of * media_pipeline_pad are created by media_pipeline_start() when it builds the * pipeline, and stored in the &media_pad.pads list. media_pipeline_stop() - * removes the entries from the list and deletes them. + * removes the entries from the list and deletes them. The context field is + * populated only if a valid context has been associated with the pad. */ +struct media_entity_context; struct media_pipeline_pad { struct list_head list; struct media_pipeline *pipe; struct media_pad *pad; + struct media_entity_context *context; }; /** @@ -1212,6 +1216,39 @@ __must_check int media_pipeline_start(struct media_pad *origin, __must_check int __media_pipeline_start(struct media_pad *origin, struct media_pipeline *pipe); +/** + * media_pipeline_start_context - Mark a pipeline as streaming + * @origin: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * Mark all pads connected to a given pad through enabled links, either + * directly or indirectly, as streaming. The given pipeline object is assigned + * to every pad in the pipeline and stored in the media_pad pipe field. + * + * Calls to this function can be nested, in which case the same number of + * media_pipeline_stop() calls will be required to stop streaming. The + * pipeline pointer must be identical for all nested calls to + * media_pipeline_start(). + */ +__must_check int +media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context); + +/** + * __media_pipeline_start_context - Mark a pipeline as streaming + * @origin: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * ..note:: This is the non-locking version of media_pipeline_start_context() + */ +__must_check int +__media_pipeline_start_context(struct media_pad *origin, + struct media_pipeline *pipe, + struct media_device_context *context); + /** * media_pipeline_stop - Mark a pipeline as not streaming * @pad: Starting pad @@ -1318,6 +1355,23 @@ __media_pipeline_entity_iter_next(struct media_pipeline *pipe, */ __must_check int media_pipeline_alloc_start(struct media_pad *pad); +/** + * media_pipeline_alloc_start_context - Mark a pipeline as streaming + * @pad: Starting pad + * @context: The media device context the pipeline belongs to + * + * media_pipeline_alloc_start_context() is similar to + * media_pipeline_start_context() but instead of working on a given pipeline the + * function will use an existing pipeline if the pad is already part of a + * pipeline, or allocate a new pipeline. + * + * Calls to media_pipeline_alloc_start_context() must be matched with + * media_pipeline_stop(). + */ +__must_check int +media_pipeline_alloc_start_context(struct media_pad *pad, + struct media_device_context *context); + /** * media_devnode_create() - creates and initializes a device node interface * diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index bab4b13b109362bec84d8d16440b6ea895206b60..93095df692e3628d4be003d25843fa744d6b20a4 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -602,6 +602,48 @@ __must_check int video_device_pipeline_start(struct video_device *vdev, __must_check int __video_device_pipeline_start(struct video_device *vdev, struct media_pipeline *pipe); +/** + * video_device_context_pipeline_start - Mark a pipeline as streaming starting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the multi-context version of video_device_pipeline_start() + * Documentation of this function only describes specific aspects of + * this version. Refer to the video_device_pipeline_start() + * documentation for a complete reference. + * + * Validate that all entities connected to a video device through enabled links + * by ensuring that a context associated with the same media device context + * exists for them. Increase the reference counting of each of the contexts part + * of the pipeline to guarantee their lifetime is maintained as long as the + * pipeline is streaming. + * + * Context validation and refcounting of all entities that are part of a + * streaming pipeline ensures that device drivers can safely access device + * contexts in a media device context during streaming. References to contexts + * retried by a call to media_device_get_entity_context(), are guaranteed to be + * valid as long as the pipeline is streaming. Likewise, the media device + * context that contains the device contexts is guaranteed to be valid as long + * as the pipeline is streaming. + */ +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + +/** + * __video_device_context_pipeline_start - Mark a pipeline as streaming starting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the non-locking version of + * __video_device_context_pipeline_start() + */ +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + /** * video_device_pipeline_stop - Mark a pipeline as not streaming * @vdev: Starting video device @@ -646,6 +688,21 @@ void __video_device_pipeline_stop(struct video_device *vdev); */ __must_check int video_device_pipeline_alloc_start(struct video_device *vdev); +/** + * video_device_context_pipeline_alloc_start - Mark a pipeline as streaming + * @context: The video device context that starts the streaming + * + * video_device_context_pipeline_alloc_start() is similar to + * video_device_context_pipeline_start() but instead of working on a given + * pipeline the function will use an existing pipeline if the video device is + * already part of a pipeline, or allocate a new pipeline. + * + * Calls to video_device_context_pipeline_alloc_start() must be matched with + * video_device_pipeline_stop(). + */ +__must_check int +video_device_context_pipeline_alloc_start(struct video_device_context *context); + /** * video_device_pipeline - Get the media pipeline a video device is part of * @vdev: The video device From patchwork Thu Jul 17 10:45:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116089 Received: from sy.mirrors.kernel.org ([147.75.48.161]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMKs-0005aB-0C for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:59:34 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 174BB7BB9A5 for ; Thu, 17 Jul 2025 10:49:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 68D9E2D5424; Thu, 17 Jul 2025 10:46:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DABlrK2a" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAD052D3219; Thu, 17 Jul 2025 10:46:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749176; cv=none; b=KnqnWLX8/eTEE9os83DfSCtVuQ5uSylZ1elvAgIk/ym0MMxs/FE9Sn2+BenpPrmqNoZ+ABB08RxFT+nCZZuMK5PYVIDWadfVz0LXsK93iTa/Mcp1Cnc80ewD+nhjmW502YdlD3vfepZCMu70gbuZqQRces/wLC8XbHyl5ousTwQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749176; c=relaxed/simple; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=A3Nbn4tn2jV91V4a1bnGFwHQcjNJ687WIJCLC46J1jqHkqEfHVdNtrTNffHC8mrgYS14Z/x+YWdJspAlmmLiVZdYvCwvmguQiHj+nOP0RoJwBsAiZwIO6kY5vhK8/d68uK78kp3HXgwpf9bHkmuE5EqFHLvmP/mo8NDMrcsC1Wo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=DABlrK2a; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3DFD71AE2; Thu, 17 Jul 2025 12:45:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749138; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DABlrK2a88A3r8J+z6KrObawZ6hDRn+XgUPs6DOolXJBUpaQReEFGJrSKwsBmUYMb pA0aR02ddyUOK3QAbnifhFNJ+NdGsrWGWfjbcoKFAUQ0/tO6xEkQ9SfrT3XuyMv4h5 ZWCWArDZcBlgLr48GxuYfzDBnGbmmERUGvYyr2Qk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:45 +0200 Subject: [PATCH 19/26] media: mc-entity: Add link_validate_context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-19-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3666; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Vmfy9P6HgOvkJ1o2D9ktCyX+PB0ofnZcx1ytCaFVfCs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRRQX0z9nktE5CicUsI35GLyrxFYdwvWu01o DbQVnGZFZmJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PLSyEACEMz3OzJzaHE7EGoRK0sBcANROBQuYYp7GHepcJ/nUiptjHw6rgsK41gGAoCbT/xnTP1s BMornojg9rzbreAVIJzUwEvOoEXg6O9lY+chAy1gcx32ZqW/l1BUtBtxRzDOb/ghXJ7WHaj0Cc/ Wg1Ub/whqUwrpm2kvuRsUoT6WgQAxvs7KqaUNXyfThxEeiu8bVgjUUMTSTkWCljGuzuASQFA4q2 tX2D3kMo7pMRc+gsTJGFeJqWSPFRwAUKl2f1zCVH9WubuIeu93G/7OVZ2CV3lXPHn5HdiEia0KY X4EOWGEoZt1DO2AuTY67kZj6SEOETWqImcTKGi13BALMPCjsMzlPpinSYQ7lZ9UYAjvrmFgmA+d igXtW2xOWHbiMNXJNCkCooqG3VPV4SIdtYqchrn/P4b71vkOgye0YG7Jaw8/IaHip/EloiAblA6 ywriQzyhd6EoaeJLWEWhZnvQjCA4v4bBgeZcmSYb5IeowEiZ8HwqyQKMROifa+N4qJOf4FUmCg3 VOnZ8M/30upNjPEK7nU0Amvcbj/Snw2ZSlPFMGMb/OGsVGN5068Tb40oa+bn4212ThCCtHSNSF5 ueysf/7Gz9j80UKcs5L+fSMFJKTXCRx3zugNHzqN6gEisRT4X8sn3Tz+ktaIt7tJsoy5xwF98tU 5MtaPun5X8+POGQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Add an 'override' of the link_validate() media entity operation that accepts a media_device_context as second argument to allow entities to validate the link state in the provided media_device context. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 23 +++++++++++++++++++---- include/media/media-entity.h | 6 ++++++ 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index f421d6d74630bb96400d39d805c5db5d3d1ff913..675ceaede0d10a2420b8ea6a89e5963dcfde5ffe 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -884,6 +884,8 @@ __media_pipeline_start_context(struct media_pad *origin, * the connected sink pad to avoid duplicating checks. */ for_each_media_entity_data_link(entity, link) { + const struct media_entity_operations *ops; + /* Skip links unrelated to the current pad. */ if (link->sink != pad && link->source != pad) continue; @@ -902,13 +904,23 @@ __media_pipeline_start_context(struct media_pad *origin, if (link->sink != pad) continue; - if (!entity->ops || !entity->ops->link_validate) + ops = entity->ops; + if (!ops || (!ops->link_validate && + !ops->link_validate_context)) continue; - ret = entity->ops->link_validate(link); + if (mdev_context && ops->link_validate_context) + ret = ops->link_validate_context(link, + mdev_context); + else + ret = entity->ops->link_validate(link); + if (ret) { dev_dbg(mdev->dev, - "Link '%s':%u -> '%s':%u failed validation: %d\n", + "%sink '%s':%u -> '%s':%u failed validation: %d\n", + (mdev_context && + ops->link_validate_context) ? + "Context l" : "L", link->source->entity->name, link->source->index, link->sink->entity->name, @@ -917,7 +929,10 @@ __media_pipeline_start_context(struct media_pad *origin, } dev_dbg(mdev->dev, - "Link '%s':%u -> '%s':%u is valid\n", + "%sink '%s':%u -> '%s':%u is valid\n", + (mdev_context && + ops->link_validate_context) ? + "Context l" : "L", link->source->entity->name, link->source->index, link->sink->entity->name, diff --git a/include/media/media-entity.h b/include/media/media-entity.h index b60c311ab390beb6931fe8f2bbe8939e11cda452..b053a0baee4031a464edf506d3d131bacb810f81 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -295,6 +295,10 @@ struct media_entity_context { * @link_validate: Return whether a link is valid from the entity point of * view. The media_pipeline_start() function * validates all links by calling this operation. Optional. + * @link_validate_context: Return whether a link is valid from the entity + * context point of view. The + * media_pipeline_start_context() function validates all + * links calling this operation. Optional. * @has_pad_interdep: Return whether two pads of the entity are * interdependent. If two pads are interdependent they are * part of the same pipeline and enabling one of the pads @@ -327,6 +331,8 @@ struct media_entity_operations { const struct media_pad *local, const struct media_pad *remote, u32 flags); int (*link_validate)(struct media_link *link); + int (*link_validate_context)(struct media_link *link, + struct media_device_context *mdev_context); bool (*has_pad_interdep)(struct media_entity *entity, unsigned int pad0, unsigned int pad1); int (*alloc_context)(struct media_entity *entity, From patchwork Thu Jul 17 10:45:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116065 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMD1-0005MA-1e for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:51:28 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 02BB73BB5B0 for ; Thu, 17 Jul 2025 10:50:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0570C2D876F; Thu, 17 Jul 2025 10:46:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jLiKp8v9" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C52E72D7808; Thu, 17 Jul 2025 10:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749179; cv=none; b=rG7nodiUGodLfm2vXxlwcAapg7YMLJJ7JPk6X64yvpmGoZU7JaUGQ84O3Aoa+TjBzQKjZMTnIuSmXREgRq3UjzL3hXq6Z9OwAbfnBVmaUGaFLONNXwtWhXEtM28iwT4qR6bXK6drhQ5QvgDSkd13ZzPk3vsieHVDosGJUSx6Bzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749179; c=relaxed/simple; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fgiwayqbi7Bq5IfWepd6S39lz55O6zKazyvenySzBwewk8WJ+pxVMmB7oDGGOyT0hwX8I10kL17Xr4PNDKXIQ49gCQjL1P91QnIxWJZdyuEJYSJ67vqzBz2d/hsGBhuY99Eh2utFMqvU6KnBWjBcuLEWOfpKPdhIshMc0WxuoXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=jLiKp8v9; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B36E71FA4; Thu, 17 Jul 2025 12:45:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749139; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=jLiKp8v9BdwuYLqpQmGwBjypsQ6178ilQW66W1ZjMl4zzhBgR2fU3z2EuToG3tfdX N8wYJfyrLp3fDa+iwD0VklWRXSRYMYJq/Bh9EuSjIBxQ4pNBrES4jBjZAlza8s3aBt pfY2gPVNQdamvZ7T+U38XmMkwwTe+xu28RbK7vfc= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:46 +0200 Subject: [PATCH 20/26] media: v4l2-subdev: Validate media links with context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-20-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=13600; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Rakpl27ZFfuZPNzLUJwkywWxLHQibWoreQq3wqBUYHs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRRk64alscu4p46p+/yv/iCju3xiabjACpse sUbQjjulguJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PMOGEACGk81s7V2vjvyY/MAbWwEzDLfpfMl7MGi737Hx9b0myW4FLZ6Ni2LVqkTnFIYAg/MdzaB 2A/4SnzZKv+RWombBsJaIIfu2CmLj4utoEL71w2WU9GCuolU6eFMpfkhzIr/NxHQgO+aHPTdibn V93geWikkND3NXn9AwMSocz780POSPYBd1KOLxldcO2i2kP+yemPvXwrDJDSSgyjJGFUA252YVj urA5rubn+MbFlgskjwzdTKMF9RcHaxhPNkWrnnfSh7MMQzIsJ7cHF2syxKaQi0kDIqe3eNnOnVS s7Nwjjo5OesURwH67DnXHsHDLfk397l/fzbVqRJfVl2qyoGti79og4eiiW3Dx0SyJxirli/FaM9 hR+5krXB34mzvJ9/rmcQK7LhRja/kjHqh/t0A5tExVQowXiihd47gqKBG+4sUFlMVmXhBfCK/2n rQBvGUcJ9Iw9ekj4HMOnxrQJPtvO8LEoyfJZFN+lIs5FSNMIdlsgqMF3OQdW3GHlQF9nOHVOqli ILXCHpVjFtlgbW+7wlh4wkeaE8zzVKzzoD6E0a35mRVZpaCI1EIX9grDRmPHojIsD1TmUzoLN/a DiMv7Sf5raxZvW6aUewZT6AsUQIaIKRLcDfb0ChXwv5WZBukGw3wNcuzGN0pw0ZQTzVqO/ECELb C4LyLIJcEfBSGMg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no The v4l2-subdev.c file provides an helper for subdevs to implement the media entity .link_validate() operation which can be used by subdevice drivers. Provide an 'overload' of the v4l2_subdev_link_validate() function that supports contexts to be used by context-aware subdev drivers to implement .link_validate_context(). When v4l2_subdev_link_validate() is used to validate a subdev-to-subdev link, propagate the media_device_context to all the call chain and introduce (and use) helpers to get the subdevice state from either the context or the subdev in case no context is available. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-subdev.c | 148 +++++++++++++++++++++++++++------- include/media/v4l2-subdev.h | 34 +++++++- 2 files changed, 151 insertions(+), 31 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 66c539d880127844893620d325a2b05ac4aa9e96..59f42a3a9755a77ea74442605dbbc2af1b67a0ea 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1460,6 +1460,71 @@ int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity, } EXPORT_SYMBOL_GPL(v4l2_subdev_get_fwnode_pad_1_to_1); +/* + * Retrieve the subdevice state from the media device context or, + * if there is no context, use the active state from the subdevice. + * + * These three functions wraps the usual subdev state helpers: + * + * - get_unlocked + * - get_locked + * - lock_and_get + */ + +static struct v4l2_subdev_state * +v4l2_subdev_get_unlocked_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx = NULL; + + if (mdev_ctx) { + ctx = v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_get_unlocked_active_state(ctx); + + return v4l2_subdev_get_unlocked_active_state(sd); +} + +static struct v4l2_subdev_state * +v4l2_subdev_get_locked_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx = NULL; + + if (mdev_ctx) { + ctx = v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_get_locked_active_state(ctx); + + return v4l2_subdev_get_locked_active_state(sd); +} + +static struct v4l2_subdev_state * +v4l2_subdev_lock_and_get_state_from_mdev_ctx(struct v4l2_subdev *sd, + struct media_device_context *mdev_ctx) +{ + struct v4l2_subdev_context *ctx = NULL; + + if (mdev_ctx) { + ctx = v4l2_subdev_context_get(mdev_ctx, sd); + if (WARN_ON(!ctx)) + return NULL; + } + + if (ctx) + return v4l2_subdev_lock_and_get_active_state(ctx); + + return v4l2_subdev_lock_and_get_active_state(sd); +} + int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd, struct media_link *link, struct v4l2_subdev_format *source_fmt, @@ -1518,8 +1583,9 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd, EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default); static int -v4l2_subdev_link_validate_get_format(struct media_pad *pad, u32 stream, - struct v4l2_subdev_format *fmt, +v4l2_subdev_link_validate_get_format(struct media_pad *pad, + struct media_device_context *mdev_context, + u32 stream, struct v4l2_subdev_format *fmt, bool states_locked) { struct v4l2_subdev_state *state; @@ -1533,9 +1599,11 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad, u32 stream, fmt->stream = stream; if (states_locked) - state = v4l2_subdev_get_locked_active_state(sd); + state = v4l2_subdev_get_locked_state_from_mdev_ctx(sd, + mdev_context); else - state = v4l2_subdev_lock_and_get_active_state(sd); + state = v4l2_subdev_lock_and_get_state_from_mdev_ctx(sd, + mdev_context); ret = v4l2_subdev_call(sd, pad, get_fmt, state, fmt); @@ -1548,6 +1616,7 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad, u32 stream, #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) static void __v4l2_link_validate_get_streams(struct media_pad *pad, + struct media_device_context *mdev_context, u64 *streams_mask, bool states_locked) { @@ -1560,10 +1629,11 @@ static void __v4l2_link_validate_get_streams(struct media_pad *pad, *streams_mask = 0; if (states_locked) - state = v4l2_subdev_get_locked_active_state(subdev); + state = v4l2_subdev_get_locked_state_from_mdev_ctx(subdev, + mdev_context); else - state = v4l2_subdev_lock_and_get_active_state(subdev); - + state = v4l2_subdev_lock_and_get_state_from_mdev_ctx(subdev, + mdev_context); if (WARN_ON(!state)) return; @@ -1592,6 +1662,7 @@ static void __v4l2_link_validate_get_streams(struct media_pad *pad, #endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ static void v4l2_link_validate_get_streams(struct media_pad *pad, + struct media_device_context *mdev_context, u64 *streams_mask, bool states_locked) { @@ -1604,14 +1675,17 @@ static void v4l2_link_validate_get_streams(struct media_pad *pad, } #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) - __v4l2_link_validate_get_streams(pad, streams_mask, states_locked); + __v4l2_link_validate_get_streams(pad, mdev_context, streams_mask, + states_locked); #else /* This shouldn't happen */ *streams_mask = 0; #endif } -static int v4l2_subdev_link_validate_locked(struct media_link *link, bool states_locked) +static int v4l2_subdev_link_validate_locked(struct media_link *link, + struct media_device_context *mdev_context, + bool states_locked) { struct v4l2_subdev *sink_subdev = media_entity_to_v4l2_subdev(link->sink->entity); @@ -1626,8 +1700,10 @@ static int v4l2_subdev_link_validate_locked(struct media_link *link, bool states link->source->entity->name, link->source->index, link->sink->entity->name, link->sink->index); - v4l2_link_validate_get_streams(link->source, &source_streams_mask, states_locked); - v4l2_link_validate_get_streams(link->sink, &sink_streams_mask, states_locked); + v4l2_link_validate_get_streams(link->source, mdev_context, + &source_streams_mask, states_locked); + v4l2_link_validate_get_streams(link->sink, mdev_context, + &sink_streams_mask, states_locked); /* * It is ok to have more source streams than sink streams as extra @@ -1654,7 +1730,8 @@ static int v4l2_subdev_link_validate_locked(struct media_link *link, bool states link->source->entity->name, link->source->index, stream, link->sink->entity->name, link->sink->index, stream); - ret = v4l2_subdev_link_validate_get_format(link->source, stream, + ret = v4l2_subdev_link_validate_get_format(link->source, + mdev_context, stream, &source_fmt, states_locked); if (ret < 0) { dev_dbg(dev, @@ -1664,7 +1741,8 @@ static int v4l2_subdev_link_validate_locked(struct media_link *link, bool states continue; } - ret = v4l2_subdev_link_validate_get_format(link->sink, stream, + ret = v4l2_subdev_link_validate_get_format(link->sink, + mdev_context, stream, &sink_fmt, states_locked); if (ret < 0) { dev_dbg(dev, @@ -1693,7 +1771,8 @@ static int v4l2_subdev_link_validate_locked(struct media_link *link, bool states return 0; } -int v4l2_subdev_link_validate(struct media_link *link) +int __v4l2_subdev_link_validate(struct media_link *link, + struct media_device_context *mdev_context) { struct v4l2_subdev *source_sd, *sink_sd; struct v4l2_subdev_state *source_state, *sink_state; @@ -1716,28 +1795,35 @@ int v4l2_subdev_link_validate(struct media_link *link) if (is_media_entity_v4l2_video_device(link->source->entity)) { struct media_entity *source = link->source->entity; - if (!source->ops || !source->ops->link_validate) { + if (!source->ops || + (mdev_context && !source->ops->link_validate_context) || + (!mdev_context && !source->ops->link_validate)) { /* - * Many existing drivers do not implement the required - * .link_validate() operation for their video devices. - * Print a warning to get the drivers fixed, and return - * 0 to avoid breaking userspace. This should - * eventually be turned into a WARN_ON() when all - * drivers will have been fixed. + * Many existing drivers do not implement the correct + * .link_validate() or .link_validate_context() + * operations for their video devices. Print a warning + * to get the drivers fixed, and return 0 to avoid + * breaking userspace. This should eventually be turned + * into a WARN_ON() when all drivers will have been + * fixed. */ - pr_warn_once("video device '%s' does not implement .link_validate(), driver bug!\n", + pr_warn_once("video device '%s' does not implement the correct .link_validate operation: driver bug!\n", source->name); return 0; } /* * Avoid infinite loops in case a video device incorrectly uses - * this helper function as its .link_validate() handler. + * this helper function as its .link_validate[_context]() + * handler. */ - if (WARN_ON(source->ops->link_validate == v4l2_subdev_link_validate)) + if (WARN_ON(source->ops->link_validate == v4l2_subdev_link_validate || + source->ops->link_validate_context == v4l2_subdev_link_validate_context)) return -EINVAL; - return source->ops->link_validate(link); + return (mdev_context && source->ops->link_validate_context) ? + source->ops->link_validate_context(link, mdev_context) : + source->ops->link_validate(link); } /* @@ -1750,22 +1836,24 @@ int v4l2_subdev_link_validate(struct media_link *link) sink_sd = media_entity_to_v4l2_subdev(link->sink->entity); source_sd = media_entity_to_v4l2_subdev(link->source->entity); - sink_state = v4l2_subdev_get_unlocked_active_state(sink_sd); - source_state = v4l2_subdev_get_unlocked_active_state(source_sd); - + sink_state = v4l2_subdev_get_unlocked_state_from_mdev_ctx(sink_sd, + mdev_context); + source_state = v4l2_subdev_get_unlocked_state_from_mdev_ctx(source_sd, + mdev_context); states_locked = sink_state && source_state; if (states_locked) v4l2_subdev_lock_states(sink_state, source_state); - ret = v4l2_subdev_link_validate_locked(link, states_locked); + ret = v4l2_subdev_link_validate_locked(link, mdev_context, states_locked); if (states_locked) v4l2_subdev_unlock_states(sink_state, source_state); return ret; + } -EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate); +EXPORT_SYMBOL_GPL(__v4l2_subdev_link_validate); bool v4l2_subdev_has_pad_interdep(struct media_entity *entity, unsigned int pad0, unsigned int pad1) diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 8087c0ae3bc0a0a95512b4b0ff5257522a104ca0..16b6b265e711cf2293ce2478ef90a622beb869e5 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1223,6 +1223,7 @@ struct v4l2_subdev { * @vfh: pointer to &struct v4l2_fh * @state: pointer to &struct v4l2_subdev_state * @owner: module pointer to the owner of this file handle + * @context: pointer to subdevice context associated with the file handle * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*`` */ struct v4l2_subdev_fh { @@ -1351,6 +1352,9 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd, struct v4l2_subdev_format *source_fmt, struct v4l2_subdev_format *sink_fmt); +int __v4l2_subdev_link_validate(struct media_link *link, + struct media_device_context *mdev_context); + /** * v4l2_subdev_link_validate - validates a media link * @@ -1368,7 +1372,35 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd, * the video devices also implement their &media_entity_ops.link_validate * operation. */ -int v4l2_subdev_link_validate(struct media_link *link); +static inline int v4l2_subdev_link_validate(struct media_link *link) +{ + return __v4l2_subdev_link_validate(link, NULL); +} + +/** + * v4l2_subdev_link_validate_context - validates a media link in a media context + * + * @link: pointer to &struct media_link + * @mdev_context: the media device context + * + * This function calls the subdev's link_validate_context ops to validate + * if a media link is valid for streaming in a media device context. It also + * internally calls v4l2_subdev_link_validate_default() to ensure that width, + * height and the media bus pixel code are equal on both source and sink of the + * link. + * + * The function can be used as a drop-in &media_entity_ops.link_validate_context + * implementation for v4l2_subdev instances. It supports all links between + * subdevs, as well as links between subdevs and video devices, provided that + * the video devices also implement their + * &media_entity_ops.link_validate_context operation. + */ +static inline int +v4l2_subdev_link_validate_context(struct media_link *link, + struct media_device_context *mdev_context) +{ + return __v4l2_subdev_link_validate(link, mdev_context); +} /** * v4l2_subdev_has_pad_interdep - MC has_pad_interdep implementation for subdevs From patchwork Thu Jul 17 10:45:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116062 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMCs-0005Lp-3C for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:51:21 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 6DD653B65EE for ; Thu, 17 Jul 2025 10:50:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6764B2D837F; Thu, 17 Jul 2025 10:46:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="L3xM2sIN" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFC7B2D7801; Thu, 17 Jul 2025 10:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749179; cv=none; b=eVk0ZNc6b8aQrKPtO3+b0jA9J9iAg4YE7Tz/99IcF1OZrOlWOqIQaGzSwkEEQEjup4LH6L9Oa8/t8mRI8oYlmECX5cLsQJulfRBspJ5sj03KntPECYVUMapjlq3wcmd4I1L54aWaT7yKYSzWjsRCYXeBTD6msbpG6gDujO/tauA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749179; c=relaxed/simple; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gEDF3/FJPgPTPL4owhfjG4bYO9rC2IRj+oVICRdAsaBW1/bYT9B6cQ9IZtt8VU5OpnswcUPbEi3s3EgQQImkDUM9IxmWQEMnzaq3XxgVFMdtV8QYZnOEdzrbjRmhGlHrCxz3g8EkNCyvgC1HfoFypmokfJPq4YKD5UoTn49+AR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=L3xM2sIN; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F4C420EE; Thu, 17 Jul 2025 12:45:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749141; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=L3xM2sINUfbupiC+4EYm9Z6p/Zv84402Yye5DQ6rY0irVFPw9GLujgTxofj00VRJM WwhBg+9npeSIX0m1lJMP877j9Y/AmNMl3/frgxTchEGe6EkNK1QOJJdSoEWdpMHA88 OBUPUDXg+FYpinYgkQw7S6jAEBfUgRd8HcRcX/Rk= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:47 +0200 Subject: [PATCH DNI 21/26] media: pisp_be: Start and stop the media pipeline Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-21-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1293; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=NRkJxIVd0ZlILnv8udnGakJfoaHUIwP9DNE9boCy8vE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRRnPxjMuVJs1HDTPiwMpU50liJADhwZgbZ+ 3n0peOrAiaJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PKqND/9lgHHdpWtrKrn0s9fq2BS22V66oiOoXeL8C/PTv242eTMZeOE94cLl9MbKi4o+5xPUSgF BeOjGegTyWCk9qatrxSXUbEIlx98/xRYLXW6k27iRYnM9mSy2VGrG/R/Os+jhwDjC+FhXRoAcj1 438gbXywzXL1ywduiOQI5XFleXFU0hnet4mRrNDVDk/oZhBC/e+79ucyLeNSYpfUaAuv6IDub68 jMNrSN729Cr7IWSOIqtzcz0xL8+81s8VY0pBCoFEspXC5CAt+1UQerJJJWGTR0lwwGdfT131EY3 pzv8dR+EEHF/5Y10fG4GkqmQMNY1K5WWbX8I4NSFSn37M7H1iQ22PTVkiWyXaI+NU5WMJN98EJV la/yOjYj0cCpdMlL2qWs3v9spHNpfBJ3KjGhh141hwkOj49CB5xoEakxlinG8nZX+q03FF41wKG QPDrk4+zDNA8ZJBwGCWtTJFatlK1s5JEaaldh6FmuP3gzsT2usT5fAbrzhg/mDT2DwskRjbUiCC d36/yQrpr7Z/G20fbDWHjFKJ9BIRmrR4wLyvmWLUDMSv1RksKgBXJjpGn/RzJdJGZFWJzNXmqoG 4oF+g2oqfjhLklBeGkcvdSPI8xUAKGB3vtA96BOx4dI6HDcxbudyhriK2e9SE9MUg7vmyxzBI57 ELeGa7Yq1gwaB4g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Call video_device_pipeline_alloc_start() and video_device_pipeline_stop() at streaming start/stop time for all video nodes. Signed-off-by: Jacopo Mondi --- drivers/media/platform/raspberrypi/pisp_be/pisp_be.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index b30891718d8df9c48ce1b83ad9fcafb201105625..423cb21298309c2ba51214b129fbf6e875370c98 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -878,6 +878,10 @@ static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) if (ret < 0) goto err_return_buffers; + ret = video_device_pipeline_alloc_start(&node->vfd); + if (ret) + goto err_return_buffers; + scoped_guard(spinlock_irq, &pispbe->hw_lock) { node->pispbe->streaming_map |= BIT(node->id); node->pispbe->sequence = 0; @@ -933,6 +937,8 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) vb2_wait_for_all_buffers(&node->queue); + video_device_pipeline_stop(&node->vfd); + spin_lock_irq(&pispbe->hw_lock); pispbe->streaming_map &= ~BIT(node->id); From patchwork Thu Jul 17 10:45:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116080 Received: from sy.mirrors.kernel.org ([147.75.48.161]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMEn-0005Pi-2U for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:53:18 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id C5DD27BD2F6 for ; Thu, 17 Jul 2025 10:50:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E0BB729B227; Thu, 17 Jul 2025 10:46:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lMH6BjkN" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE4F22D8391; Thu, 17 Jul 2025 10:46:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749183; cv=none; b=lCkgQ1xYftcWN74yyNlt3xoYpTftFz2PW/ee3si1nHCBsIjeIubsbjMTsVCYWqM/MjpXl9wU9hT4iW3RBRADTLXdY3Mwv0KzfxBc7l5SWDX9LzLVinBCkWrN1FUP/60ksu7sT1oXXROHK2FwWPJoJ03B8q3Vu3GBByxHXMv3JjE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749183; c=relaxed/simple; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qWZzp+k+2Wlh8TDzV5JkVT4l49xbMXfVB5dVl/K/iEblUPm/7fV3SImTnD/0FWFAizJl5pjz63Oj2rPUxd67Bswt3MK4H8PqDPE71UkJEApMvsLViy9WDFM3vscOuVw1iLsQ6/mLvLu1MTVTaYBikuxt38748LIPwg50xWkNSZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=lMH6BjkN; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7AF2C1E74; Thu, 17 Jul 2025 12:45:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749142; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lMH6BjkNJfBOxwgNQfFoFlDmJyVde8+lHri01kln6JgFe7kf4uq6RWQWnNq+MIDBT PM9kBe0e1b16Ej8L8uqmqRPz5jHnVJNKV5gmxP6y2HBOYj1/jHj5z5A1RtNcDnvDx0 JBLbUklElR/VldW0dozOFWdWxUXYldqIumwveGaQ= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:48 +0200 Subject: [PATCH DNI 22/26] media: pisp_be: Add support for subdev state Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-22-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2271; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=9aZhUEi8nzk0nQ1lg/OYG7ZJZJnqA3OQHgxh4Ti+sSs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRRVJywCBAo7BIzYjV82f8QM3uGxHTpYi2Rt C39bakUCimJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PIFBD/4uLBHQuvpaSyM6QrS1Mwp9UC008FWTYRM5RQrbx+5/10SzLLpWWEjsismkOyWFAe+ld5N 97fTwHqiVHEIne/j+39cXN3J2Dyb+4DYRTfeaG8poBPhLsBQexXvsHlehsMKTN2cUrjqqdC3P+L Go8W3IqAV3hirQDAGN0Y78z/DWu7XbAFwZtdDd1Y2GosdLfw9x1WI2Cv6Lu0ZvS1jB5MTBdzpvk y/WpbOmSDvxMRHzdYWiLxcE3wrYBEK6LNk939H5Zbu3Tx6GGHGnn4KwwcQIuuw3CfB00l4Vamv5 ebbkBRD3gBn7QWJ21KQySTiv2i9c9gq3dWDTevk9I8dfzzlU1ggpotKjZG5butVooPSFuB9Vefi bF81TLpLk+gvQY+X34EyxD6bkrfmeVGcz0gl2bn1oN8ndw9/IC9hMaWCwmSHtsJBoo1L+kqacaO KSp1M/qgLLkOgTBfzKG84OI3W9NlEqqMDUrikaOUCAhzb1qOFiAIjqBXPoDn7epL6bo6wBkOqs3 RzCqVC4qf1wOLsDjs6j63CuprP29p+KbnkcYPAlGgbw4P3VGS19TVWsCmFcvA9bwhYpaw3jsP0c +0bCV7MiA/zb8Kk8g2+eytbe4vvjLzmsFVgT33leutuwIASJKTvuXLTewSX40hJ0/Gazhyw+qPu KzBIVTUKJJYVbNA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Add support for subdev state in the ISP subdevice by providing an init_state() callback and by calling init_finalize() before registering the subdev. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 41 ++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 423cb21298309c2ba51214b129fbf6e875370c98..5aec4f8979053500c870e71ce7171bbd1cac9606 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1496,12 +1496,49 @@ static const struct v4l2_subdev_ops pispbe_sd_ops = { .pad = &pispbe_pad_ops, }; +static int pispbe_init_state(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state) +{ + struct v4l2_mbus_framefmt *fmt; + + for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) { + fmt = v4l2_subdev_state_get_format(state, i); + + switch (i) { + case MAIN_INPUT_NODE: + fallthrough; + case OUTPUT0_NODE: + fallthrough; + case OUTPUT1_NODE: + fmt->width = 1920; + fmt->height = 1080; + fmt->code = MEDIA_BUS_FMT_FIXED; + break; + case CONFIG_NODE: + fmt->width = sizeof(struct pisp_be_tiles_config); + fmt->height = 1; + fmt->code = MEDIA_BUS_FMT_METADATA_FIXED; + break; + default: + /* No need to configure other nodes. */ + continue; + } + } + + return 0; +} + +static const struct v4l2_subdev_internal_ops pispbe_subdev_internal_ops = { + .init_state = pispbe_init_state, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd = &pispbe->sd; int ret; v4l2_subdev_init(sd, &pispbe_sd_ops); + sd->internal_ops = &pispbe_subdev_internal_ops; sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; sd->owner = THIS_MODULE; sd->dev = pispbe->dev; @@ -1517,6 +1554,10 @@ static int pispbe_init_subdev(struct pispbe_dev *pispbe) if (ret) goto error; + ret = v4l2_subdev_init_finalize(sd); + if (ret) + goto error; + ret = v4l2_device_register_subdev(&pispbe->v4l2_dev, sd); if (ret) goto error; From patchwork Thu Jul 17 10:45:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116071 Received: from am.mirrors.kernel.org ([147.75.80.249]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMDh-0005NS-0K for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:52:09 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 067FF1C24A68 for ; Thu, 17 Jul 2025 10:52:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B72FB2D979C; Thu, 17 Jul 2025 10:46:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uiwxmilU" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AC422D8DA4; Thu, 17 Jul 2025 10:46:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749184; cv=none; b=KCPBi9Yys/i4JqQ38n35/b6ZSTooaxZBy2CqAfOJqPUSp/QTme7dcxkHuKM3GNL30oR1y4I3qWIqLMmno8GyWtQtG9tkz+Kgb56FebS+hABW2uyCgqHBaxR8OYMXAtmwmjxt82OIPg1Th7QifeHH8/iF+vzHh1WEVWX8HEKWyCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749184; c=relaxed/simple; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a2cetHZ6nxEsT2Ku4Vd609c/V07AeowFxo8BunSJlYrcn4QW0+SjwaRoAk378r8+XmKnXr1G3t9GhF+I86VclkEO32iQTHYor6qcUism5Ubfaxpi0+mnzVXLwBz/gLjrE0qphW/LlJR+dTXwxOQJblv5TIJ4+qrgmxQe3yOKh9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=uiwxmilU; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C7D201AE2; Thu, 17 Jul 2025 12:45:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749144; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uiwxmilUyeUwUbhXT+Z3DOUylJFzWaOUMiyPqvYTUp4+ecDcr2ZRpyrJJ/pClK7EP HC9VLTioj6+n9rvkpV7bdvKWoF7FHTjE4wDOpktqUwrNqJmS2Etgzw5wAfkf9kTtL7 2rYDY8h6AG3srtfYxn3SaqJT9Ma2qMdDM8ZZUPDA= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:49 +0200 Subject: [PATCH DNI 23/26] media: pisp_be: Implement set/get_pad_fmt Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-23-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1831; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=YyXmr6Wtr5A4ih2ag5qo99kLDMbBWLONqHu1vyrmcVI=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRR446grg45jSOazWmLg7XRPDLLG/B80g+S+ 8a4h6emRMuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PHIcD/4nIDePGNOBSTZXua3WG5IPv5k4ZIU7nI1QDA4K1hbYrxMBgTT4OSo+Cme46A2XAe1vzgU nweba/418w7v34UBvBbQuVKa4HnWC1apjPZYAQ9QNkYkVXBSDeMoApB83UvIBgYzqpSzB4msTy+ HrvY4dzEVVNe7RrFUbtJJf313vzRfKcaEUtLz5JXE+CtdXLhv+iRLl3hovKuWv0Tpl9xw9cw0ca JyrLlnLiQ6/wfQaDdOe7obChvMgeErN9tv/8OEV4T56RCcGABsDlRJQAlRx1649PkLmuoxOD3ln cqeeVKdT9xjK0oFihEJtCvGJISxv11x2QfV6xjQpDg/EDMZ73SJC1zMtGcF0FEPGEOg2PCb8dAd jvx55GGnAfoc3PBl1+/Vu+q0H9GutBmNVhhMq4gdStnXr5GnfWu/D0TmdiJQtdG92/yBg10EClw 3uq0nvKh6JaITdRdIyDCzLI0ibnmH7nzVkvDC/KrhS3rhgw8GxxIfUWjk25Z6Db2q1bqRwx48DP HzCx0YF8z1KT0j54/+RxGp1w6pg3X/I7MvFtjFG+np8fRhvqXTBy2ZDYJhUd8LW2056qo0P+uf8 ZS1Bgt5F2kzwSWJL9GLjF3uz95lO1uyk9wihRscneEATpBN2VPzjLBgwFJkanvsR8NjRC/1aGDp i6TdcGazjri2pQA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Implement operation handlers for the set and get pad_fmt subdev pad operations. Format is only modifiable on the image nodes (of both output and capture types). Only sizes can be modified. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 29 ++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 5aec4f8979053500c870e71ce7171bbd1cac9606..22e440f387b5e5560b2cc80a8b3bf6064dc12d7c 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1488,7 +1488,36 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) return ret; } +static int pispbe_subdev_set_pad_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct v4l2_mbus_framefmt *fmt = &format->format; + + /* Only allow setting sizes on the image input and output pads. */ + + switch (format->pad) { + case TDN_INPUT_NODE: + case STITCH_INPUT_NODE: + case TDN_OUTPUT_NODE: + case STITCH_OUTPUT_NODE: + return -EINVAL; + } + + fmt->width = clamp(fmt->width, PISP_BACK_END_MIN_TILE_WIDTH, + PISP_BACK_END_MAX_TILE_WIDTH); + fmt->height = clamp(fmt->height, PISP_BACK_END_MIN_TILE_HEIGHT, + PISP_BACK_END_MAX_TILE_HEIGHT); + fmt->code = MEDIA_BUS_FMT_FIXED; + + *v4l2_subdev_state_get_format(state, format->pad) = *fmt; + + return 0; +} + static const struct v4l2_subdev_pad_ops pispbe_pad_ops = { + .set_fmt = pispbe_subdev_set_pad_fmt, + .get_fmt = v4l2_subdev_get_fmt, .link_validate = v4l2_subdev_link_validate_default, }; From patchwork Thu Jul 17 10:45:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116074 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMET-0005P3-31 for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:53:03 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 6205E3B4775 for ; Thu, 17 Jul 2025 10:52:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D29BA2DA77C; Thu, 17 Jul 2025 10:46:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cQ2dcqSA" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E1F62D8DBD; Thu, 17 Jul 2025 10:46:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749186; cv=none; b=K7ZyzdlAC7eX0Am3i9HIaw2POwO7eekzKZtASBFScZ2HvDWdi3Bn1Oqo+eBcPbBsjkNDtCaNJEuAtbabMnAdYm5tC+OxoRiC6j6fGGpxj0ka5UU/PNr9LNDAMDk2XcvVW9r5l0WHpYGK9ihGq6AK2j70d3ZYyJbXqkeDGPKiWlI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749186; c=relaxed/simple; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=A+q++r9x+7SFo0j5QEPZU9tMfOcYFRHeLqJy8nmWp8KBOP1+KhJtpTLZSj4yg9LRckjOM5T4mxvU6CvVuI3+BcHArUgEt4FhP7GANcyucTid9tlgxrceTOtWWIeOM1jDnZ9QGDnZHef0fiN3Pg4/4NPITSOn1djAXxIA7+IXVhY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=cQ2dcqSA; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 39114198D; Thu, 17 Jul 2025 12:45:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749145; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cQ2dcqSA5NULHDFYF95BOzb2Z9+5jQEIZxGFqamQec1RH/cSXPus8dG4wUAW+JuDl tgIiPG4d9ZvUBt0XZaCrgYAKMYFVllO6gZ6pjztSU6P2KlL+efH+qkhyHJZlJU8LMF uzjGUT62eqiDvb9VY8xNNSLZryTZ2/kDz+W9CyIM= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:50 +0200 Subject: [PATCH DNI 24/26] media: pisp_be: Implement link validation Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-24-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3786; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Ee9J4mCFwsXOb6mRepzI/4oK3qWzitZE/F4KVxjoUTk=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRR8lqbOlQJMHVVCjAWrrf8cZGeJbBx/gxq1 dWWByn84cOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PMdFD/9l8t4aehW6Q1OdtHw9gsMaWhBbA3/zHf/j9Up9Ky3olpwFQQpkb9xaD8oXN7lUHHkZp5P bjCX81l5L0bCbiCPSy5KoQITMtxvwg24/VDI2fbbyiP1U4yQwTyeLxuzgldRJkuaBj7BsVZcPP4 Zo2LEGzbkfAyTHIKHB20mUPXyfSSa09MNRCeoDrpnwS0rs69LGM9MhlbAl619YPcyYP2nGKWu5t 9a/W+2H1iNFxHGriNqktbAR7ht5LRo8yxHeNUSsmusQOkFrkbEtIMZ/8sz38RROqAUr6lCsOSH5 5sWctmkRaY6DbGIA6MSZVUdFqcjHYxzAzQLpIw7UPYSFTKu5ma/MXR9F40Z3UYOeMJL8rUImuOZ m83BBFza3CUplaCjJ1gyPq2m9hSkrRoIIPjiKgnQPpci1lI/UwINgF0K1qWVQ9n6wiJoqBwU7Pw 3X/GrcEOEcQ4JbKdbwqSm2z7kgELHym5MpdHmJY78pOP0L9noX7s/oYehA5/0C6QvIGUi6zxrMz lKyDl23RPMtmyjEXcCZdmgcr7zAreAbwhb7MqlvdrxQZtjxJ9HaxazaGaoE90LTLINGZMDAz3Jj vObO9EtrWQh1We0RrOYpL688ktx7H9KxBDGEEkHEj3fSMvRGAVTW1TkRACfQBgP0uX+OZ6fceEo RoZT4adGK6nmaxA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no Implement link validation to validate that the sizes of the format on the video device matches the size programmed on the ISP subdevice connected pad. Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 67 ++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 22e440f387b5e5560b2cc80a8b3bf6064dc12d7c..2a8c09a9c70952c9f99e542271e994d62392c617 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1362,6 +1362,67 @@ static const struct v4l2_ioctl_ops pispbe_node_ioctl_ops = { .vidioc_streamoff = vb2_ioctl_streamoff, }; +static int pispbe_link_validate(struct media_link *link) +{ + const struct v4l2_mbus_framefmt *sd_fmt; + struct v4l2_pix_format_mplane *pix_mp; + struct v4l2_subdev_state *state; + struct media_entity *vdev_ent; + struct media_entity *sd_ent; + struct pispbe_node *node; + struct v4l2_subdev *sd; + + if (is_media_entity_v4l2_video_device(link->source->entity)) { + vdev_ent = link->source->entity; + sd_ent = link->sink->entity; + } else { + sd_ent = link->source->entity; + vdev_ent = link->sink->entity; + } + + node = container_of(media_entity_to_video_device(vdev_ent), + struct pispbe_node, vfd); + switch (node->id) { + case TDN_INPUT_NODE: + fallthrough; + case STITCH_INPUT_NODE: + fallthrough; + case TDN_OUTPUT_NODE: + fallthrough; + case STITCH_OUTPUT_NODE: + fallthrough; + case CONFIG_NODE: + /* Skip validation for these nodes. */ + return 0; + } + pix_mp = &node->format.fmt.pix_mp; + + sd = media_entity_to_v4l2_subdev(sd_ent); + state = v4l2_subdev_get_unlocked_active_state(sd); + sd_fmt = v4l2_subdev_state_get_format(state, node->id); + + /* Only check for sizes. */ + if (pix_mp->width != sd_fmt->width) { + dev_dbg(node->pispbe->dev, + "%s: width does not match (vdev %u, sd %u)\n", + __func__, pix_mp->width, sd_fmt->width); + return -EPIPE; + } + + if (pix_mp->height != sd_fmt->height) { + dev_dbg(node->pispbe->dev, + "%s: height does not match (vdev %u, sd %u)\n", + __func__, pix_mp->height, sd_fmt->height); + return -EPIPE; + } + + return 0; +} + +static const struct media_entity_operations pispbe_node_entity_ops = { + .link_validate = pispbe_link_validate, +}; + static const struct video_device pispbe_videodev = { .name = PISPBE_NAME, .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ @@ -1445,6 +1506,7 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) vdev->device_caps = V4L2_CAP_STREAMING | node_desc[id].caps; node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; + entity->ops = &pispbe_node_entity_ops; ret = media_entity_pads_init(entity, 1, &node->pad); if (ret) { dev_err(pispbe->dev, @@ -1561,6 +1623,10 @@ static const struct v4l2_subdev_internal_ops pispbe_subdev_internal_ops = { .init_state = pispbe_init_state, }; +static const struct media_entity_operations pispbe_subdev_entity_ops = { + .link_validate = v4l2_subdev_link_validate, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd = &pispbe->sd; @@ -1569,6 +1635,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pispbe) v4l2_subdev_init(sd, &pispbe_sd_ops); sd->internal_ops = &pispbe_subdev_internal_ops; sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; + sd->entity.ops = &pispbe_subdev_entity_ops; sd->owner = THIS_MODULE; sd->dev = pispbe->dev; strscpy(sd->name, PISPBE_NAME, sizeof(sd->name)); From patchwork Thu Jul 17 10:45:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116077 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMEf-0005PT-0K for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:53:09 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id ECC993B84CF for ; Thu, 17 Jul 2025 10:52:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9538E2DAFC6; Thu, 17 Jul 2025 10:46:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BrD1IhGT" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E566B2D9EE7; Thu, 17 Jul 2025 10:46:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749187; cv=none; b=OML+bDUMlOaZ4FAQxxUPx+2WVSQdSYt1rJvGcUtj6PH3ZbFBDdX1TgqSvrj1AtYGO4T0tENzjk6mdcZqRjLynEPFMGvQdehXQHI907EOYAJFc/Oct67jlAQyHJiXBnUqrca0U7KelPWpF7crKUSearvjuuamzixdDNju8ZNokYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749187; c=relaxed/simple; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TxQpnWYhriAwn4Kh5VWWO7T1AdPZacQwkP6JZTfbwVenB7XCG67cb7PES8CnuEBrNVEV2fDtpBW4Oadg5u/fhTyQxfHrsW7FpDYjLuBnMiI8R0p4jnwEmkBaE3CuF44aG/zihGP6Wo0zVhD+9ZfCpbC1Lxg4SKWXpB1Yk3Hgt70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=BrD1IhGT; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD40E21F2; Thu, 17 Jul 2025 12:45:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749147; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BrD1IhGTWABRNCLptFDK6B2BG+aNtn5V2O8P7iT8TFlXY2TVKXZ3oVX2NvecRkAtG A10+98QdZn1Y70fvBsT4sEXHihkJpX2Bsvp0LgjcoV+YMXlhUszsRS54d8aFTkdOIv reVy7OIWUATev7qnUsj+4hviUokkock5b2Gvxj9A= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:51 +0200 Subject: [PATCH DNI 25/26] media: pisp_be: Register devnode to userspace Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-25-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1240; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=veD5bCafCFdh1GJ2SEAikts/NXjEHZdIvuKFJ7cMJ+I=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRREpXqgCve0J6ofHIN6a1A+7ldn/yTcHDdL 1B9xdg25KWJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PEQ1D/90/rpes/8rJaL1PpQbGCrRfT1XMHhyFHUNb/dkqPzOtYdBmhVzf8FhN5C+4sH1ONE38mv IfRq218bZFaln6qGAB45EMK3euim6tJb3Lw+vMCmgpSnNQjd/J9yBnGb49ZGoCXUscsAIZprBCE qzKjmZOE+S+fQDxoxlILFlNzrIVRITViA3YGtXK1Avzc5fIQw9TE4SOHX5GoBBGKlYIhiB8vWZK Xys0e+BQ9BTDn9TE2J7HAaDkQHkGuFGB+bdIj6TydwKMzisjENEFdubINbDO7RbrZljuxs9br91 n84gh+ojvCebu1Pu6kHyFQdZQF0LlHL+FoWq/TScV4MajMB9XE67rp02fCO0PkNl6UdOuN9QuJT 5O8R8j3UynCIOrhvpD9d3lkIfZg2zL2A0C4J8ddK80a5luABOqH4gFS9Oqbk84IZcc58AHwvtZB FzQjK14Jx5y+o/ZKMdCX3mtMJdjebdaQD2fgV9/kzwgzFwFYcj9HZcf6zvdP9PTKQpkgch7RC63 EPgXlh/8bkNbwr+zEnJzhDXMnaKufVwth8sZAxy3QR/06GrDSQARuRhuIMC95XLXLtD7idEIsma Vdztnoki6OCWrCAQpBcADWs2uxiRo6d8j+xtFQJwSVLyEwQaPWafrdiEUWH0KAp53nSHnRPuXG5 b2+er40/ZWwL2vw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=ham autolearn_force=no To set formats on the subdev, we need to have the devnode registered to userspace. Signed-off-by: Jacopo Mondi --- drivers/media/platform/raspberrypi/pisp_be/pisp_be.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 2a8c09a9c70952c9f99e542271e994d62392c617..df3cdd81843376abf98bb184cde74d4d66b0ecfe 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -1635,6 +1635,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pispbe) v4l2_subdev_init(sd, &pispbe_sd_ops); sd->internal_ops = &pispbe_subdev_internal_ops; sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER; + sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE; sd->entity.ops = &pispbe_subdev_entity_ops; sd->owner = THIS_MODULE; sd->dev = pispbe->dev; @@ -1658,7 +1659,7 @@ static int pispbe_init_subdev(struct pispbe_dev *pispbe) if (ret) goto error; - return 0; + return v4l2_device_register_subdev_nodes(&pispbe->v4l2_dev); error: media_entity_cleanup(&sd->entity); From patchwork Thu Jul 17 10:45:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 116086 Received: from sv.mirrors.kernel.org ([139.178.88.99]) by linuxtv.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ucMFJ-0005Qt-1b for patchwork@linuxtv.org; Thu, 17 Jul 2025 10:53:52 +0000 Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id DE95E4E7695 for ; Thu, 17 Jul 2025 10:53:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7512B2DCF53; Thu, 17 Jul 2025 10:46:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vNnVg/8+" X-Original-To: linux-media@vger.kernel.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFEAD2DBF5C; Thu, 17 Jul 2025 10:46:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749191; cv=none; b=joJxaUDUcTqc1usxKfNIrDSqKOjXv9Hk8mJUkohUD6LWGA5GgHr5eN5rjSUvzCzh9ImjWS5glLNCFnrxPDQI+d1v02MDsqiTI+7ElEBrxW4DcuQgd1XY4Qy1LHlf7leeI2gJ8zMkc6pO0UUJtqS7IpvQIbdnDPvd1eNC3oe4Hzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752749191; c=relaxed/simple; bh=9hY/8OW4RZ/aCyB+nwuYkoJcODl5p0hvMLLamBNGSMU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eG+PdsbAxTlqnmtRpZc6TaTKFemXUu3+oPK1L/bFDxJNHkeGh4qg8gH9KDbW1fOK0+cDhuqcTv8Yf8hHGrnwf2nwrFUolD51hsba462ueg+tWfwL8E3/Rsj9nDXnlyL9ljEUe/kiyr7wBrxgcNL/JAxzxP32XwwlFDvRsgA09ZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=vNnVg/8+; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Received: from [192.168.0.172] (mob-5-90-140-254.net.vodafone.it [5.90.140.254]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7617820EE; Thu, 17 Jul 2025 12:45:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1752749149; bh=9hY/8OW4RZ/aCyB+nwuYkoJcODl5p0hvMLLamBNGSMU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=vNnVg/8+ptDjIXdB8Z583VRGWeNglLZJ9Zryg6QhZUUZpX9aXndkL/rS+oUFG4kg/ m+mKVZN6+3yozL3jyRys4t/hPaggqFkTNHIVxZZihgCAUxT5RhvokOjbOjz3Tay+n5 4wZfv2wLVz1B86K3Y9VsvZBZh3GdE8ErXl+NaUNI= From: Jacopo Mondi Date: Thu, 17 Jul 2025 12:45:52 +0200 Subject: [PATCH DNI 26/26] media: pisp_be: Add support for multi-context Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250717-multicontext-mainline-2025-v1-26-81ac18979c03@ideasonboard.com> References: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> In-Reply-To: <20250717-multicontext-mainline-2025-v1-0-81ac18979c03@ideasonboard.com> To: Sakari Ailus , Laurent Pinchart , Tomi Valkeinen , Kieran Bingham , Nicolas Dufresne , Mauro Carvalho Chehab , Tomasz Figa , Marek Szyprowski , Raspberry Pi Kernel Maintenance , Florian Fainelli , Broadcom internal kernel review list , Hans Verkuil Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=45175; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=9hY/8OW4RZ/aCyB+nwuYkoJcODl5p0hvMLLamBNGSMU=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBoeNRRDhnVf74fFKeO3LlsZ9mZJ6cstOH8Ga7n/ ZSnie2j0xeJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaHjUUQAKCRByNAaPFqFW PCc/D/9HFAMA2b5HijfPO5HM8rV64RDKLFCffJe8NstsZkVu/Wx35U/6KPuTqeXye69dJBkWW2b ghDu/OkbnqdWYzuld4ISX9pGV9udcld5aPYU0CadLThJrrgHRD/ZVMYJ6aAwLrtnyif+ocOO192 xtxVpIwF+M6TrzdDw0wOwWMI1SnxpKWyGUPgFtwfkdUosS+JlPT11WxELEkDXYG8+DE+x0pPeV4 tWU+h3Mza5ki+IU23bj0kzaO4X4hQi7b8jwlsAFhNISJRonQA6nOk4GTpFjt5/UyATOBSboq6Zf LqIf4GSNtzVS7taZKe8/K4zq24lR5ZybF4igRQaSrlfxr25CagTWStD1QGFBYrUAzmY2YdeJjZW JPJUMng9cEFc6cx8rtkTw9YvIc1+5mdjYb+QBe5xn9Vo5z8F/1YeUQXaPqfHhma+F+SG9lakqJX s+MguDwLbR+Z1xucgQa/nRIYL2HRFqEZG0Ef/CE8ONc3EUitXNbDnDA93ev2PmYKPXVA20NKjpL iKi5BZ5siHWqv8uZCQbrCOBSoe0GiAkMMw2HYf5bPavCkk2WGPoPs9Z1p/yLp70q/WlNF3+0smk H21QJHD38s1DVUIEqQiQcz+51P2yrKgjwLhvqy2bn+Sn8pScTNXlIhq9vMjd7vjd4bwHDaZZDti J2yzAXXtTQlv0tw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-LSpam-Score: -8.6 (--------) X-LSpam-Report: No, score=-8.6 required=5.0 tests=ARC_SIGNED=0.001,ARC_VALID=-0.1,BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,DMARC_PASS=-0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_VALIDITY_CERTIFIED=-3,RCVD_IN_VALIDITY_RPBL=1.31,RCVD_IN_VALIDITY_SAFE=-2,SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no Add support for context multiplexing to the PiSP BackEnd driver. Move information that used to be per-node to the video context and global information to the media context. Implement media and video context allocation and release operations to allocate and initialize a context and initialize the video queue there contained. Remove the per-node video device format and the buffer queues and port the driver to use the ones in the video device context. The operations that used to work with a pispbe_node now operates on a pispbe_context. - v4l2 ioctl ops: receive a file pointer, retrieve the associated context from the open file handle - vb2 ops: receive a queue, retrieve the context from the queue - internal driver ops: given a media device context retrieve the video context from the other video devices when assembling a job Signed-off-by: Jacopo Mondi --- .../media/platform/raspberrypi/pisp_be/pisp_be.c | 651 +++++++++++++++------ 1 file changed, 473 insertions(+), 178 deletions(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index df3cdd81843376abf98bb184cde74d4d66b0ecfe..742fc9dddbead84fb2c9a615c9dccb3afcc66a55 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -146,6 +146,29 @@ static const struct pispbe_node_description node_desc[PISPBE_NUM_NODES] = { ((node)->buf_type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) || \ ((node)->buf_type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) +/* ---------------------------------------------------------------------------- + * Media device context + */ + +struct pispbe_media_context { + struct media_device_context mdev_context; + u32 streaming_map; + unsigned int sequence; + dma_addr_t config_dma_addr; + struct pisp_be_tiles_config *config; +}; + +static struct pispbe_media_context * +pispbe_media_context(struct media_device_context *mdev_context) +{ + return container_of(mdev_context, struct pispbe_media_context, + mdev_context); +} + +/* ---------------------------------------------------------------------------- + * Video device context + */ + /* * Structure to describe a single node /dev/video which represents a single * input or output queue to the PiSP Back End device. @@ -161,14 +184,96 @@ struct pispbe_node { struct pispbe_dev *pispbe; /* Video device lock */ struct mutex node_lock; - /* vb2_queue lock */ - struct mutex queue_lock; - struct list_head ready_queue; - struct vb2_queue queue; +}; + +static struct pispbe_node *pispbe_node_from_vdev(struct video_device *vdev) +{ + return container_of(vdev, struct pispbe_node, vfd); +} + +/* Structure to describe a single execution context for a node. */ + + struct pispbe_node_context { + struct video_device_context vdev_context; + struct v4l2_format format; + struct list_head ready_queue; const struct pisp_be_format *pisp_format; + struct pispbe_dev *pispbe; + struct pispbe_node *node; +}; + +static struct pispbe_node_context * +pispbe_node_context(struct video_device_context *ctx) +{ + return container_of(ctx, struct pispbe_node_context, vdev_context); +} + +static struct pispbe_node_context * +pispbe_node_context_from_entity(struct media_entity_context *ctx) +{ + struct video_device_context *vdev_context = + container_of(ctx, struct video_device_context, base); + + return pispbe_node_context(vdev_context); +} + +static struct pispbe_node_context * +pispbe_node_context_from_queue(struct vb2_queue *queue) +{ + return pispbe_node_context(video_device_context_from_queue(queue)); +} + +static struct pispbe_node_context * +pispbe_node_context_from_file(struct file *file, struct video_device *vfd) +{ + return pispbe_node_context(video_device_context_from_file(file, vfd)); +} + +static struct pispbe_node_context * +pispbe_get_dev_context(struct pispbe_media_context *pispbe_mdev_context, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_get(&pispbe_mdev_context->mdev_context, + vdev); + + return ctx ? pispbe_node_context(ctx) : NULL; +} + +static void pispbe_put_dev_context(struct pispbe_node_context *ctx) +{ + video_device_context_put(&ctx->vdev_context); +} + +static struct pispbe_media_context * +pispbe_media_context_from_dev(struct pispbe_node_context *context) +{ + return pispbe_media_context(context->vdev_context.base.mdev_context); +} + +/* ---------------------------------------------------------------------------- + * ISP subdevice context + */ +struct pispbe_subdev_context { + struct v4l2_subdev_context sd_context; }; +static struct pispbe_subdev_context * +pispbe_subdev_context(struct v4l2_subdev_context *ctx) +{ + return container_of(ctx, struct pispbe_subdev_context, sd_context); +} + +static struct pispbe_subdev_context * +pispbe_subdev_context_from_entity(struct media_entity_context *ctx) +{ + struct v4l2_subdev_context *sd_context = + container_of(ctx, struct v4l2_subdev_context, base); + + return pispbe_subdev_context(sd_context); +} + /* For logging only, use the entity name with "pispbe" and separator removed */ #define NODE_NAME(node) \ (node_desc[(node)->id].ent_name + sizeof(PISPBE_NAME)) @@ -181,6 +286,7 @@ struct pispbe_job { * then captures, then metadata last. */ struct pispbe_buffer *buf[PISPBE_NUM_NODES]; + struct pispbe_media_context *context; }; struct pispbe_hw_enables { @@ -196,6 +302,7 @@ struct pispbe_job_descriptor { struct pisp_be_tiles_config *config; struct pispbe_hw_enables hw_enables; dma_addr_t tiles; + struct pispbe_media_context *context; }; /* @@ -205,7 +312,6 @@ struct pispbe_job_descriptor { struct pispbe_dev { struct device *dev; struct pispbe_dev *pispbe; - struct pisp_be_tiles_config *config; void __iomem *be_reg_base; struct clk *clk; struct v4l2_device v4l2_dev; @@ -213,17 +319,15 @@ struct pispbe_dev { struct media_device mdev; struct media_pad pad[PISPBE_NUM_NODES]; /* output pads first */ struct pispbe_node node[PISPBE_NUM_NODES]; - dma_addr_t config_dma_addr; - unsigned int sequence; - u32 streaming_map; struct pispbe_job queued_job, running_job; - /* protects "hw_busy" flag, streaming_map and job_queue */ + /* protects "hw_busy" flag and job_queue */ spinlock_t hw_lock; bool hw_busy; /* non-zero if a job is queued or is being started */ struct list_head job_queue; int irq; u32 hw_version; u8 done, started; + struct media_pipeline pipe; }; static u32 pispbe_rd(struct pispbe_dev *pispbe, unsigned int offset) @@ -307,30 +411,31 @@ struct pispbe_buffer { }; static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, - struct pispbe_node *node) + struct pispbe_node_context *context) { - unsigned int num_planes = node->format.fmt.pix_mp.num_planes; + struct v4l2_format *format = &context->format; + unsigned int num_planes = format->fmt.pix_mp.num_planes; unsigned int plane_factor = 0; unsigned int size; unsigned int p; - if (!buf || !node->pisp_format) + if (!buf || !context->pisp_format) return 0; /* * Determine the base plane size. This will not be the same - * as node->format.fmt.pix_mp.plane_fmt[0].sizeimage for a single + * as format->fmt.pix_mp.plane_fmt[0].sizeimage for a single * plane buffer in an mplane format. */ - size = node->format.fmt.pix_mp.plane_fmt[0].bytesperline * - node->format.fmt.pix_mp.height; + size = format->fmt.pix_mp.plane_fmt[0].bytesperline * + format->fmt.pix_mp.height; for (p = 0; p < num_planes && p < PISPBE_MAX_PLANES; p++) { addr[p] = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, p); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } - for (; p < PISPBE_MAX_PLANES && node->pisp_format->plane_factor[p]; p++) { + for (; p < PISPBE_MAX_PLANES && context->pisp_format->plane_factor[p]; p++) { /* * Calculate the address offset of this plane as needed * by the hardware. This is specifically for non-mplane @@ -338,7 +443,7 @@ static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, * for the V4L2_PIX_FMT_YUV420 format. */ addr[p] = addr[0] + ((size * plane_factor) >> 3); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } return num_planes; @@ -354,11 +459,13 @@ static dma_addr_t pispbe_get_addr(struct pispbe_buffer *buf) static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, struct pispbe_job_descriptor *job, + struct pispbe_media_context *mdev_context, struct pispbe_buffer *buf[PISPBE_NUM_NODES]) { struct pispbe_hw_enables *hw_en = &job->hw_enables; struct pisp_be_tiles_config *config = job->config; dma_addr_t *addrs = job->hw_dma_addrs; + struct pispbe_node_context *ctx; int ret; /* Take a copy of the "enable" bitmaps so we can modify them. */ @@ -369,8 +476,10 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * Main input first. There are 3 address pointers, corresponding to up * to 3 planes. */ - ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], - &pispbe->node[MAIN_INPUT_NODE]); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[MAIN_INPUT_NODE].vfd); + ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) { /* Shouldn't happen, we have validated an input is available. */ dev_warn(pispbe->dev, "ISP-BE missing input\n"); @@ -426,9 +535,11 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, /* Main image output channels. */ for (unsigned int i = 0; i < PISP_BACK_END_NUM_OUTPUTS; i++) { + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + i].vfd); ret = pispbe_get_planes_addr(addrs + 7 + 3 * i, - buf[OUTPUT0_NODE + i], - &pispbe->node[OUTPUT0_NODE + i]); + buf[OUTPUT0_NODE + i], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) hw_en->rgb_enables &= ~(PISP_BE_RGB_ENABLE_OUTPUT0 << i); } @@ -449,37 +560,48 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * * Returns 0 if a job has been successfully prepared, < 0 otherwise. */ -static int pispbe_prepare_job(struct pispbe_dev *pispbe) +static int pispbe_prepare_job(struct pispbe_dev *pispbe, + struct pispbe_node_context *context) { + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); struct pispbe_job_descriptor __free(kfree) *job = NULL; struct pispbe_buffer *buf[PISPBE_NUM_NODES] = {}; + struct pispbe_node_context *ctx; unsigned int streaming_map; unsigned int config_index; - struct pispbe_node *node; lockdep_assert_irqs_enabled(); scoped_guard(spinlock_irq, &pispbe->hw_lock) { static const u32 mask = BIT(CONFIG_NODE) | BIT(MAIN_INPUT_NODE); - if ((pispbe->streaming_map & mask) != mask) + if ((mdev_context->streaming_map & mask) != mask) return -ENODEV; /* * Take a copy of streaming_map: nodes activated after this * point are ignored when preparing this job. */ - streaming_map = pispbe->streaming_map; + streaming_map = mdev_context->streaming_map; } job = kzalloc(sizeof(*job), GFP_KERNEL); if (!job) return -ENOMEM; - node = &pispbe->node[CONFIG_NODE]; - buf[CONFIG_NODE] = list_first_entry_or_null(&node->ready_queue, + job->context = mdev_context; + + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[CONFIG_NODE].vfd); + if (!ctx) + return -ENODEV; + + buf[CONFIG_NODE] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (!buf[CONFIG_NODE]) return -ENODEV; @@ -487,8 +609,8 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) job->buffers[CONFIG_NODE] = buf[CONFIG_NODE]; config_index = buf[CONFIG_NODE]->vb.vb2_buf.index; - job->config = &pispbe->config[config_index]; - job->tiles = pispbe->config_dma_addr + + job->config = &mdev_context->config[config_index]; + job->tiles = mdev_context->config_dma_addr + config_index * sizeof(struct pisp_be_tiles_config) + offsetof(struct pisp_be_tiles_config, tiles); @@ -529,12 +651,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) ignore_buffers = true; } - node = &pispbe->node[i]; - /* Pull a buffer from each V4L2 queue to form the queued job */ - buf[i] = list_first_entry_or_null(&node->ready_queue, + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx && !ignore_buffers) + goto err_return_buffers; + + buf[i] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (buf[i]) { list_del(&buf[i]->ready_list); job->buffers[i] = buf[i]; @@ -545,7 +672,7 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) } /* Convert buffers to DMA addresses for the hardware */ - pispbe_xlate_addrs(pispbe, job, buf); + pispbe_xlate_addrs(pispbe, job, mdev_context, buf); scoped_guard(spinlock_irq, &pispbe->hw_lock) { list_add_tail(&job->queue, &pispbe->job_queue); @@ -558,19 +685,25 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) err_return_buffers: for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) { - struct pispbe_node *n = &pispbe->node[i]; - if (!buf[i]) continue; /* Return the buffer to the ready_list queue */ - list_add(&buf[i]->ready_list, &n->ready_queue); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx) + continue; + + list_add(&buf[i]->ready_list, &ctx->ready_queue); + pispbe_put_dev_context(ctx); } return -ENODEV; } -static void pispbe_schedule(struct pispbe_dev *pispbe, bool clear_hw_busy) +static void pispbe_schedule(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, + bool clear_hw_busy) { struct pispbe_job_descriptor *job; @@ -591,9 +724,11 @@ static void pispbe_schedule(struct pispbe_dev *pispbe, bool clear_hw_busy) for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) pispbe->queued_job.buf[i] = job->buffers[i]; - pispbe->queued_job.valid = true; pispbe->hw_busy = true; + + pispbe->queued_job.context = job->context; + pispbe->queued_job.valid = true; } /* @@ -615,13 +750,13 @@ static void pispbe_isr_jobdone(struct pispbe_dev *pispbe, for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) { if (buf[i]) { buf[i]->vb.vb2_buf.timestamp = ts; - buf[i]->vb.sequence = pispbe->sequence; + buf[i]->vb.sequence = job->context->sequence; vb2_buffer_done(&buf[i]->vb.vb2_buf, VB2_BUF_STATE_DONE); } } - pispbe->sequence++; + job->context->sequence++; } static irqreturn_t pispbe_isr(int irq, void *dev) @@ -674,18 +809,20 @@ static irqreturn_t pispbe_isr(int irq, void *dev) } /* check if there's more to do before going to sleep */ - pispbe_schedule(pispbe, can_queue_another); + pispbe_schedule(pispbe, NULL, can_queue_another); return IRQ_HANDLED; } static int pisp_be_validate_config(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, struct pisp_be_tiles_config *config) { u32 bayer_enables = config->config.global.bayer_enables; u32 rgb_enables = config->config.global.rgb_enables; + struct pispbe_node_context *context; + struct v4l2_pix_format_mplane *mp; struct device *dev = pispbe->dev; - struct v4l2_format *fmt; unsigned int bpl, size; if (!(bayer_enables & PISP_BE_BAYER_ENABLE_INPUT) == @@ -702,36 +839,50 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, } /* Ensure output config strides and buffer sizes match the V4L2 formats. */ - fmt = &pispbe->node[TDN_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[TDN_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_TDN_OUTPUT) { bpl = config->config.tdn_output_format.stride; size = bpl * config->config.tdn_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on tdn_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on tdn_output\n", __func__); return -EINVAL; } } - fmt = &pispbe->node[STITCH_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[STITCH_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_STITCH_OUTPUT) { bpl = config->config.stitch_output_format.stride; size = bpl * config->config.stitch_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on stitch_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on stitch_output\n", __func__); return -EINVAL; @@ -746,8 +897,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_WALLPAPER_ROLL) continue; /* TODO: Size checks for wallpaper formats */ - fmt = &pispbe->node[OUTPUT0_NODE + j].format; - for (unsigned int i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + j].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + + for (unsigned int i = 0; i < mp->num_planes; i++) { bpl = !i ? config->config.output_format[j].image.stride : config->config.output_format[j].image.stride2; size = bpl * config->config.output_format[j].image.height; @@ -756,13 +914,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_SAMPLING_420) size >>= 1; - if (fmt->fmt.pix_mp.plane_fmt[i].bytesperline < bpl) { + if (mp->plane_fmt[i].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on output %d\n", __func__, j); + dev_dbg(dev, "Expected %u, got %u\n", + bpl, mp->plane_fmt[i].bytesperline); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[i].sizeimage < size) { + if (mp->plane_fmt[i].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on output\n", __func__); return -EINVAL; @@ -777,10 +937,12 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, unsigned int *nplanes, unsigned int sizes[], struct device *alloc_devs[]) { + struct pispbe_node_context *context = pispbe_node_context_from_queue(q); + struct v4l2_format *format = &context->format; struct pispbe_node *node = vb2_get_drv_priv(q); struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; if (*nplanes) { if (*nplanes != num_planes) @@ -788,8 +950,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (sizes[i] < size) return -EINVAL; @@ -801,8 +963,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, *nplanes = num_planes; for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; sizes[i] = size; } @@ -815,15 +977,20 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) { - struct pispbe_node *node = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_queue *q = vb->vb2_queue; + struct pispbe_node_context *context = pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct v4l2_format *format = &context->format; + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; for (unsigned int i = 0; i < num_planes; i++) { unsigned long size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (vb2_plane_size(vb, i) < size) { dev_dbg(pispbe->dev, @@ -836,12 +1003,12 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) } if (node->id == CONFIG_NODE) { - void *dst = &node->pispbe->config[vb->index]; + void *dst = &mdev_context->config[vb->index]; void *src = vb2_plane_vaddr(vb, 0); memcpy(dst, src, sizeof(struct pisp_be_tiles_config)); - return pisp_be_validate_config(pispbe, dst); + return pisp_be_validate_config(pispbe, mdev_context, dst); } return 0; @@ -849,27 +1016,34 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) static void pispbe_node_buffer_queue(struct vb2_buffer *buf) { - struct vb2_v4l2_buffer *vbuf = - container_of(buf, struct vb2_v4l2_buffer, vb2_buf); - struct pispbe_buffer *buffer = - container_of(vbuf, struct pispbe_buffer, vb); - struct pispbe_node *node = vb2_get_drv_priv(buf->vb2_queue); + struct vb2_v4l2_buffer *vbuf = container_of(buf, struct vb2_v4l2_buffer, + vb2_buf); + struct pispbe_buffer *buffer = container_of(vbuf, struct pispbe_buffer, + vb); + struct vb2_queue *q = buf->vb2_queue; + struct pispbe_node_context *context = pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); - list_add_tail(&buffer->ready_list, &node->ready_queue); + list_add_tail(&buffer->ready_list, &context->ready_queue); /* * Every time we add a buffer, check if there's now some work for the hw * to do. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); } static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_node_context *context = pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_buffer *buf, *tmp; int ret; @@ -878,28 +1052,28 @@ static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) if (ret < 0) goto err_return_buffers; - ret = video_device_pipeline_alloc_start(&node->vfd); + ret = video_device_context_pipeline_alloc_start(&context->vdev_context); if (ret) goto err_return_buffers; scoped_guard(spinlock_irq, &pispbe->hw_lock) { - node->pispbe->streaming_map |= BIT(node->id); - node->pispbe->sequence = 0; + mdev_context->streaming_map |= BIT(node->id); + mdev_context->sequence = 0; } dev_dbg(pispbe->dev, "%s: for node %s (count %u)\n", __func__, NODE_NAME(node), count); dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - node->pispbe->streaming_map); + mdev_context->streaming_map); /* Maybe we're ready to run. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); return 0; err_return_buffers: - list_for_each_entry_safe(buf, tmp, &node->ready_queue, ready_list) { + list_for_each_entry_safe(buf, tmp, &context->ready_queue, ready_list) { list_del(&buf->ready_list); vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); } @@ -909,7 +1083,10 @@ static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) static void pispbe_node_stop_streaming(struct vb2_queue *q) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_node_context *context = pispbe_node_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_job_descriptor *job, *temp; struct pispbe_buffer *buf; @@ -926,7 +1103,7 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) */ dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); do { - buf = list_first_entry_or_null(&node->ready_queue, + buf = list_first_entry_or_null(&context->ready_queue, struct pispbe_buffer, ready_list); if (buf) { @@ -935,14 +1112,14 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) } } while (buf); - vb2_wait_for_all_buffers(&node->queue); + vb2_wait_for_all_buffers(&context->vdev_context.queue); video_device_pipeline_stop(&node->vfd); spin_lock_irq(&pispbe->hw_lock); - pispbe->streaming_map &= ~BIT(node->id); + mdev_context->streaming_map &= ~BIT(node->id); - if (pispbe->streaming_map == 0) { + if (mdev_context->streaming_map == 0) { /* * If all nodes have stopped streaming release all jobs * without holding the lock. @@ -960,7 +1137,7 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) pm_runtime_put_autosuspend(pispbe->dev); dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - pispbe->streaming_map); + mdev_context->streaming_map); } static const struct vb2_ops pispbe_node_queue_ops = { @@ -1001,6 +1178,8 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); if (!NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1009,7 +1188,7 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get capture format for node %s\n", NODE_NAME(node)); @@ -1021,6 +1200,8 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); if (NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1029,7 +1210,7 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for node %s\n", NODE_NAME(node)); @@ -1041,6 +1222,8 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); if (!NODE_IS_META(node) || NODE_IS_CAPTURE(node)) { dev_dbg(pispbe->dev, @@ -1049,7 +1232,7 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for meta node %s\n", NODE_NAME(node)); @@ -1219,17 +1402,19 @@ static int pispbe_node_s_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_cap(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set capture format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1242,17 +1427,19 @@ static int pispbe_node_s_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set output format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1265,17 +1452,19 @@ static int pispbe_node_s_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_meta_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = &meta_out_supported_formats[0]; + context->format = *f; + context->pisp_format = &meta_out_supported_formats[0]; dev_dbg(pispbe->dev, "Set output format for meta node %s to %p4cc\n", NODE_NAME(node), &f->fmt.meta.dataformat); @@ -1287,8 +1476,10 @@ static int pispbe_node_enum_fmt(struct file *file, void *priv, struct v4l2_fmtdesc *f) { struct pispbe_node *node = video_drvdata(file); + struct pispbe_node_context *context = + pispbe_node_context_from_file(file, &node->vfd); - if (f->type != node->queue.type) + if (f->type != context->vdev_context.queue.type) return -EINVAL; if (NODE_IS_META(node)) { @@ -1362,13 +1553,17 @@ static const struct v4l2_ioctl_ops pispbe_node_ioctl_ops = { .vidioc_streamoff = vb2_ioctl_streamoff, }; -static int pispbe_link_validate(struct media_link *link) +static int pispbe_link_validate(struct media_link *link, + struct media_device_context *mdev_context) { + struct pispbe_subdev_context *isp_context; + struct pispbe_node_context *node_context; const struct v4l2_mbus_framefmt *sd_fmt; struct v4l2_pix_format_mplane *pix_mp; struct v4l2_subdev_state *state; struct media_entity *vdev_ent; struct media_entity *sd_ent; + struct video_device *vdev; struct pispbe_node *node; struct v4l2_subdev *sd; @@ -1380,8 +1575,12 @@ static int pispbe_link_validate(struct media_link *link) vdev_ent = link->sink->entity; } - node = container_of(media_entity_to_video_device(vdev_ent), - struct pispbe_node, vfd); + vdev = media_entity_to_video_device(vdev_ent); + node_context = pispbe_node_context(video_device_context_get(mdev_context, + vdev)); + + node = node_context->node; + switch (node->id) { case TDN_INPUT_NODE: fallthrough; @@ -1395,10 +1594,13 @@ static int pispbe_link_validate(struct media_link *link) /* Skip validation for these nodes. */ return 0; } - pix_mp = &node->format.fmt.pix_mp; + pix_mp = &node_context->format.fmt.pix_mp; sd = media_entity_to_v4l2_subdev(sd_ent); - state = v4l2_subdev_get_unlocked_active_state(sd); + isp_context = pispbe_subdev_context(v4l2_subdev_context_get(mdev_context, + sd)); + + state = v4l2_subdev_get_unlocked_active_state(&isp_context->sd_context); sd_fmt = v4l2_subdev_state_get_format(state, node->id); /* Only check for sizes. */ @@ -1419,28 +1621,16 @@ static int pispbe_link_validate(struct media_link *link) return 0; } -static const struct media_entity_operations pispbe_node_entity_ops = { - .link_validate = pispbe_link_validate, -}; - -static const struct video_device pispbe_videodev = { - .name = PISPBE_NAME, - .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ - .fops = &pispbe_fops, - .ioctl_ops = &pispbe_node_ioctl_ops, - .minor = -1, - .release = video_device_release_empty, -}; - -static void pispbe_node_def_fmt(struct pispbe_node *node) +static void pispbe_node_def_fmt(struct pispbe_node *node, + struct pispbe_node_context *context) { + struct v4l2_format *format = &context->format; + if (NODE_IS_META(node) && NODE_IS_OUTPUT(node)) { /* Config node */ - struct v4l2_format *f = &node->format; - - f->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; - f->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); - f->type = node->buf_type; + format->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; + format->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); + format->type = node->buf_type; } else { struct v4l2_format f = { .fmt.pix_mp.pixelformat = V4L2_PIX_FMT_YUV420, @@ -1449,36 +1639,39 @@ static void pispbe_node_def_fmt(struct pispbe_node *node) .type = node->buf_type, }; pispbe_try_format(&f, node); - node->format = f; + *format = f; } - node->pisp_format = pispbe_find_fmt(node->format.fmt.pix_mp.pixelformat); + context->pisp_format = pispbe_find_fmt(format->fmt.pix_mp.pixelformat); } -/* - * Initialise a struct pispbe_node and register it as /dev/video - * to represent one of the PiSP Back End's input or output streams. - */ -static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +static int pispbe_alloc_node_context(struct media_entity *entity, + struct media_entity_context **ctx) { - bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); - struct pispbe_node *node = &pispbe->node[id]; - struct media_entity *entity = &node->vfd.entity; - struct video_device *vdev = &node->vfd; - struct vb2_queue *q = &node->queue; + struct video_device *vdev = container_of(entity, struct video_device, + entity); + struct pispbe_node *node = pispbe_node_from_vdev(vdev); + struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_node_context *context; + struct vb2_queue *q; int ret; - node->id = id; - node->pispbe = pispbe; - node->buf_type = node_desc[id].buf_type; + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_node_context *)*ctx; - mutex_init(&node->node_lock); - mutex_init(&node->queue_lock); - INIT_LIST_HEAD(&node->ready_queue); + video_device_init_context(vdev, &context->vdev_context); + + context->pispbe = pispbe; + context->node = node; - node->format.type = node->buf_type; - pispbe_node_def_fmt(node); + INIT_LIST_HEAD(&context->ready_queue); + context->format.type = node->buf_type; + pispbe_node_def_fmt(node, context); + + q = &context->vdev_context.queue; q->type = node->buf_type; q->io_modes = VB2_MMAP | VB2_DMABUF; q->mem_ops = &vb2_dma_contig_memops; @@ -1488,21 +1681,70 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; q->dev = pispbe->dev; /* get V4L2 to handle node->queue locking */ - q->lock = &node->queue_lock; + q->lock = &context->vdev_context.queue_lock; ret = vb2_queue_init(q); if (ret < 0) { dev_err(pispbe->dev, "vb2_queue_init failed\n"); - goto err_mutex_destroy; + goto err_cleanup; } + return 0; + +err_cleanup: + kfree(*ctx); + return ret; +} + +static void pispbe_free_node_context(struct media_entity_context *ctx) +{ + struct pispbe_node_context *context = + pispbe_node_context_from_entity(ctx); + + list_del_init(&context->ready_queue); + video_device_cleanup_context(&context->vdev_context); + kfree(context); +} + +static const struct media_entity_operations pispbe_node_entity_ops = { + .alloc_context = pispbe_alloc_node_context, + .destroy_context = pispbe_free_node_context, + .link_validate_context = pispbe_link_validate, +}; + +static const struct video_device pispbe_videodev = { + .name = PISPBE_NAME, + .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ + .fops = &pispbe_fops, + .ioctl_ops = &pispbe_node_ioctl_ops, + .minor = -1, + .release = video_device_release_empty, +}; + +/* + * Initialise a struct pispbe_node and register it as /dev/video + * to represent one of the PiSP Back End's input or output streams. + */ +static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +{ + bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); + struct pispbe_node *node = &pispbe->node[id]; + struct media_entity *entity = &node->vfd.entity; + struct video_device *vdev = &node->vfd; + int ret; + + node->id = id; + node->pispbe = pispbe; + node->buf_type = node_desc[id].buf_type; + + mutex_init(&node->node_lock); + *vdev = pispbe_videodev; /* default initialization */ strscpy(vdev->name, node_desc[id].ent_name, sizeof(vdev->name)); vdev->v4l2_dev = &pispbe->v4l2_dev; vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX; /* get V4L2 to serialise our ioctls */ vdev->lock = &node->node_lock; - vdev->queue = &node->queue; vdev->device_caps = V4L2_CAP_STREAMING | node_desc[id].caps; node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; @@ -1512,15 +1754,16 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) dev_err(pispbe->dev, "Failed to register media pads for %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { dev_err(pispbe->dev, "Failed to register video %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } video_set_drvdata(vdev, node); @@ -1542,14 +1785,15 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) err_unregister_video_dev: video_unregister_device(&node->vfd); -err_unregister_queue: - vb2_queue_release(&node->queue); err_mutex_destroy: mutex_destroy(&node->node_lock); - mutex_destroy(&node->queue_lock); return ret; } +/* ---------------------------------------------------------------------------- + * ISP subdevice operations + */ + static int pispbe_subdev_set_pad_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, struct v4l2_subdev_format *format) @@ -1623,8 +1867,83 @@ static const struct v4l2_subdev_internal_ops pispbe_subdev_internal_ops = { .init_state = pispbe_init_state, }; +static int pispbe_alloc_subdev_context(struct media_entity *entity, + struct media_entity_context **ctx) +{ + struct v4l2_subdev *sd = container_of(entity, struct v4l2_subdev, + entity); + struct pispbe_subdev_context *context; + + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_subdev_context *)*ctx; + + v4l2_subdev_init_context(sd, &context->sd_context); + + return 0; +} + +static void pispbe_free_subdev_context(struct media_entity_context *ctx) +{ + struct pispbe_subdev_context *context = + pispbe_subdev_context_from_entity(ctx); + + v4l2_subdev_cleanup_context(&context->sd_context); + kfree(context); +} + static const struct media_entity_operations pispbe_subdev_entity_ops = { - .link_validate = v4l2_subdev_link_validate, + .alloc_context = pispbe_alloc_subdev_context, + .destroy_context = pispbe_free_subdev_context, + .link_validate_context = v4l2_subdev_link_validate_context, +}; + +static int pispbe_media_alloc_context(struct media_device *mdev, + struct media_device_context **ctx) +{ + struct pispbe_media_context *context; + + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_media_context *)*ctx; + + media_device_init_context(mdev, &context->mdev_context); + + context->streaming_map = 0; + + context->config = + dma_alloc_coherent(mdev->dev, + sizeof(struct pisp_be_tiles_config) * + PISP_BE_NUM_CONFIG_BUFFERS, + &context->config_dma_addr, GFP_KERNEL); + if (!context->config) { + dev_err(mdev->dev, + "Unable to allocate cached config buffers.\n"); + return -ENOMEM; + } + + return 0; +} +static void pispbe_media_destroy_context(struct media_device_context *ctx) +{ + struct pispbe_media_context *context = + (struct pispbe_media_context *)ctx; + + if (context->config) + dma_free_coherent(ctx->mdev->dev, + sizeof(struct pisp_be_tiles_config) * + PISP_BE_NUM_CONFIG_BUFFERS, + context->config, context->config_dma_addr); + + media_device_cleanup_context(&context->mdev_context); + kfree(context); +} + +static const struct media_device_ops pispbe_media_device_ops = { + .alloc_context = pispbe_media_alloc_context, + .destroy_context = pispbe_media_destroy_context, }; static int pispbe_init_subdev(struct pispbe_dev *pispbe) @@ -1677,6 +1996,7 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) mdev = &pispbe->mdev; mdev->hw_revision = pispbe->hw_version; mdev->dev = pispbe->dev; + mdev->ops = &pispbe_media_device_ops; strscpy(mdev->model, PISPBE_NAME, sizeof(mdev->model)); media_device_init(mdev); @@ -1704,26 +2024,11 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) if (ret) goto err_unregister_nodes; - pispbe->config = - dma_alloc_coherent(pispbe->dev, - sizeof(struct pisp_be_tiles_config) * - PISP_BE_NUM_CONFIG_BUFFERS, - &pispbe->config_dma_addr, GFP_KERNEL); - if (!pispbe->config) { - dev_err(pispbe->dev, "Unable to allocate cached config buffers.\n"); - ret = -ENOMEM; - goto err_unregister_mdev; - } - return 0; -err_unregister_mdev: - media_device_unregister(mdev); err_unregister_nodes: - while (num_regist-- > 0) { + while (num_regist-- > 0) video_unregister_device(&pispbe->node[num_regist].vfd); - vb2_queue_release(&pispbe->node[num_regist].queue); - } v4l2_device_unregister_subdev(&pispbe->sd); media_entity_cleanup(&pispbe->sd.entity); err_unregister_v4l2: @@ -1735,14 +2040,6 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) static void pispbe_destroy_devices(struct pispbe_dev *pispbe) { - if (pispbe->config) { - dma_free_coherent(pispbe->dev, - sizeof(struct pisp_be_tiles_config) * - PISP_BE_NUM_CONFIG_BUFFERS, - pispbe->config, - pispbe->config_dma_addr); - } - dev_dbg(pispbe->dev, "Unregister from media controller\n"); v4l2_device_unregister_subdev(&pispbe->sd); @@ -1751,9 +2048,7 @@ static void pispbe_destroy_devices(struct pispbe_dev *pispbe) for (int i = PISPBE_NUM_NODES - 1; i >= 0; i--) { video_unregister_device(&pispbe->node[i].vfd); - vb2_queue_release(&pispbe->node[i].queue); mutex_destroy(&pispbe->node[i].node_lock); - mutex_destroy(&pispbe->node[i].queue_lock); } media_device_cleanup(&pispbe->mdev);