Message ID | 20231228131617.3411561-1-liujianfeng1994@gmail.com (mailing list archive) |
---|---|
Headers |
Received: from sv.mirrors.kernel.org ([139.178.88.99]) by www.linuxtv.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <linux-media+bounces-3068-patchwork=linuxtv.org@vger.kernel.org>) id 1rIqFx-0015W4-6g for patchwork@linuxtv.org; Thu, 28 Dec 2023 13:17:01 +0000 Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 62D08284FF1 for <patchwork@linuxtv.org>; Thu, 28 Dec 2023 13:16:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 217F58465; Thu, 28 Dec 2023 13:16:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LICwZNZ+" X-Original-To: linux-media@vger.kernel.org Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 985E079CF; Thu, 28 Dec 2023 13:16:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1d3cfb1568eso49343665ad.1; Thu, 28 Dec 2023 05:16:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703769409; x=1704374209; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ZdxMA0IFnQNSF87xNJWS4MQoHn8cWo2ZJ5T/rZOTLd4=; b=LICwZNZ+nOf3lXg54F3hV5fE2iRbdB5vVNY5xDIRg+CwB1NwSlxIozDXrT8BMAcTMo 0EnT9Zxd6chwRviwffDdA0kXFKqdQJTRCM4SdmqSEDhm3hqIp4zTXx+4hqJe+XvlFySg rTd4+/EotHmceEhbg/T4FFFKKfHm6vQovStBoHHB1ZsJW8+iRP0ttQOTy8u3EsMDE8pz D7dgJ5kYXvK4y3vOQOHozQkYFDS4WfOaGxkzeNxIywEu4oZ3l+huxFWEgYpP0NmUVMQn +gusuD/21XQyolkOU7/9h5ySxiFrc+oUWEzyvce88U2hqcyDxdALq9oSp5kstFQuogyJ tj6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703769409; x=1704374209; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZdxMA0IFnQNSF87xNJWS4MQoHn8cWo2ZJ5T/rZOTLd4=; b=f3K5K7RjOCn+NZOUxbqGmEQSzBUdZEt1foEOQ0N9Oem05p79cvdzij22Aj8sDJRWM3 pidvyzRDrIp/FNmBJcD6tnz5q81KX8hQHVe/jYrpf9ywmYDgUelUrlmrrMVl6MjZRR/B 1qlQIz4jlOMndxu4223P80+5ZqaWEYYwPShy4PdwpNITgclwu/fQyQFbW9aoejMVz2bG 8vNm1Esa68mYcjF4AG6WaYUPRBj5bPz0MiF+MgiaAdzhPqBRmfuXT8SzhR6AP+bO7+zb Atkf/TqCcbMqzOEyqRwI++odQk+hAnhwhaakCBCardPCcJP8TaNYJ0V3nW/pBSDoPmGW qpcw== X-Gm-Message-State: AOJu0YwS5kfP8IoLtHhqKhPVW7TkT/NIfKl3YkkvsYRRgg2o/0WV8iCN z6BaXhQtNn8GBGpoCEgBLdc= X-Google-Smtp-Source: AGHT+IH8H9gke14Hfds3PK61sbjEENzKw+rV/8YgR/y3014PaI2W0cSmtiFb5LSDfFmIPkfRF8gw9w== X-Received: by 2002:a17:903:44d:b0:1d0:8554:5dd5 with SMTP id iw13-20020a170903044d00b001d085545dd5mr10136244plb.37.1703769408774; Thu, 28 Dec 2023 05:16:48 -0800 (PST) Received: from localhost.localdomain ([2408:8207:2540:8c00:3708:559:ea20:9883]) by smtp.gmail.com with ESMTPSA id m10-20020a170902db0a00b001d0b4693539sm13864221plx.189.2023.12.28.05.16.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 05:16:48 -0800 (PST) From: Jianfeng Liu <liujianfeng1994@gmail.com> To: ezequiel@vanguardiasur.com.ar, p.zabel@pengutronix.de, mchehab@kernel.org, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, heiko@sntech.de, sfr@canb.auug.org.au Cc: liujianfeng1994@gmail.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 0/3] Add hantro g1 video decoder support for RK3588 Date: Thu, 28 Dec 2023 21:16:14 +0800 Message-Id: <20231228131617.3411561-1-liujianfeng1994@gmail.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: <linux-media.vger.kernel.org> List-Subscribe: <mailto:linux-media+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-media+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-LSpam-Score: -4.8 (----) X-LSpam-Report: No, score=-4.8 required=5.0 tests=BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FORGED_FROMDOMAIN=0.001,FREEMAIL_FROM=0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3 autolearn=unavailable autolearn_force=no |
Series |
Add hantro g1 video decoder support for RK3588
|
|
Message
Jianfeng Liu
Dec. 28, 2023, 1:16 p.m. UTC
This is the v2 version of this series adding hantro g1 video decoder support for rk3588. Changes in v2: - Fix alphabetical order in patch1 and patch3 - Sort device tree node by bus-address - Drop rk3588_vpu_variant fron v1 because that is exactly the same as rk3568_vpu_variant - Link to v1: https://lore.kernel.org/all/20231227173911.3295410-1-liujianfeng1994@gmail.com Jianfeng Liu (3): media: verisilicon: Add support for Hantro G1 on RK3588 arm64: dts: rockchip: Add Hantro G1 VPU support for RK3588 dt-bindings: media: rockchip-vpu: Add RK3588 compatible .../bindings/media/rockchip-vpu.yaml | 1 + arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 20 +++++++++++++++++++ .../media/platform/verisilicon/hantro_drv.c | 1 + 3 files changed, 22 insertions(+)
Comments
On Thu, 28 Dec 2023 at 13:16, Jianfeng Liu <liujianfeng1994@gmail.com> wrote: > > This is the v2 version of this series adding hantro g1 video decoder > support for rk3588. > > Changes in v2: > - Fix alphabetical order in patch1 and patch3 > - Sort device tree node by bus-address > - Drop rk3588_vpu_variant fron v1 because that is exactly the same as rk3568_vpu_variant > - Link to v1: https://lore.kernel.org/all/20231227173911.3295410-1-liujianfeng1994@gmail.com > > Jianfeng Liu (3): > media: verisilicon: Add support for Hantro G1 on RK3588 > arm64: dts: rockchip: Add Hantro G1 VPU support for RK3588 > dt-bindings: media: rockchip-vpu: Add RK3588 compatible > > .../bindings/media/rockchip-vpu.yaml | 1 + > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 20 +++++++++++++++++++ > .../media/platform/verisilicon/hantro_drv.c | 1 + > 3 files changed, 22 insertions(+) > > -- > 2.34.1 Tested H.264 and VP8 decode with Fluster on NanoPC T6; H.264 JVT-AVC_V1 test suite: Ran 129/135 tests successfully. (matches FFmpeg-H.264 software decoder) H.264 JVT-FR-EXT test suite: Ran 44/69 tests successfully (some failures caused by lack of support for Hi10P and 4:2:2) VP8-TEST-VECTORS test suite: Ran 59/61 tests successfully (matches FFmpeg-VP8 software decoder) Full results at: https://gist.github.com/sigmaris/d3d8586bfced5ddc021aa9dab94d4ab8 Tested-by: Hugh Cole-Baker <sigmaris@gmail.com>
Hi, Jianfeng Am 28.12.23 um 14:16 schrieb Jianfeng Liu: > This is the v2 version of this series adding hantro g1 video decoder > support for rk3588. > > Changes in v2: > - Fix alphabetical order in patch1 and patch3 > - Sort device tree node by bus-address > - Drop rk3588_vpu_variant fron v1 because that is exactly the same as rk3568_vpu_variant if the RK3568 and RK3588 variants match, patch [1/3] is not necessary. Just document a additional compatible in a similar way it's being done for rk3188/rk3066 or rk3228/rk3399. If there are ever differences we don't know about yet, a additional variant can still be added in the driver. Alex > - Link to v1: https://lore.kernel.org/all/20231227173911.3295410-1-liujianfeng1994@gmail.com > > Jianfeng Liu (3): > media: verisilicon: Add support for Hantro G1 on RK3588 > arm64: dts: rockchip: Add Hantro G1 VPU support for RK3588 > dt-bindings: media: rockchip-vpu: Add RK3588 compatible > > .../bindings/media/rockchip-vpu.yaml | 1 + > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 20 +++++++++++++++++++ > .../media/platform/verisilicon/hantro_drv.c | 1 + > 3 files changed, 22 insertions(+) >
Thanks for your test with fluster. I also tested on my rock-5b(rk3588) and orangepi-3b(rk3566) which are sharing the same hantro g1 ip. I am running on ubuntu jammy so I'm using gstreamer 1.20.1 rk3566 and rk3588 are sharing the same results: JVT-AVC_V1 test suite with decoder GStreamer-H.264-V4L2SL-Gst1.0: Ran 112/135 tests successfully. JVT-FR_EXT test suite with decoder GStreamer-H.264-V4L2SL-Gst1.0: Ran 27/69 tests successfully. VP8-TEST-VECTORS test suite with decoder GStreamer-VP8-V4L2SL-Gst1.0: Ran 59/61 tests successfully. H264 decoder test has less test case passing. I think that's because of my low gstreamer version. If you have a rk356x board I guess you will get the same result as rk3588. That should be the mainline support status of hantro g1 decoder. For ffmpeg at the moment fluster doesn't support v4l2-request decoder. I tried Kwiboo's fork[1] but failed to pass tests with decoder FFmpeg-H.264-V4L2-request. I can decode video with ffmpeg command like: "ffmpeg -benchmark -hwaccel drm -hwaccel_output_format drm_prime -i Big_Buck_Bunny_1080_10s_30MB.mp4 -f null -" [1] https://github.com/Kwiboo/fluster/tree/v4l2-request
Jianfeng
>On Sat, 30 Dec 2023 10:46:42 +0100, Alex Bee <knaerzche@gmail.com> wrote: >if the RK3568 and RK3588 variants match, patch [1/3] is not necessary. Just >document a additional compatible in a similar way it's being done for >rk3188/rk3066 or rk3228/rk3399. >If there are ever differences we don't know about yet, a additional variant >can still be added in the driver. Hi Alex, I did try to not touching hantro driver code and just enabling the node in devicetree by a compatible property like: compatible = "rockchip,rk3588-vpu", "rockchip,rk3568-vpu"; but the hantro g1 node is not binded by hantro driver. I guess that is because hantro driver is already binded to the hantro av1 vpu on rk3588, so I added patch[1/3] to make it work. Jianfeng
Hi, On 2023-12-30 10:52, amazingfate wrote: > Thanks for your test with fluster. I also tested on my rock-5b(rk3588) > and orangepi-3b(rk3566) which are sharing the same hantro g1 ip. I am > running on ubuntu jammy so I'm using gstreamer 1.20.1 > > rk3566 and rk3588 are sharing the same results: > JVT-AVC_V1 test suite with decoder GStreamer-H.264-V4L2SL-Gst1.0: > Ran 112/135 tests successfully. > JVT-FR_EXT test suite with decoder GStreamer-H.264-V4L2SL-Gst1.0: > Ran 27/69 tests successfully. > VP8-TEST-VECTORS test suite with decoder GStreamer-VP8-V4L2SL-Gst1.0: > Ran 59/61 tests successfully. > > H264 decoder test has less test case passing. I think that's because > of my low gstreamer version. If you have a rk356x board I guess you > will get the same result as rk3588. That should be the mainline support > status of hantro g1 decoder. > > For ffmpeg at the moment fluster doesn't support v4l2-request decoder. > I tried Kwiboo's fork[1] but failed to pass tests with decoder > FFmpeg-H.264-V4L2-request. I can decode video with ffmpeg command like: > "ffmpeg -benchmark -hwaccel drm -hwaccel_output_format drm_prime -i Big_Buck_Bunny_1080_10s_30MB.mp4 -f null -" I have only tested this fork of fluster with ffmpeg 6.x, what version of ffmpeg did you test with? I was expecting it to also work on ffmpeg 5.x. Please also note that ffmpeg v4l2-request patches at [2] contain some NV15/NV20 ffmpeg pix_fmt patches that fail ffmpeg tests. They are not needed for decoding of 10-bit frames using rkvdec but they are required to be able to run fluster test suite JVT-FR_EXT on rkvdec. (hantro g1 do not support 10-bit frames on rk) [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1 Regards, Jonas > > [1] https://github.com/Kwiboo/fluster/tree/v4l2-request >
Hi Jonas, On Sat, 30 Dec 2023 12:49:38 +0100, Jonas Karlman <jonas@kwiboo.se> wrote: >I have only tested this fork of fluster with ffmpeg 6.x, what version of >ffmpeg did you test with? I was expecting it to also work on ffmpeg 5.x. I am using ffmpeg 6.0 with v4l2-request patches from libreelec[1]. Ffmpeg v4l2 decoder in fluster fork is using ffmpeg args: "-hwaccel_device /dev/dri/renderD128" which make the test fall with hantro g1 on rk3588. After removing it I can run tests by ffmpeg v4l2-request decoder. Rk3566 and rk3588 are sharing the same results: JVT-AVC_V1 test suite with decoder FFmpeg-H.264-V4L2-request: Ran 127/135 tests successfully. JVT-FR_EXT test suite with decoder FFmpeg-H.264-V4L2-request: Ran 44/69 tests successfully. VP8-TEST-VECTORS test suite with decoder FFmpeg-VP8-V4L2-request: Ran 59/61 tests successfully. [1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch Regards, Jianfeng
Hi Alex, On Sat, 30 Dec 2023 10:46:42 +0100, Alex Bee <knaerzche@gmail.com> wrote: >if the RK3568 and RK3588 variants match, patch [1/3] is not necessary. Just >document a additional compatible in a similar way it's being done for >rk3188/rk3066 or rk3228/rk3399. >If there are ever differences we don't know about yet, a additional variant >can still be added in the driver. You are right, this way works. I should have encounted a ccache issue which made me running a devicetree I didn't want. I will drop patch[1/3] and only add devicetree node and dt-binding in v3. Best regards, Jianfeng
Hi Jianfeng, On 2023-12-30 16:31, amazingfate wrote: > Hi Jonas, > > On Sat, 30 Dec 2023 12:49:38 +0100, Jonas Karlman <jonas@kwiboo.se> wrote: >> I have only tested this fork of fluster with ffmpeg 6.x, what version of >> ffmpeg did you test with? I was expecting it to also work on ffmpeg 5.x. > > I am using ffmpeg 6.0 with v4l2-request patches from libreelec[1]. Great, that patch should match my v4l2-request-n6.0.1 branch [2], and does not have any NV15/NV20 ffmpeg pix fmt patches that could interfere. [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.0.1/ > Ffmpeg v4l2 decoder in fluster fork is using ffmpeg args: > "-hwaccel_device /dev/dri/renderD128" > which make the test fall with hantro g1 on rk3588. After removing it I > can run tests by ffmpeg v4l2-request decoder. Thanks for confirming and testing again, and I fully understand why "-hwaccel_device /dev/dri/renderD128" caused issues on rk3588 :-) The commit "HACK: hwcontext_drm: do not require drm device" was required to run ffmpeg without a hwaccel_device, and I tested fluster without it. Regards, Jonas > > Rk3566 and rk3588 are sharing the same results: > JVT-AVC_V1 test suite with decoder FFmpeg-H.264-V4L2-request: > Ran 127/135 tests successfully. > JVT-FR_EXT test suite with decoder FFmpeg-H.264-V4L2-request: > Ran 44/69 tests successfully. > VP8-TEST-VECTORS test suite with decoder FFmpeg-VP8-V4L2-request: > Ran 59/61 tests successfully. > > [1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch > > Regards, > Jianfeng