| Message ID | 20211205181618.1041699-1-aford173@gmail.com (mailing list archive) |
|---|---|
| Headers |
Received: from vger.kernel.org ([23.128.96.18])
by www.linuxtv.org with esmtp (Exim 4.92)
(envelope-from <linux-media-owner@vger.kernel.org>)
id 1mtw44-007tBg-7T; Sun, 05 Dec 2021 18:16:44 +0000
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
id S237131AbhLESUH (ORCPT <rfc822;mkrufky@linuxtv.org> + 1 other);
Sun, 5 Dec 2021 13:20:07 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56862 "EHLO
lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
with ESMTP id S237102AbhLESUD (ORCPT
<rfc822;linux-media@vger.kernel.org>); Sun, 5 Dec 2021 13:20:03 -0500
Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com
[IPv6:2607:f8b0:4864:20::12c])
by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FB55C061714;
Sun, 5 Dec 2021 10:16:36 -0800 (PST)
Received: by mail-il1-x12c.google.com with SMTP id t8so8075984ilu.8;
Sun, 05 Dec 2021 10:16:36 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20210112;
h=from:to:cc:subject:date:message-id:mime-version
:content-transfer-encoding;
bh=b7hPh5mvYUUQHhkNNfoyCMD5i8ZznhnbU9nW0aRlESc=;
b=cmpP6hAnfSA5CEdGDnS04nX/ONCTYoYT50LViDGS8fK2PSD/NeynRzFthz1B5u+skM
QjWeeDp39fs2dDsVddhPiqzf0Ur3P+3mDHVo/sBOf+K6lPmwwTik7Y99N9AQeAeva9Vt
1RY3TxyDPVOcIVmxDbjDROtrZPB3ejjY09s/hC7cK6QtZ54dXzohEcN7U9M5KcR+DTCX
G1T8bmkQG+IoL+IiKoVbpe5XsnuzXFBcgrCakoO44sd1U6O9YSf/jHVLBe0P/C64BU0D
ysxHUI+y5+mY7uoVfCrQIEukijEybRXcTo5NsE7ooj1maicuFdbGVmqkBm75/WLg4SD/
Wvig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
:content-transfer-encoding;
bh=b7hPh5mvYUUQHhkNNfoyCMD5i8ZznhnbU9nW0aRlESc=;
b=xhHy8l/IIMICQlQNCutRheAQpIlqI8fa6pnDOepvAzgHRfDmPmeeW1L9g/134ur8+1
I4wbjyWnjTzFg2T9iVmro6hWzktd0WDJI2T4lZWbZdguLjyKYuvha4SeEdC5SXcq6Yly
46TQ7/v37nDo+h5OiXe6XarZ3PS+UM5IuSVPqhC+oyOfWDePGGmqvY1JWf77F3cqKOl/
A6OEdI0Yvj33z7NznMIZAgANI1SriN7pxfLJ+lyJhTeg5khtTEjdU7CVrwOkEHyFb0vl
0tJnU3sQumJQ7Jl4s9VUk/NxI5quhAVArM9XzzxOjzitB70VQtkWoTdGg8VygyZ2ZIyq
WQEQ==
X-Gm-Message-State: AOAM5300gBFA/3wKjUxE2J6Fj4ZAKjggTGmTnO3FzZJhZA/znteKdPLu
T8K4SZUEniy2Qf7DkA7U56+36jQgBPu7iQ==
X-Google-Smtp-Source:
ABdhPJyHMiqyQNSp4p0YcCTauDSHb7YDwtcIhgnq+pwoSX0gBorVeEiMc6bRja2+eWJlJbWpI6vdiQ==
X-Received: by 2002:a05:6e02:1a08:: with SMTP id
s8mr6025833ild.158.1638728195036;
Sun, 05 Dec 2021 10:16:35 -0800 (PST)
Received: from aford-IdeaCentre-A730.lan
([2601:448:8400:9e8:9ea0:bb99:b16a:6159])
by smtp.gmail.com with ESMTPSA id
z12sm5808164ilu.27.2021.12.05.10.16.32
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Sun, 05 Dec 2021 10:16:34 -0800 (PST)
From: Adam Ford <aford173@gmail.com>
To: linux-media@vger.kernel.org
Cc: benjamin.gaignard@collabora.com, cphealy@gmail.com,
nicolas@ndufresne.ca, Adam Ford <aford173@gmail.com>,
Rob Herring <robh+dt@kernel.org>,
Shawn Guo <shawnguo@kernel.org>,
Sascha Hauer <s.hauer@pengutronix.de>,
Pengutronix Kernel Team <kernel@pengutronix.de>,
Fabio Estevam <festevam@gmail.com>,
NXP Linux Team <linux-imx@nxp.com>,
Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>,
Philipp Zabel <p.zabel@pengutronix.de>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Lucas Stach <l.stach@pengutronix.de>,
devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org,
linux-staging@lists.linux.dev
Subject: [RFC 0/5] Split iMX8MQ Hantro VPU into G1 and G2 with blk-ctrl
support
Date: Sun, 5 Dec 2021 12:16:13 -0600
Message-Id: <20211205181618.1041699-1-aford173@gmail.com>
X-Mailer: git-send-email 2.32.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-media.vger.kernel.org>
X-Mailing-List: linux-media@vger.kernel.org
X-LSpam-Score: -2.5 (--)
X-LSpam-Report: No,
score=-2.5 required=5.0 tests=BAYES_00=-1.9,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FORGED_FROMDOMAIN=0.001,FREEMAIL_FROM=0.001,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1
autolearn=ham autolearn_force=no
|
| Series |
Split iMX8MQ Hantro VPU into G1 and G2 with blk-ctrl support
|
|
Message
Adam Ford
Dec. 5, 2021, 6:16 p.m. UTC
Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's two IP blocks called G1 and G2. There is initialization code in VPU code to pull some clocks, resets and other features which has been integrated into the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make the VPU codec's operate as stand-alone cores without having to know the details of each other or the quirks unique to the i.MX8MQ, so the remaining code can be left more generic. This series was started by Lucas Stach, and picked up by me so some patches have his s-o-b and mine where I might have changed a few minor items. It's in an RFC state because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't appear to show VP9 support, and it looks like it should. since the g-streamer and media trees are in a constant state of change, this series is based on git://linuxtv.org/hverkuil/media_tree.git for-v5.17e Adam Ford (2): media: hantro: split i.MX8MQ G1 and G2 code arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl Lucas Stach (3): dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++ arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++---- drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++ drivers/staging/media/hantro/hantro_drv.c | 4 +- drivers/staging/media/hantro/hantro_hw.h | 2 +- drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++--------------- include/dt-bindings/power/imx8mq-power.h | 3 + 7 files changed, 205 insertions(+), 130 deletions(-) create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
Comments
On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <aford173@gmail.com> wrote: > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's > two IP blocks called G1 and G2. There is initialization code in VPU code to > pull some clocks, resets and other features which has been integrated into > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make > the VPU codec's operate as stand-alone cores without having to know the details > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be > left more generic. > > This series was started by Lucas Stach, and picked up by me so some patches have > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't > appear to show VP9 support, and it looks like it should. > > since the g-streamer and media trees are in a constant state of change, this series is based on > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e > I forgot to post Fluster results. Before the patches to this branch: 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for GStreamer 1.0... ✔️ GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ❌ GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for GStreamer 1.0... ✔️ GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for GStreamer 1.0... ❌ GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for GStreamer 1.0... ❌ [gst-main] root@localhost:~/gstreamer/fluster# There was no VP9 support. ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 GStreamer-H.264-V4L2SL-Gst1.0 Ran 90/135 tests successfully in 58.787 secs AFTER this series, two decoders appear: [ 15.919137] hantro-vpu 38300000.video-codec: registered nxp,imx8mq-vpu-g1-dec as /dev/video0 [ 15.983579] hantro-vpu 38310000.video-codec: registered nxp,imx8mq-vpu-g2-dec as /dev/video1 VP9 was listed: GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for GStreamer 1.0... ✔️ GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ❌ GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for GStreamer 1.0... ✔️ GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for GStreamer 1.0... ✔️ GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for GStreamer 1.0... ❌ ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0 Ran 55/61 tests successfully in 8.565 secs ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 Ran 90/135 tests successfully in 60.269 secs Same results for H.264 VP9 Hangs, where it didn't even appear as available before: ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 **************************************************************************************************** Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0 Using 4 parallel job(s) **************************************************************************************************** [TEST SUITE ] (DECODER ) TEST VECTOR ... RESULT ---------------------------------------------------------------------- [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) vp90-2-00-quantizer-00.webm ... Success [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) vp90-2-00-quantizer-01.webm ... Success [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) vp90-2-00-quantizer-02.webm ... Succes <hang > - and yes, 'Success' didnt' finish writing to the serial port. > Adam Ford (2): > media: hantro: split i.MX8MQ G1 and G2 code > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl > > Lucas Stach (3): > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++ > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++---- > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++ > drivers/staging/media/hantro/hantro_drv.c | 4 +- > drivers/staging/media/hantro/hantro_hw.h | 2 +- > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++--------------- > include/dt-bindings/power/imx8mq-power.h | 3 + > 7 files changed, 205 insertions(+), 130 deletions(-) > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml > > -- > 2.32.0 >
Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit : > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <aford173@gmail.com> wrote: > > > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's > > two IP blocks called G1 and G2. There is initialization code in VPU code to > > pull some clocks, resets and other features which has been integrated into > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make > > the VPU codec's operate as stand-alone cores without having to know the details > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be > > left more generic. > > > > This series was started by Lucas Stach, and picked up by me so some patches have > > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't > > appear to show VP9 support, and it looks like it should. > > > > since the g-streamer and media trees are in a constant state of change, this series is based on > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e > > > > I forgot to post Fluster results. > > Before the patches to this branch: > > 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > GStreamer 1.0... ❌ > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > GStreamer 1.0... ✔️ > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > GStreamer 1.0... ❌ > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > GStreamer 1.0... ❌ > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > GStreamer 1.0... ✔️ > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > GStreamer 1.0... ❌ > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > GStreamer 1.0... ❌ > [gst-main] root@localhost:~/gstreamer/fluster# > > There was no VP9 support. > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > GStreamer-H.264-V4L2SL-Gst1.0 > Ran 90/135 tests successfully in 58.787 secs > > AFTER this series, two decoders appear: > > [ 15.919137] hantro-vpu 38300000.video-codec: registered > nxp,imx8mq-vpu-g1-dec as /dev/video0 > [ 15.983579] hantro-vpu 38310000.video-codec: registered > nxp,imx8mq-vpu-g2-dec as /dev/video1 > > VP9 was listed: > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > GStreamer 1.0... ❌ > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > GStreamer 1.0... ✔️ > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > GStreamer 1.0... ❌ > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > GStreamer 1.0... ❌ > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > GStreamer 1.0... ✔️ > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > GStreamer 1.0... ✔️ > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > GStreamer 1.0... ❌ > > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0 > Ran 55/61 tests successfully in 8.565 secs > > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > Ran 90/135 tests successfully in 60.269 secs > > Same results for H.264 > > VP9 Hangs, where it didn't even appear as available before: > > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 > **************************************************************************************************** > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0 > Using 4 parallel job(s) > **************************************************************************************************** > > [TEST SUITE ] (DECODER ) TEST VECTOR > ... RESULT > ---------------------------------------------------------------------- > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > vp90-2-00-quantizer-00.webm ... Success > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > vp90-2-00-quantizer-01.webm ... Success > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > vp90-2-00-quantizer-02.webm ... Succes > > <hang > - and yes, 'Success' didnt' finish writing to the serial port. Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ? > > > > > Adam Ford (2): > > media: hantro: split i.MX8MQ G1 and G2 code > > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl > > > > Lucas Stach (3): > > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains > > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl > > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl > > > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++ > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++---- > > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++ > > drivers/staging/media/hantro/hantro_drv.c | 4 +- > > drivers/staging/media/hantro/hantro_hw.h | 2 +- > > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++--------------- > > include/dt-bindings/power/imx8mq-power.h | 3 + > > 7 files changed, 205 insertions(+), 130 deletions(-) > > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml > > > > -- > > 2.32.0 > >
On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote: > > Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit : > > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <aford173@gmail.com> wrote: > > > > > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's > > > two IP blocks called G1 and G2. There is initialization code in VPU code to > > > pull some clocks, resets and other features which has been integrated into > > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make > > > the VPU codec's operate as stand-alone cores without having to know the details > > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be > > > left more generic. > > > > > > This series was started by Lucas Stach, and picked up by me so some patches have > > > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state > > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't > > > appear to show VP9 support, and it looks like it should. > > > > > > since the g-streamer and media trees are in a constant state of change, this series is based on > > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e > > > > > > > I forgot to post Fluster results. > > > > Before the patches to this branch: > > > > 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l > > > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > > GStreamer 1.0... ❌ > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > > GStreamer 1.0... ✔️ > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > > GStreamer 1.0... ❌ > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > > GStreamer 1.0... ❌ > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > > GStreamer 1.0... ✔️ > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > > GStreamer 1.0... ❌ > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > > GStreamer 1.0... ❌ > > [gst-main] root@localhost:~/gstreamer/fluster# > > > > There was no VP9 support. > > > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > > GStreamer-H.264-V4L2SL-Gst1.0 > > Ran 90/135 tests successfully in 58.787 secs > > > > AFTER this series, two decoders appear: > > > > [ 15.919137] hantro-vpu 38300000.video-codec: registered > > nxp,imx8mq-vpu-g1-dec as /dev/video0 > > [ 15.983579] hantro-vpu 38310000.video-codec: registered > > nxp,imx8mq-vpu-g2-dec as /dev/video1 > > > > VP9 was listed: > > > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > > GStreamer 1.0... ❌ > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > > GStreamer 1.0... ✔️ > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > > GStreamer 1.0... ❌ > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > > GStreamer 1.0... ❌ > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > > GStreamer 1.0... ✔️ > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > > GStreamer 1.0... ✔️ > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > > GStreamer 1.0... ❌ > > > > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0 > > Ran 55/61 tests successfully in 8.565 secs > > > > > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > > Ran 90/135 tests successfully in 60.269 secs > > > > Same results for H.264 > > > > VP9 Hangs, where it didn't even appear as available before: > > > > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 > > **************************************************************************************************** > > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0 > > Using 4 parallel job(s) > > **************************************************************************************************** > > > > [TEST SUITE ] (DECODER ) TEST VECTOR > > ... RESULT > > ---------------------------------------------------------------------- > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > vp90-2-00-quantizer-00.webm ... Success > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > vp90-2-00-quantizer-01.webm ... Success > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > vp90-2-00-quantizer-02.webm ... Succes > > > > <hang > - and yes, 'Success' didnt' finish writing to the serial port. > > Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ? Anything after the first attempt appears to fail. I haven't figured out if it's the G2 code or the power domain stuff. Since VP9 didn't work I migrated it to use the vpu-blk-ctrl, I can't really compare to know if it was a regression or an improvement. :-) adam > > > > > > > > > > Adam Ford (2): > > > media: hantro: split i.MX8MQ G1 and G2 code > > > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl > > > > > > Lucas Stach (3): > > > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains > > > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl > > > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl > > > > > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++ > > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++---- > > > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++ > > > drivers/staging/media/hantro/hantro_drv.c | 4 +- > > > drivers/staging/media/hantro/hantro_hw.h | 2 +- > > > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++--------------- > > > include/dt-bindings/power/imx8mq-power.h | 3 + > > > 7 files changed, 205 insertions(+), 130 deletions(-) > > > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml > > > > > > -- > > > 2.32.0 > > > >
On Mon, Dec 6, 2021 at 4:30 PM Adam Ford <aford173@gmail.com> wrote: > > On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote: > > > > Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit : > > > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <aford173@gmail.com> wrote: > > > > > > > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's > > > > two IP blocks called G1 and G2. There is initialization code in VPU code to > > > > pull some clocks, resets and other features which has been integrated into > > > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make > > > > the VPU codec's operate as stand-alone cores without having to know the details > > > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be > > > > left more generic. > > > > > > > > This series was started by Lucas Stach, and picked up by me so some patches have > > > > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state > > > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't > > > > appear to show VP9 support, and it looks like it should. > > > > > > > > since the g-streamer and media trees are in a constant state of change, this series is based on > > > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e > > > > > > > > > > I forgot to post Fluster results. > > > > > > Before the patches to this branch: > > > > > > 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l > > > > > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > > > GStreamer 1.0... ✔️ > > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > > > GStreamer 1.0... ✔️ > > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > > > GStreamer 1.0... ❌ > > > [gst-main] root@localhost:~/gstreamer/fluster# > > > > > > There was no VP9 support. > > > > > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > > > GStreamer-H.264-V4L2SL-Gst1.0 > > > Ran 90/135 tests successfully in 58.787 secs > > > > > > AFTER this series, two decoders appear: > > > > > > [ 15.919137] hantro-vpu 38300000.video-codec: registered > > > nxp,imx8mq-vpu-g1-dec as /dev/video0 > > > [ 15.983579] hantro-vpu 38310000.video-codec: registered > > > nxp,imx8mq-vpu-g2-dec as /dev/video1 > > > > > > VP9 was listed: > > > > > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for > > > GStreamer 1.0... ✔️ > > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > > > GStreamer 1.0... ❌ > > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌ > > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for > > > GStreamer 1.0... ✔️ > > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for > > > GStreamer 1.0... ✔️ > > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for > > > GStreamer 1.0... ❌ > > > > > > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0 > > > Ran 55/61 tests successfully in 8.565 secs > > > > > > > > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0 > > > Ran 90/135 tests successfully in 60.269 secs > > > > > > Same results for H.264 > > > > > > VP9 Hangs, where it didn't even appear as available before: > > > > > > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 > > > **************************************************************************************************** > > > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0 > > > Using 4 parallel job(s) > > > **************************************************************************************************** > > > > > > [TEST SUITE ] (DECODER ) TEST VECTOR > > > ... RESULT > > > ---------------------------------------------------------------------- > > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > > vp90-2-00-quantizer-00.webm ... Success > > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > > vp90-2-00-quantizer-01.webm ... Success > > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0) > > > vp90-2-00-quantizer-02.webm ... Succes > > > > > > <hang > - and yes, 'Success' didnt' finish writing to the serial port. > > > > Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ? > > Anything after the first attempt appears to fail. I haven't figured > out if it's the G2 code or the power domain stuff. Since VP9 didn't > work I migrated it to use the vpu-blk-ctrl, I can't really compare to > know if it was a regression or an improvement. > > :-) > I managed to get the vpu-blk-ctrl updated to address the G2 hanging. ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 Ran 144/303 tests successfully in 224.847 secs Since I couldn't get it to work before the split, I am not sure if this is good or not. I also wasn't sure if I needed to allocate more CMA for this. If/when people can give me some feedback on the RFC's, I'll submit an updated formal patch for both the i.MQ8MQ and the i.MQ8M Mini as one series based on "for-v5.17e" from git://linuxtv.org/hverkuil/media_tree.git I wasn't sure if I needed to break up the migration from a combined VPU to split G1 and G2. I might need some help/suggestions on the YAML, because I struggle with that part a little, but I'd like the 8MM to use the 8MQ dt-binding yaml file since they are so similar. I hope to have some more time this week because my weekends are completely booked for the rest of the year, and leave for London, on 23 Dec 2021 and I won't return until 5 Jan 2022. > adam > > > > > > > > > > > > > > > > Adam Ford (2): > > > > media: hantro: split i.MX8MQ G1 and G2 code > > > > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl > > > > > > > > Lucas Stach (3): > > > > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains > > > > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl > > > > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl > > > > > > > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++ > > > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++---- > > > > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++ > > > > drivers/staging/media/hantro/hantro_drv.c | 4 +- > > > > drivers/staging/media/hantro/hantro_hw.h | 2 +- > > > > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++--------------- > > > > include/dt-bindings/power/imx8mq-power.h | 3 + > > > > 7 files changed, 205 insertions(+), 130 deletions(-) > > > > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml > > > > > > > > -- > > > > 2.32.0 > > > > > >