Message ID | 20220627025625.8956-1-irui.wang@mediatek.com (mailing list archive) |
---|---|
Headers |
Received: from vger.kernel.org ([23.128.96.18]) by www.linuxtv.org with esmtp (Exim 4.92) (envelope-from <linux-media-owner@vger.kernel.org>) id 1o5evZ-003ahS-E0; Mon, 27 Jun 2022 02:56:41 +0000 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232257AbiF0C4j (ORCPT <rfc822;mkrufky@linuxtv.org> + 1 other); Sun, 26 Jun 2022 22:56:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232243AbiF0C4g (ORCPT <rfc822;linux-media@vger.kernel.org>); Sun, 26 Jun 2022 22:56:36 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 890F838B7; Sun, 26 Jun 2022 19:56:31 -0700 (PDT) X-UUID: 0fd6e71bc2f640828f0c5ab297aa1f18-20220627 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.6,REQID:9725ddd3-5e3e-4c87-bb08-3fb75c7e475c,OB:0,LO B:0,IP:0,URL:0,TC:0,Content:-5,EDM:0,RT:0,SF:0,FILE:0,RULE:Release_Ham,ACT ION:release,TS:-5 X-CID-META: VersionHash:b14ad71,CLOUDID:108e90ea-f7af-4e69-92ee-0fd74a0c286c,C OID:IGNORED,Recheck:0,SF:nil,TC:nil,Content:0,EDM:-3,IP:nil,URL:1,File:nil ,QS:nil,BEC:nil,COL:0 X-UUID: 0fd6e71bc2f640828f0c5ab297aa1f18-20220627 Received: from mtkmbs11n1.mediatek.inc [(172.21.101.185)] by mailgw02.mediatek.com (envelope-from <irui.wang@mediatek.com>) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1743380513; Mon, 27 Jun 2022 10:56:28 +0800 Received: from mtkmbs07n1.mediatek.inc (172.21.101.16) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Mon, 27 Jun 2022 10:56:27 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 27 Jun 2022 10:56:26 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 27 Jun 2022 10:56:26 +0800 From: Irui Wang <irui.wang@mediatek.com> To: Hans Verkuil <hverkuil-cisco@xs4all.nl>, Mauro Carvalho Chehab <mchehab@kernel.org>, Rob Herring <robh+dt@kernel.org>, Matthias Brugger <matthias.bgg@gmail.com>, Tzung-Bi Shih <tzungbi@chromium.org>, <angelogioacchino.delregno@collabora.com>, <nicolas.dufresne@collabora.com>, <wenst@chromium.org> CC: <Project_Global_Chrome_Upstream_Group@mediatek.com>, <linux-media@vger.kernel.org>, <devicetree@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-arm-kernel@lists.infradead.org>, <linux-mediatek@lists.infradead.org>, Tomasz Figa <tfiga@chromium.org>, <xia.jiang@mediatek.com>, <maoguang.meng@mediatek.com>, kyrie wu <kyrie.wu@mediatek.com>, <srv_heupstream@mediatek.com> Subject: [V10,0/7] Enable two hardware jpeg encoder for MT8195 Date: Mon, 27 Jun 2022 10:56:18 +0800 Message-ID: <20220627025625.8956-1-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 7BIT Content-Type: text/plain; charset=US-ASCII X-MTK: N X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-media.vger.kernel.org> X-Mailing-List: linux-media@vger.kernel.org X-LSpam-Score: -2.4 (--) X-LSpam-Report: No, score=-2.4 required=5.0 tests=BAYES_00=-1.9,HEADER_FROM_DIFFERENT_DOMAINS=0.5,MAILING_LIST_MULTI=-1,UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no |
Series |
Enable two hardware jpeg encoder for MT8195
|
|
Message
Irui Wang
June 27, 2022, 2:56 a.m. UTC
From: kyrie wu <kyrie.wu@mediatek.com>
This series adds support for multi hardware jpeg encoding, by first
adding use of_platform_populate to manage each hardware information:
interrupt, clock, register bases and power. Secondly add encoding
work queue to deal with the encoding requestsof multi-hardware
at the same time. Lastly, add output picture reorder function
interface to eliminate the out of order images.
This series has been tested with MT8195 Gstreamer.
Encoding worked for this chip.
Patches 1 Adds jpeg encoder dt-bindings for mt8195
Patches 2 jpeg encoder builds two module for using Multi-HW,
export some functions to make them visible by other modules.
Patches 3 use devm_of_platform_populate to manage multi-hardware.
Patch 4 add jpeg encoding timeout function to judge hardware timeout.
Patch 5 add encoding work queue to deal with multi-hardware encoding
at the same time.
Patch 6 add output picture reorder function to order images.
Patch 7 add stop cmd function to deal with EOS operation.
---
This series patches dependent on:
media_stage tree:
[1]
https://git.linuxtv.org/media_stage.git/commit/?id=b3627647f9ea7473d10fb08a95fd7c4133a17ca4
patch1 new jpegenc dt-bindings included files
[2] MM IOMMU binding:
https://patchwork.kernel.org/project/linux-mediatek/patch/20220217113453.13658-2-yong.wu@mediatek.com/
[3] MT8195 power domain:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579
Changes compared with v9:
- some modifications for patch v9's review comments.
Changes compared with v8:
- some modifications for patch v8's review comments.
- add stop cmd function.
Changes compared with v7:
- some modifications for patch v6's review comments.
Changes compared with v6:
- new yaml file for mt8195 jpeg encoder.
- some modifications for patch v5's review comments.
Changes compared with v5:
- use of_platform_populate to replace component framework to
manage multi-hardware in patch 2.
Changes compared with v4:
- No change compaered with v4
Changes compared with v3:
- Structure patches for consistency, non-backward
compatible and do not break any existing functionality
Changes compared with v2:
- Split the last two patches into several patches
to enhance readability
- Correct some syntax errors
- Explain why the component framework is used
Changes compared with v1:
- Add jpeg encoder dt-bindings for MT8195
- Use component framework to manage jpegenc HW
- Add jpegenc output pic reorder function interface
kyrie wu (7):
dt-bindings: mediatek: Add mediatek, mt8195-jpgenc compatible
mtk-jpegenc: export jpeg encoder functions
mtk-jpegenc: manage jpegenc multi-hardware
mtk-jpegenc: add jpegenc timeout func interface
mtk-jpegenc: add jpeg encode worker interface
mtk-jpegenc: add output pic reorder interface
mtk-jpegenc: add stop cmd interface for jpgenc
.../media/mediatek,mt8195-jpegenc.yaml | 154 ++++++++++
drivers/media/platform/mediatek/jpeg/Makefile | 11 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 290 +++++++++++++++---
.../platform/mediatek/jpeg/mtk_jpeg_core.h | 82 ++++-
.../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 1 +
.../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h | 3 +-
.../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 253 +++++++++++++++
7 files changed, 743 insertions(+), 51 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
Comments
Il 27/06/22 04:56, Irui Wang ha scritto: > From: kyrie wu <kyrie.wu@mediatek.com> > > This series adds support for multi hardware jpeg encoding, by first > adding use of_platform_populate to manage each hardware information: > interrupt, clock, register bases and power. Secondly add encoding > work queue to deal with the encoding requestsof multi-hardware > at the same time. Lastly, add output picture reorder function > interface to eliminate the out of order images. > > This series has been tested with MT8195 Gstreamer. > Encoding worked for this chip. > Hello Irui, Unfortunately, I can't reproduce your successful test with gstreamer: when I try to encode a jpeg with videotestsrc input, I get a kernel panic. To reproduce this behavior, please run the following: gst-launch-1.0 videotestsrc ! v4l2jpegenc ! filesink location=hwenctest.jpg Here's the panic log: [ 342.567211] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000108 [ 342.576014] Mem abort info: [ 342.578805] ESR = 0x0000000096000004 [ 342.582552] EC = 0x25: DABT (current EL), IL = 32 bits [ 342.587863] SET = 0, FnV = 0 [ 342.590919] EA = 0, S1PTW = 0 [ 342.594062] FSC = 0x04: level 0 translation fault [ 342.598941] Data abort info: [ 342.601826] ISV = 0, ISS = 0x00000004 [ 342.605668] CM = 0, WnR = 0 [ 342.608643] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000119029000 [ 342.615094] [0000000000000108] pgd=0000000000000000, p4d=0000000000000000 [ 342.621907] Internal error: Oops: 96000004 [#1] SMP [ 342.626800] Modules linked in: cdc_ether usbnet r8152 af_alg qrtr mt7921e mt7921_common mt76_connac_lib mt76 mac80211 btusb btrtl btintel btmtk btbcm bluetooth cfg80211 mtk_vcodec_dec_hw uvcvideo snd_sof_mt8195 ecdh_generic mtk_vcodec_dec ecc mtk_adsp_common videobuf2_vmalloc snd_sof_xtensa_dsp rfkill snd_sof_of v4l2_vp9 panfrost mtk_jpeg snd_sof v4l2_h264 mtk_jpeg_enc_hw 8021q mtk_vcodec_common cros_ec_sensors gpu_sched cros_ec_lid_angle mtk_jpeg_dec_hw hid_multitouch snd_sof_utils cros_ec_sensors_core crct10dif_ce garp cros_usbpd_logger sbs_battery mrp stp llc ipmi_devintf ipmi_msghandler fuse ipv6 [ 342.680332] CPU: 3 PID: 180 Comm: kworker/u16:6 Tainted: G W 5.19.0-rc3-next-20220622+ #814 [ 342.689982] Hardware name: MediaTek Tomato (rev2) board (DT) [ 342.695637] Workqueue: mtk-jpeg mtk_jpegenc_worker [mtk_jpeg] [ 342.701402] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 342.708360] pc : mtk_jpegenc_worker+0x6c/0x3e0 [mtk_jpeg] [ 342.713763] lr : mtk_jpegenc_worker+0x64/0x3e0 [mtk_jpeg] [ 342.719166] sp : ffff8000093cbcd0 [ 342.722478] x29: ffff8000093cbcd0 x28: ffffb1f341cdb000 x27: ffff6089885d0505 [ 342.729618] x26: ffffb1f341ce1ee0 x25: ffffb1f341d0a2d0 x24: ffff60899078d000 [ 342.736758] x23: ffffb1f2e869c590 x22: ffff608990785458 x21: ffff60898a0ea120 [ 342.743898] x20: ffff60898a0ea080 x19: ffff60898a0ea080 x18: 0000000000000000 [ 342.751038] x17: 0000000000000000 x16: ffffb1f3405a58c0 x15: 0000aaaac092a3d0 [ 342.758178] x14: 0000000000000000 x13: 0000000000000c5f x12: 071c71c71c71c71c [ 342.765318] x11: 0000000000000c5f x10: 00000000000027a0 x9 : ffffb1f33f32ce9c [ 342.772458] x8 : fefefefefefefeff x7 : 0000000000000018 x6 : ffffb1f2e8699804 [ 342.779597] x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff608985d3c570 [ 342.786737] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 [ 342.793876] Call trace: [ 342.796319] mtk_jpegenc_worker+0x6c/0x3e0 [mtk_jpeg] [ 342.801376] process_one_work+0x294/0x664 [ 342.805392] worker_thread+0x7c/0x45c [ 342.809055] kthread+0x104/0x110 [ 342.812283] ret_from_fork+0x10/0x20 [ 342.815863] Code: aa1503e0 9400037e f9415662 aa0003e1 (b9410840) [ 342.821956] ---[ end trace 0000000000000000 ]--- Regards, Angelo
Il 05/07/22 14:56, AngeloGioacchino Del Regno ha scritto: > Il 27/06/22 04:56, Irui Wang ha scritto: >> From: kyrie wu <kyrie.wu@mediatek.com> >> >> This series adds support for multi hardware jpeg encoding, by first >> adding use of_platform_populate to manage each hardware information: >> interrupt, clock, register bases and power. Secondly add encoding >> work queue to deal with the encoding requestsof multi-hardware >> at the same time. Lastly, add output picture reorder function >> interface to eliminate the out of order images. >> >> This series has been tested with MT8195 Gstreamer. >> Encoding worked for this chip. >> > > Hello Irui, > > Unfortunately, I can't reproduce your successful test with gstreamer: when > I try to encode a jpeg with videotestsrc input, I get a kernel panic. > > To reproduce this behavior, please run the following: > > gst-launch-1.0 videotestsrc ! v4l2jpegenc ! filesink location=hwenctest.jpg > > Adding context: I've just discovered that this issue is due to a little overlook in the probe function, which I've pointed out in the review for the specific patch that's introducing the typo. After fixing a one-liner, there's no more kernel panic, but another issue was discovered regarding runtime PM: [ 184.309766] mtk-jpeg soc:jpgenc_master: Runtime PM usage count underflow! [ 184.317970] mtk-jpeg soc:jpgenc_master: Runtime PM usage count underflow! [ 184.326128] mtk-jpeg soc:jpgenc_master: Runtime PM usage count underflow! > Here's the panic log: > > [ 342.567211] Unable to handle kernel NULL pointer dereference at virtual address > 0000000000000108 > > [ 342.576014] Mem abort info: > > [ 342.578805] ESR = 0x0000000096000004 > > [ 342.582552] EC = 0x25: DABT (current EL), IL = 32 bits > > [ 342.587863] SET = 0, FnV = 0 > > [ 342.590919] EA = 0, S1PTW = 0 > > [ 342.594062] FSC = 0x04: level 0 translation fault > > [ 342.598941] Data abort info: > > [ 342.601826] ISV = 0, ISS = 0x00000004 > > [ 342.605668] CM = 0, WnR = 0 > > [ 342.608643] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000119029000 > > [ 342.615094] [0000000000000108] pgd=0000000000000000, p4d=0000000000000000 > > [ 342.621907] Internal error: Oops: 96000004 [#1] SMP > > [ 342.626800] Modules linked in: cdc_ether usbnet r8152 af_alg qrtr mt7921e > mt7921_common mt76_connac_lib mt76 mac80211 btusb btrtl btintel btmtk btbcm > bluetooth cfg80211 mtk_vcodec_dec_hw uvcvideo snd_sof_mt8195 ecdh_generic > mtk_vcodec_dec ecc mtk_adsp_common videobuf2_vmalloc snd_sof_xtensa_dsp rfkill > snd_sof_of v4l2_vp9 panfrost mtk_jpeg snd_sof v4l2_h264 mtk_jpeg_enc_hw 8021q > mtk_vcodec_common cros_ec_sensors gpu_sched cros_ec_lid_angle mtk_jpeg_dec_hw > hid_multitouch snd_sof_utils cros_ec_sensors_core crct10dif_ce garp > cros_usbpd_logger sbs_battery mrp stp llc ipmi_devintf ipmi_msghandler fuse ipv6 > > [ 342.680332] CPU: 3 PID: 180 Comm: kworker/u16:6 Tainted: G W > 5.19.0-rc3-next-20220622+ #814 > > [ 342.689982] Hardware name: MediaTek Tomato (rev2) board (DT) > > [ 342.695637] Workqueue: mtk-jpeg mtk_jpegenc_worker [mtk_jpeg] > > [ 342.701402] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) > > [ 342.708360] pc : mtk_jpegenc_worker+0x6c/0x3e0 [mtk_jpeg] > > [ 342.713763] lr : mtk_jpegenc_worker+0x64/0x3e0 [mtk_jpeg] > > [ 342.719166] sp : ffff8000093cbcd0 > > [ 342.722478] x29: ffff8000093cbcd0 x28: ffffb1f341cdb000 x27: ffff6089885d0505 > > [ 342.729618] x26: ffffb1f341ce1ee0 x25: ffffb1f341d0a2d0 x24: ffff60899078d000 > > [ 342.736758] x23: ffffb1f2e869c590 x22: ffff608990785458 x21: ffff60898a0ea120 > > [ 342.743898] x20: ffff60898a0ea080 x19: ffff60898a0ea080 x18: 0000000000000000 > > [ 342.751038] x17: 0000000000000000 x16: ffffb1f3405a58c0 x15: 0000aaaac092a3d0 > > [ 342.758178] x14: 0000000000000000 x13: 0000000000000c5f x12: 071c71c71c71c71c > > [ 342.765318] x11: 0000000000000c5f x10: 00000000000027a0 x9 : ffffb1f33f32ce9c > > [ 342.772458] x8 : fefefefefefefeff x7 : 0000000000000018 x6 : ffffb1f2e8699804 > > [ 342.779597] x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff608985d3c570 > > [ 342.786737] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 > > [ 342.793876] Call trace: > > [ 342.796319] mtk_jpegenc_worker+0x6c/0x3e0 [mtk_jpeg] > > [ 342.801376] process_one_work+0x294/0x664 > > [ 342.805392] worker_thread+0x7c/0x45c > > [ 342.809055] kthread+0x104/0x110 > > [ 342.812283] ret_from_fork+0x10/0x20 > > [ 342.815863] Code: aa1503e0 9400037e f9415662 aa0003e1 (b9410840) > > [ 342.821956] ---[ end trace 0000000000000000 ]--- > > > Regards, > Angelo