Message ID | 20170312202240.GT21222@n2100.armlinux.org.uk (mailing list archive) |
---|---|
State | Superseded, archived |
Headers |
Received: from mail.tu-berlin.de ([130.149.7.33]) by www.linuxtv.org with esmtp (Exim 4.84_2) (envelope-from <linux-media-owner@vger.kernel.org>) id 1cnA1n-0003vT-O4; Sun, 12 Mar 2017 20:23:43 +0000 X-tubIT-Incoming-IP: 209.132.180.67 Received: from vger.kernel.org ([209.132.180.67]) by mail.tu-berlin.de (exim-4.84_2/mailfrontend-8) with esmtp id 1cnA1l-0000uG-kX; Sun, 12 Mar 2017 21:23:43 +0100 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935327AbdCLUXh (ORCPT <rfc822;mkrufky@linuxtv.org> + 1 other); Sun, 12 Mar 2017 16:23:37 -0400 Received: from pandora.armlinux.org.uk ([78.32.30.218]:46768 "EHLO pandora.armlinux.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934692AbdCLUXg (ORCPT <rfc822;linux-media@vger.kernel.org>); Sun, 12 Mar 2017 16:23:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2014; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=I/UtYiy77GjxDETcLw4rvXPtSl9vmBRRqg8O+35Zqc0=; b=l4UWBAy7uZveaVMJAW6Bz4QanmiKCvyj18/KjxZUG9FRVEni5AzA/H5FuP+dRH+oWYNOjVo65zetvj7Ywyy1e1UMiJAxR7neJ/sGFDT06dNDwjJSfxuiRzigGWREBpN22DnoQly3GeJa7Iq9tUXtNl++b4lzrMZXKThoVBuuJqs=; Received: from n2100.armlinux.org.uk ([fd8f:7570:feb6:1:214:fdff:fe10:4f86]:39307) by pandora.armlinux.org.uk with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from <linux@armlinux.org.uk>) id 1cnA0r-00080X-PP; Sun, 12 Mar 2017 20:22:46 +0000 Received: from linux by n2100.armlinux.org.uk with local (Exim 4.76) (envelope-from <linux@n2100.armlinux.org.uk>) id 1cnA0n-0003Ym-Jj; Sun, 12 Mar 2017 20:22:41 +0000 Date: Sun, 12 Mar 2017 20:22:41 +0000 From: Russell King - ARM Linux <linux@armlinux.org.uk> To: Steve Longerbeam <slongerbeam@gmail.com> Cc: robh+dt@kernel.org, mark.rutland@arm.com, shawnguo@kernel.org, kernel@pengutronix.de, fabio.estevam@nxp.com, mchehab@kernel.org, hverkuil@xs4all.nl, nick@shmanahar.org, markus.heiser@darmarIT.de, p.zabel@pengutronix.de, laurent.pinchart+renesas@ideasonboard.com, bparrot@ti.com, geert@linux-m68k.org, arnd@arndb.de, sudipm.mukherjee@gmail.com, minghsiu.tsai@mediatek.com, tiffany.lin@mediatek.com, jean-christophe.trotin@st.com, horms+renesas@verge.net.au, niklas.soderlund+renesas@ragnatech.se, robert.jarzmik@free.fr, songjun.wu@microchip.com, andrew-ct.chen@mediatek.com, gregkh@linuxfoundation.org, shuah@kernel.org, sakari.ailus@linux.intel.com, pavel@ucw.cz, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-media@vger.kernel.org, devel@driverdev.osuosl.org, Steve Longerbeam <steve_longerbeam@mentor.com> Subject: Re: [PATCH v5 00/39] i.MX Media Driver Message-ID: <20170312202240.GT21222@n2100.armlinux.org.uk> References: <1489121599-23206-1-git-send-email-steve_longerbeam@mentor.com> <20170310201356.GA21222@n2100.armlinux.org.uk> <47542ef8-3e91-b4cd-cc65-95000105f172@gmail.com> <20170312195741.GS21222@n2100.armlinux.org.uk> <ea3ccdb8-903f-93ab-6875-90da440fc52a@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <ea3ccdb8-903f-93ab-6875-90da440fc52a@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: <linux-media.vger.kernel.org> X-Mailing-List: linux-media@vger.kernel.org X-PMX-Version: 6.0.0.2142326, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2017.3.12.201816 X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report=' MULTIPLE_RCPTS 0.1, HTML_00_01 0.05, HTML_00_10 0.05, BODY_SIZE_3000_3999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, DKIM_SIGNATURE 0, FROM_NAME_PHRASE 0, IN_REP_TO 0, LEGITIMATE_SIGNS 0, MSG_THREAD 0, MULTIPLE_REAL_RCPTS 0, NO_URI_HTTPS 0, REFERENCES 0, SINGLE_URI_IN_BODY 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CC_NAME 0, __CC_NAME_DIFF_FROM_ACC 0, __CC_REAL_NAMES 0, __CD 0, __CP_MEDIA_BODY 0, __CP_URI_IN_BODY 0, __CT 0, __CT_TEXT_PLAIN 0, __FORWARDED_MSG 0, __HAS_CC_HDR 0, __HAS_FROM 0, __HAS_LIST_ID 0, __HAS_MSGID 0, __HAS_X_MAILING_LIST 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0, __MIME_VERSION 0, __MULTIPLE_RCPTS_CC_X2 0, __NO_HTML_TAG_RAW 0, __REFERENCES 0, __SANE_MSGID 0, __SINGLE_URI_TEXT 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NAME 0, __TO_NAME_DIFF_FROM_ACC 0, __TO_REAL_NAMES 0, __URI_IN_BODY 0, __URI_NO_MAILTO 0, __URI_NS , __URI_WITH_PATH 0, __USER_AGENT 0' |
Commit Message
Russell King (Oracle)
March 12, 2017, 8:22 p.m. UTC
On Sun, Mar 12, 2017 at 01:05:06PM -0700, Steve Longerbeam wrote: > > > On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote: > >On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote: > >>If it's too difficult to get the imx219 csi-2 transmitter into the > >>LP-11 state on power on, perhaps the csi-2 receiver can be a little > >>more lenient on the transmitter and make the LP-11 timeout a warning > >>instead of error-out. > >> > >>Can you try the attached change on top of the version 5 patchset? > >> > >>If that doesn't work then you're just going to have to fix the bug > >>in imx219. > > > >That patch gets me past that hurdle, only to reveal that there's another > >issue: > > Yeah, ipu_cpmem_set_image() failed because it doesn't recognize the > bayer formats. Wait, didn't we fix this already? I've lost track. > Ah, right, we were going to move this support into the IPUv3 driver, > but in the meantime I think you had some patches to get around this. What I had was this patch for your v3. I never got to testing your v4 because of the LP-11 problem. In v5, you've changed to propagate the ipu_cpmem_set_image() error code to avoid the resulting corruption, but that leaves the other bits of this patch unaddressed, along my "media: imx: smfc: add support for bayer formats" patch. Your driver basically has no support for bayer formats.
Comments
On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote: > On Sun, Mar 12, 2017 at 01:05:06PM -0700, Steve Longerbeam wrote: >> >> >> On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote: >>> On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote: >>>> If it's too difficult to get the imx219 csi-2 transmitter into the >>>> LP-11 state on power on, perhaps the csi-2 receiver can be a little >>>> more lenient on the transmitter and make the LP-11 timeout a warning >>>> instead of error-out. >>>> >>>> Can you try the attached change on top of the version 5 patchset? >>>> >>>> If that doesn't work then you're just going to have to fix the bug >>>> in imx219. >>> >>> That patch gets me past that hurdle, only to reveal that there's another >>> issue: >> >> Yeah, ipu_cpmem_set_image() failed because it doesn't recognize the >> bayer formats. Wait, didn't we fix this already? I've lost track. >> Ah, right, we were going to move this support into the IPUv3 driver, >> but in the meantime I think you had some patches to get around this. > > What I had was this patch for your v3. I never got to testing your > v4 because of the LP-11 problem. > > In v5, you've changed to propagate the ipu_cpmem_set_image() error > code to avoid the resulting corruption, but that leaves the other bits > of this patch unaddressed, along my "media: imx: smfc: add support > for bayer formats" patch. > > Your driver basically has no support for bayer formats. You added the patches to this driver that adds the bayer support, I don't think there is anything more required of the driver at this point to support bayer, the remaining work needs to happen in the IPUv3 driver. I'll see if I have time to write that patch to IPUv3, but it's simple, in fact what you wrote below can be translate directly into ipu_cpmem_set_image(). There's a few other places bayer needs to be treated in IPUv3, but it should be obvious by grepping for the reference to pixel formats. Steve > > diff --git a/drivers/staging/media/imx/imx-smfc.c b/drivers/staging/media/imx/imx-smfc.c > index 313732201a52..4351c0365cf4 100644 > --- a/drivers/staging/media/imx/imx-smfc.c > +++ b/drivers/staging/media/imx/imx-smfc.c > @@ -234,11 +234,6 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) > buf1 = imx_media_dma_buf_get_next_queued(priv->out_ring); > priv->next = buf1; > > - image.phys0 = buf0->phys; > - image.phys1 = buf1->phys; > - ipu_cpmem_set_image(priv->smfc_ch, &image); > - > - > switch (image.pix.pixelformat) { > case V4L2_PIX_FMT_SBGGR8: > case V4L2_PIX_FMT_SGBRG8: > @@ -247,6 +242,10 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) > burst_size = 8; > passthrough = true; > passthrough_bits = 8; > + ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height); > + ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline); > + ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys); > + ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys); > break; > > case V4L2_PIX_FMT_SBGGR16: > @@ -256,9 +255,17 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) > burst_size = 4; > passthrough = true; > passthrough_bits = 16; > + ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height); > + ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline); > + ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys); > + ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys); > break; > > default: > + image.phys0 = buf0->phys; > + image.phys1 = buf1->phys; > + ipu_cpmem_set_image(priv->smfc_ch, &image); > + > burst_size = (outfmt->width & 0xf) ? 8 : 16; > > /* >
On Sun, Mar 12, 2017 at 09:26:41PM -0700, Steve Longerbeam wrote: > On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote: > >What I had was this patch for your v3. I never got to testing your > >v4 because of the LP-11 problem. > > > >In v5, you've changed to propagate the ipu_cpmem_set_image() error > >code to avoid the resulting corruption, but that leaves the other bits > >of this patch unaddressed, along my "media: imx: smfc: add support > >for bayer formats" patch. > > > >Your driver basically has no support for bayer formats. > > You added the patches to this driver that adds the bayer support, > I don't think there is anything more required of the driver at this > point to support bayer, the remaining work needs to happen in the IPUv3 > driver. There is more work, because the way you've merged my changes to imx_smfc_setup_channel() into csi_idmac_setup_channel() is wrong with respect to the burst size. You always set it to 8 or 16 depending on the width: burst_size = (image.pix.width & 0xf) ? 8 : 16; ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size); and then you have my switch() statement which assigns burst_size. My _tested_ code removed the above, added the switch, which had a default case which reflected the above setting: default: burst_size = (outfmt->width & 0xf) ? 8 : 16; and then went on to set the burst size _after_ the switch statement: ipu_cpmem_set_burstsize(priv->smfc_ch, burst_size); The effect is unchanged for non-bayer formats. For bayer formats, the burst size is determined by the bayer data size. So, even if it's appropriate to fix ipu_cpmem_set_image(), fixing the above is still required. I'm not convinced that fixing ipu_cpmem_set_image() is even the best solution - it's not as trivial as it looks on the surface: ipu_cpmem_set_resolution(ch, image->rect.width, image->rect.height); ipu_cpmem_set_stride(ch, pix->bytesperline); this is fine, it doesn't depend on the format. However, the next line: ipu_cpmem_set_fmt(ch, v4l2_pix_fmt_to_drm_fourcc(pix->pixelformat)); does - v4l2_pix_fmt_to_drm_fourcc() is a locally defined function (it isn't v4l2 code) that converts a v4l2 pixel format to a DRM fourcc. DRM knows nothing about bayer formats, there aren't fourcc codes in DRM for it. The result is that v4l2_pix_fmt_to_drm_fourcc() returns -EINVAL cast to a u32, which gets passed unchecked into ipu_cpmem_set_fmt(). ipu_cpmem_set_fmt() won't recognise that, and also returns -EINVAL - and it's a bug that this is not checked and propagated. If it is checked and propagated, then we need this to support bayer formats, and I don't see DRM people wanting bayer format fourcc codes added without there being a real DRM driver wanting to use them. Then there's the business of calculating the top-left offset of the image, which for bayer always needs to be an even number of pixels - as this function takes the top-left offset, it ought to respect it, but if it doesn't meet this criteria, what should it do? csi_idmac_setup_channel() always sets them to zero, but that's not really something that ipu_cpmem_set_image() should assume.
On 03/13/2017 01:16 AM, Russell King - ARM Linux wrote: > On Sun, Mar 12, 2017 at 09:26:41PM -0700, Steve Longerbeam wrote: >> On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote: >>> What I had was this patch for your v3. I never got to testing your >>> v4 because of the LP-11 problem. >>> >>> In v5, you've changed to propagate the ipu_cpmem_set_image() error >>> code to avoid the resulting corruption, but that leaves the other bits >>> of this patch unaddressed, along my "media: imx: smfc: add support >>> for bayer formats" patch. >>> >>> Your driver basically has no support for bayer formats. >> You added the patches to this driver that adds the bayer support, >> I don't think there is anything more required of the driver at this >> point to support bayer, the remaining work needs to happen in the IPUv3 >> driver. > There is more work, because the way you've merged my changes to > imx_smfc_setup_channel() into csi_idmac_setup_channel() is wrong with > respect to the burst size. > > You always set it to 8 or 16 depending on the width: > > burst_size = (image.pix.width & 0xf) ? 8 : 16; > > ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size); > > and then you have my switch() statement which assigns burst_size. > My _tested_ code removed the above, added the switch, which had > a default case which reflected the above setting: > > default: > burst_size = (outfmt->width & 0xf) ? 8 : 16; > > and then went on to set the burst size _after_ the switch statement: > > ipu_cpmem_set_burstsize(priv->smfc_ch, burst_size); > > The effect is unchanged for non-bayer formats. For bayer formats, the > burst size is determined by the bayer data size. > > So, even if it's appropriate to fix ipu_cpmem_set_image(), fixing the > above is still required. Oops, sorry missed that. I'll fix. > > I'm not convinced that fixing ipu_cpmem_set_image() is even the best > solution - it's not as trivial as it looks on the surface: > > ipu_cpmem_set_resolution(ch, image->rect.width, image->rect.height); > ipu_cpmem_set_stride(ch, pix->bytesperline); > > this is fine, it doesn't depend on the format. However, the next line: > > ipu_cpmem_set_fmt(ch, v4l2_pix_fmt_to_drm_fourcc(pix->pixelformat)); > > does - v4l2_pix_fmt_to_drm_fourcc() is a locally defined function (it > isn't v4l2 code) that converts a v4l2 pixel format to a DRM fourcc. > DRM knows nothing about bayer formats, there aren't fourcc codes in > DRM for it. right, yeah that's a problem. > The result is that v4l2_pix_fmt_to_drm_fourcc() returns > -EINVAL cast to a u32, which gets passed unchecked into ipu_cpmem_set_fmt(). Ugh. > > ipu_cpmem_set_fmt() won't recognise that, and also returns -EINVAL - and > it's a bug that this is not checked and propagated. If it is checked and > propagated, then we need this to support bayer formats, and I don't see > DRM people wanting bayer format fourcc codes added without there being > a real DRM driver wanting to use them. true. > > Then there's the business of calculating the top-left offset of the image, > which for bayer always needs to be an even number of pixels - as this > function takes the top-left offset, it ought to respect it, but if it > doesn't meet this criteria, what should it do? csi_idmac_setup_channel() > always sets them to zero, but that's not really something that > ipu_cpmem_set_image() should assume. Well, I will integrate your patch above. Thanks for doing this work for me. We do need to address the issues you brought up in ipu_cpmem at some point. Steve
diff --git a/drivers/staging/media/imx/imx-smfc.c b/drivers/staging/media/imx/imx-smfc.c index 313732201a52..4351c0365cf4 100644 --- a/drivers/staging/media/imx/imx-smfc.c +++ b/drivers/staging/media/imx/imx-smfc.c @@ -234,11 +234,6 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) buf1 = imx_media_dma_buf_get_next_queued(priv->out_ring); priv->next = buf1; - image.phys0 = buf0->phys; - image.phys1 = buf1->phys; - ipu_cpmem_set_image(priv->smfc_ch, &image); - - switch (image.pix.pixelformat) { case V4L2_PIX_FMT_SBGGR8: case V4L2_PIX_FMT_SGBRG8: @@ -247,6 +242,10 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) burst_size = 8; passthrough = true; passthrough_bits = 8; + ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height); + ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline); + ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys); + ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys); break; case V4L2_PIX_FMT_SBGGR16: @@ -256,9 +255,17 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv) burst_size = 4; passthrough = true; passthrough_bits = 16; + ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height); + ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline); + ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys); + ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys); break; default: + image.phys0 = buf0->phys; + image.phys1 = buf1->phys; + ipu_cpmem_set_image(priv->smfc_ch, &image); + burst_size = (outfmt->width & 0xf) ? 8 : 16; /*