[v2,6/7] media: renesas: vsp1: Implement .link_validate() for video devices

  The v4l2_subdev_link_validate() helper prints a warning if the
.link_validate() operation is not implemented for video devices
connected to the subdevs. Implement the operation to silence the
warning.

Ideally validation of the link between the video device and the subdev
should be implemented in that operation. That would however break
userspace that does not configure formats on all video devices before
starting streaming. While this mode of operation may not be considered
valid by the V4L2 API specification (interpretation differ), it is
nonetheless supported by the vsp1 driver at the moment and used by at
least the vsp1 unit test suite, and possibly other userspace
applciations. Removing it would be a regression.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 .../media/platform/renesas/vsp1/vsp1_video.c  | 22 +++++++++++++++++++
 1 file changed, 22 insertions(+)
  

Hi,

On 22/08/2024 18:45, Laurent Pinchart wrote:
> The v4l2_subdev_link_validate() helper prints a warning if the
> .link_validate() operation is not implemented for video devices
> connected to the subdevs. Implement the operation to silence the
> warning.
> 
> Ideally validation of the link between the video device and the subdev
> should be implemented in that operation. That would however break
> userspace that does not configure formats on all video devices before
> starting streaming. While this mode of operation may not be considered
> valid by the V4L2 API specification (interpretation differ), it is
> nonetheless supported by the vsp1 driver at the moment and used by at
> least the vsp1 unit test suite, and possibly other userspace
> applciations. Removing it would be a regression.

"applications"

If the media graph is validated when the first stream is enabled, does 
that mean that when the graph is "enabled", we can never change e.g. the 
resolution, even for streams that have not been enabled and even if all 
the drivers would support this?

The pad interdependency should help there, right? Would it help here, too?

  Tomi

> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   .../media/platform/renesas/vsp1/vsp1_video.c  | 22 +++++++++++++++++++
>   1 file changed, 22 insertions(+)
> 
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> index fdb46ec0c872..e728f9f5160e 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> @@ -1081,6 +1081,27 @@ static const struct v4l2_file_operations vsp1_video_fops = {
>   	.mmap = vb2_fop_mmap,
>   };
>   
> +/* -----------------------------------------------------------------------------
> + * Media entity operations
> + */
> +
> +static int vsp1_video_link_validate(struct media_link *link)
> +{
> +	/*
> +	 * Ideally, link validation should be implemented here instead of
> +	 * calling vsp1_video_verify_format() in vsp1_video_streamon()
> +	 * manually. That would however break userspace that start one video
> +	 * device before configures formats on other video devices in the
> +	 * pipeline. This operation is just a no-op to silence the warnings
> +	 * from v4l2_subdev_link_validate().
> +	 */
> +	return 0;
> +}
> +
> +static const struct media_entity_operations vsp1_video_media_ops = {
> +	.link_validate = vsp1_video_link_validate,
> +};
> +
>   /* -----------------------------------------------------------------------------
>    * Suspend and Resume
>    */
> @@ -1215,6 +1236,7 @@ struct vsp1_video *vsp1_video_create(struct vsp1_device *vsp1,
>   
>   	/* ... and the video node... */
>   	video->video.v4l2_dev = &video->vsp1->v4l2_dev;
> +	video->video.entity.ops = &vsp1_video_media_ops;
>   	video->video.fops = &vsp1_video_fops;
>   	snprintf(video->video.name, sizeof(video->video.name), "%s %s",
>   		 rwpf->entity.subdev.name, direction);
  

Hi Tomi,

On Mon, Aug 26, 2024 at 02:49:44PM +0300, Tomi Valkeinen wrote:
> On 22/08/2024 18:45, Laurent Pinchart wrote:
> > The v4l2_subdev_link_validate() helper prints a warning if the
> > .link_validate() operation is not implemented for video devices
> > connected to the subdevs. Implement the operation to silence the
> > warning.
> > 
> > Ideally validation of the link between the video device and the subdev
> > should be implemented in that operation. That would however break
> > userspace that does not configure formats on all video devices before
> > starting streaming. While this mode of operation may not be considered
> > valid by the V4L2 API specification (interpretation differ), it is
> > nonetheless supported by the vsp1 driver at the moment and used by at
> > least the vsp1 unit test suite, and possibly other userspace
> > applciations. Removing it would be a regression.
> 
> "applications"
> 
> If the media graph is validated when the first stream is enabled, does 
> that mean that when the graph is "enabled", we can never change e.g. the 
> resolution, even for streams that have not been enabled and even if all 
> the drivers would support this?

In the general case, yes. Drivers can already support this use case by
implementing their own link validation. Improvements to generic helpers
could also be possible in the future.

> The pad interdependency should help there, right? Would it help here, too?

Possibly, yes.

> > Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> >   .../media/platform/renesas/vsp1/vsp1_video.c  | 22 +++++++++++++++++++
> >   1 file changed, 22 insertions(+)
> > 
> > diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> > index fdb46ec0c872..e728f9f5160e 100644
> > --- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
> > +++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> > @@ -1081,6 +1081,27 @@ static const struct v4l2_file_operations vsp1_video_fops = {
> >   	.mmap = vb2_fop_mmap,
> >   };
> >   
> > +/* -----------------------------------------------------------------------------
> > + * Media entity operations
> > + */
> > +
> > +static int vsp1_video_link_validate(struct media_link *link)
> > +{
> > +	/*
> > +	 * Ideally, link validation should be implemented here instead of
> > +	 * calling vsp1_video_verify_format() in vsp1_video_streamon()
> > +	 * manually. That would however break userspace that start one video
> > +	 * device before configures formats on other video devices in the
> > +	 * pipeline. This operation is just a no-op to silence the warnings
> > +	 * from v4l2_subdev_link_validate().
> > +	 */
> > +	return 0;
> > +}
> > +
> > +static const struct media_entity_operations vsp1_video_media_ops = {
> > +	.link_validate = vsp1_video_link_validate,
> > +};
> > +
> >   /* -----------------------------------------------------------------------------
> >    * Suspend and Resume
> >    */
> > @@ -1215,6 +1236,7 @@ struct vsp1_video *vsp1_video_create(struct vsp1_device *vsp1,
> >   
> >   	/* ... and the video node... */
> >   	video->video.v4l2_dev = &video->vsp1->v4l2_dev;
> > +	video->video.entity.ops = &vsp1_video_media_ops;
> >   	video->video.fops = &vsp1_video_fops;
> >   	snprintf(video->video.name, sizeof(video->video.name), "%s %s",
> >   		 rwpf->entity.subdev.name, direction);