[0/1] virtio-media: Add device specification

  Hi,

This a formal attempt of including virtio-media
device specification.

Virtio-media came from a discussion on virtio-dev
mailing list, which lead to presenting virtio-v4l2[1]
specification as an alternative to virtio-video.

Later, virtio-v4l2 was renamed to virtio-media[2]
and published through:

https://github.com/chromeos/virtio-media

The repository above includes a guest V4L2 driver able
to pass v4l2-compliance when proxying the vivid/vicodec
virtual devices or an actual UVC camera using the
crosvm V4L2 proxy device. Steps to reproduce are
also detailed[3].

There is some overlap with virtio-video in regards
to which devices it can handle. However,
as virtio-media will likely be the virtualization
solution for ChromeOS (already landed into the chromeos
organization) and possibly other Google projects for
media devices, it would be desirable to include the
specification in the next virtio release despite
the aforementioned overlap.

The device ID in this document differs from
the ID in the virtio-media project repository.
And it will probably need some discussion on which
would be the correct definitive ID.

Full PDF: https://drive.google.com/file/d/1PG1YxzbSvQHPphFhbUKyKKdvuwO6shyi/view?usp=sharing
PDF with the media section only: https://drive.google.com/file/d/1Y7kAGzlUfl30VIUx9wQtz5sFTkWnBiyA/view?usp=sharing

[1] https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0
[2] https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html
[3] https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md

Albert Esteve (1):
  virtio-media: Add virtio media device specification

 conformance.tex                           |  13 +-
 content.tex                               |   1 +
 device-types/media/description.tex        | 574 ++++++++++++++++++++++
 device-types/media/device-conformance.tex |  11 +
 device-types/media/driver-conformance.tex |   9 +
 5 files changed, 604 insertions(+), 4 deletions(-)
 create mode 100644 device-types/media/description.tex
 create mode 100644 device-types/media/device-conformance.tex
 create mode 100644 device-types/media/driver-conformance.tex
  

+CC Matti

Hi Albert,

On 10.05.24 15:43, Albert Esteve wrote:
> Hi,
> 
> This a formal attempt of including virtio-media
> device specification.
> 
> Virtio-media came from a discussion on virtio-dev
> mailing list, which lead to presenting virtio-v4l2[1]
> specification as an alternative to virtio-video.
> 
> Later, virtio-v4l2 was renamed to virtio-media[2]
> and published through:
> 
> https://github.com/chromeos/virtio-media
> 
> The repository above includes a guest V4L2 driver able
> to pass v4l2-compliance when proxying the vivid/vicodec
> virtual devices or an actual UVC camera using the
> crosvm V4L2 proxy device. Steps to reproduce are
> also detailed[3].
> 
> There is some overlap with virtio-video in regards
> to which devices it can handle. However,
> as virtio-media will likely be the virtualization
> solution for ChromeOS (already landed into the chromeos
> organization) and possibly other Google projects for
> media devices, it would be desirable to include the
> specification in the next virtio release despite
> the aforementioned overlap.

Well, last year Cornelia made it clear, that this kind of overlap is not 
desirable in the specification. After long email discussion we had a 
video call with some selected experts on June 1st discussing the 
proposed approaches. The conclusion was that virtio-video development 
should be continued, that the virtio-v4l2 use-case is valid, but the 
proposed approach is not desirable in the specification, it should be 
instead implemented with multiple device types. It was also concluded, 
that a new device ID should not be reserved. It is a pity, that there 
were no public announcement after that, so now it looks like we're going 
to have the same debate again. But I'll be happy to discuss all of this 
with Albert and the new joiners in the discussion. I believe my 
arguments are still valid.

I suggest, that at least both of us (and hopefully others) start with 
familiarizing ourselves with both virtio-video and virtio-media. I'd be 
happy to present the current state of the virtio-video spec in details 
and answer questions in a video call. I think, I'll need two weeks to 
prepare. Maybe we can even resolve the issue ourselves. I think that 
would be a good start. WDYT?

> The device ID in this document differs from
> the ID in the virtio-media project repository.
> And it will probably need some discussion on which
> would be the correct definitive ID.
> 
> Full PDF: https://drive.google.com/file/d/1PG1YxzbSvQHPphFhbUKyKKdvuwO6shyi/view?usp=sharing
> PDF with the media section only: https://drive.google.com/file/d/1Y7kAGzlUfl30VIUx9wQtz5sFTkWnBiyA/view?usp=sharing
> 
> [1] https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0
> [2] https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html
> [3] https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md
> 
> Albert Esteve (1):
>    virtio-media: Add virtio media device specification
> 
>   conformance.tex                           |  13 +-
>   content.tex                               |   1 +
>   device-types/media/description.tex        | 574 ++++++++++++++++++++++
>   device-types/media/device-conformance.tex |  11 +
>   device-types/media/driver-conformance.tex |   9 +
>   5 files changed, 604 insertions(+), 4 deletions(-)
>   create mode 100644 device-types/media/description.tex
>   create mode 100644 device-types/media/device-conformance.tex
>   create mode 100644 device-types/media/driver-conformance.tex
> 

Kind regards
  

On 13.05.24 13:18, Albert Esteve wrote:
> 
> 
> On Mon, May 13, 2024 at 11:56 AM Alexander Gordeev 
> <alexander.gordeev@opensynergy.com 
> <mailto:alexander.gordeev@opensynergy.com>> wrote:
> 
>     +CC Matti
> 
>     Hi Albert,
> 
>     On 10.05.24 15:43, Albert Esteve wrote:
>      > Hi,
>      >
>      > This a formal attempt of including virtio-media
>      > device specification.
>      >
>      > Virtio-media came from a discussion on virtio-dev
>      > mailing list, which lead to presenting virtio-v4l2[1]
>      > specification as an alternative to virtio-video.
>      >
>      > Later, virtio-v4l2 was renamed to virtio-media[2]
>      > and published through:
>      >
>      > https://github.com/chromeos/virtio-media
>     <https://github.com/chromeos/virtio-media>
>      >
>      > The repository above includes a guest V4L2 driver able
>      > to pass v4l2-compliance when proxying the vivid/vicodec
>      > virtual devices or an actual UVC camera using the
>      > crosvm V4L2 proxy device. Steps to reproduce are
>      > also detailed[3].
>      >
>      > There is some overlap with virtio-video in regards
>      > to which devices it can handle. However,
>      > as virtio-media will likely be the virtualization
>      > solution for ChromeOS (already landed into the chromeos
>      > organization) and possibly other Google projects for
>      > media devices, it would be desirable to include the
>      > specification in the next virtio release despite
>      > the aforementioned overlap.
> 
>     Well, last year Cornelia made it clear, that this kind of overlap is
>     not
>     desirable in the specification. After long email discussion we had a
>     video call with some selected experts on June 1st discussing the
>     proposed approaches. The conclusion was that virtio-video development
>     should be continued, that the virtio-v4l2 use-case is valid, but the
>     proposed approach is not desirable in the specification, it should be
>     instead implemented with multiple device types. It was also concluded,
>     that a new device ID should not be reserved. It is a pity, that there
>     were no public announcement after that, so now it looks like we're
>     going
>     to have the same debate again. But I'll be happy to discuss all of this
>     with Albert and the new joiners in the discussion. I believe my
>     arguments are still valid.
> 
> 
> Hi Alexander,
> 
> My intention is not to re-open the debate. The debate already happened
> and the points given were clear enough. I will certainly noy be able to
> enrich the debate with my own points. Sadly, we did not get a clear
> conclusion, nor an agreement, on the way to move forward.
> Instead, we ended up with two different specifications, one
> that is still working towards standarization, one that did not.
> 
> I completely agree with Cornelia, the overlap is not desirable. I
> really wish there had been an agreement and moved together
> towards a common goal. But alas, that did not happen. And I
> think that having a non-standard virtual device used in some
> systems is the worst case, and hence decided to attempt
> standardization.

Thanks for the clarification!
Yeah, I also wish we could move together to a common goal. But it looks 
like we have different sets of requirements and therefore quite 
different short term goals. I hope virtio-video can become suitable for 
all of us in the long term, if there is also the virtio-camera, so that 
the compatibility is much less a problem.

>     I suggest, that at least both of us (and hopefully others) start with
>     familiarizing ourselves with both virtio-video and virtio-media. I'd be
>     happy to present the current state of the virtio-video spec in details
>     and answer questions in a video call. I think, I'll need two weeks to
>     prepare. Maybe we can even resolve the issue ourselves. I think that
>     would be a good start. WDYT?
> 
> 
> I am more or less familiar with virtio-video (not so much in the latest
> drops, but I still checked them). But I do not own virtio-media, or have
> any intellectual involvement in its design. I merely followed its
> progress, and decided to collect the specification written at
> https://github.com/chromeos/virtio-media 
> <https://github.com/chromeos/virtio-media>.
> 
> So I am in no position to solve the issue. But I decided to try
> and alleviate its impact by having both in the standard.
> 
> That said, I will read your next virtio-video drop and try to
> provide feedback :)

Thanks, looking forward to your feedback. :)

> BR,
> Albert
> 
> 
>      > The device ID in this document differs from
>      > the ID in the virtio-media project repository.
>      > And it will probably need some discussion on which
>      > would be the correct definitive ID.
>      >
>      > Full PDF:
>     https://drive.google.com/file/d/1PG1YxzbSvQHPphFhbUKyKKdvuwO6shyi/view?usp=sharing <https://drive.google.com/file/d/1PG1YxzbSvQHPphFhbUKyKKdvuwO6shyi/view?usp=sharing>
>      > PDF with the media section only:
>     https://drive.google.com/file/d/1Y7kAGzlUfl30VIUx9wQtz5sFTkWnBiyA/view?usp=sharing <https://drive.google.com/file/d/1Y7kAGzlUfl30VIUx9wQtz5sFTkWnBiyA/view?usp=sharing>
>      >
>      > [1]
>     https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0 <https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0>
>      > [2]
>     https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html <https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html>
>      > [3]
>     https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md
>     <https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md>
>      >
>      > Albert Esteve (1):
>      >    virtio-media: Add virtio media device specification
>      >
>      >   conformance.tex                           |  13 +-
>      >   content.tex                               |   1 +
>      >   device-types/media/description.tex        | 574
>     ++++++++++++++++++++++
>      >   device-types/media/device-conformance.tex |  11 +
>      >   device-types/media/driver-conformance.tex |   9 +
>      >   5 files changed, 604 insertions(+), 4 deletions(-)
>      >   create mode 100644 device-types/media/description.tex
>      >   create mode 100644 device-types/media/device-conformance.tex
>      >   create mode 100644 device-types/media/driver-conformance.tex
>      >
> 
>     Kind regards
> 
>     -- 
>     Alexander Gordeev
>     Senior Software Engineer
> 
>     OpenSynergy GmbH
>     Rotherstr. 20, 10245 Berlin
>     www.opensynergy.com <http://www.opensynergy.com>
>