FF card AV sync problems, possible fix to VDR (fwd)

  Hi,

Morfsta wrote:

> OK, so Karsta and I are suffering dropped TS packets. I know for sure
> that my DVB-T signal is not perfect so it is likely to occasionally lose
> some TS packets (freeview in the UK is susceptible to all sorts of
> interference from cars, motorbikes, DECT, etc) however, saying there is
> nothing that we can do about this because you don't have a *perfect* DVB
> signal surely isn't the right answer! My TV copes perfectly well with

Well, what I wanted to say is that I cannot help you any further at this
point. You have to check your hardware/driver whether there is a chance
to minimize packet loss. You did this already as you wrote below -- fine.

> dropped packets and doesn't go out of sync, as does my freeview box.
> Surely if some TS packets are dropped this shouldn't result in a loss of
> sync, the system should resync the audio and continue as before?? It
> can't be right to say, well we lost some data, so we can't recover until
> you change channels up and down, or fastforward/rewind by 1 minute!

You are right -- but don't you think that this thread got a bit off
topic? I've replied in this thread because there were cAudioRepacker
messages and as I've contributed this code to VDR I just wanted to make
sure that it works correctly in this case. Let's now come back to the
topic and concentrate on the AV sync issue.

> I have spent the last 2 weeks trying different combinations, playing
> with my kernel settings, trying USB DVB-T devices rather than the PCI
> and it makes absolutely no difference. The TS continuity errors do not
> go away so I am convinced it is simply that DVB-T is not as robust as
> other signals such as DVB-S or DVB-C. However, VDR should be able to
> cope and resync with lost data.

Well, basically this is not a matter of VDR but of the FF card's
firmware. For example, I do not own a FF card and do not see this AV
sync issue with xine, even when I intentionally drop TS packets by the
attached patch.

I'd like users of FF cards to test this patch. If this triggers AV sync
issues, firmware developers are invited to fix this issue.

Feel free to modify this patch to better simulate real TS packet losses
like bursts of 16 frames.

BE WARNED: *** do not use this patch on a production system ***

Bye.
  

Richard,

Many thanks for your continued support and assistance. I will test your
patch when I return home.

A week or two ago I tried to isolate a recording that was going out of sync,
so that I could send it to Oliver. The problem I found is that it didn't do
it all the time!
If I spotted a recording that had gone out of sync and then rewound a bit to
see where the sync problem started, the problem was corrected. I tried this
a few times
and couldn't isolate a single start and end point that I could cut to send.
Of course, I cannot do this on live TV..

What I do not understand and perhaps you could help me with, is that you say
that there is a problem with the firmware, yet there are entries in the log
from VDR
saying that it is dropping x bytes to sync on the next audio frame... This
suggests to me that VDR is doing *something* with respect to audio
synchronisation. Why then is the problem always reported as being an issue
with the firmware?

Kind Regards,

Morfsta
  

Hi,

Morfsta wrote:

> What I do not understand and perhaps you could help me with, is that you
> say that there is a problem with the firmware, yet there are entries in
> the log from VDR
> saying that it is dropping x bytes to sync on the next audio frame...
> This suggests to me that VDR is doing *something* with respect to audio
> synchronisation. Why then is the problem always reported as being an
> issue with the firmware?

As I tried to explain already, VDR doesn't see the TS packets when you
for example just watch live TV on a FF card (i. e. no recording running
in background), and the watched channel is available via the FF card.

When VDR runs a transfer thread e. g. when a budget card supplies a
channel that is to be watched on a FF card or when VDR is recording,
then the TS packets travel through cAudioRepacker and sync messages are
reported.

What happens when e. g. a TS packet gets lost in an audio stream?

Consider the following TS packets numbered Tx. An even and an odd
numbered packed shall build an audio frame (i. e., the length
information at the beginning of an audio packet shall report a length
which suits with this assumption) which is stored in a PES packet
numbered Pxy:

	T0 T1 T2 T3 T4 T5 T6 T7
	 \ /   \ /   \ /   \ /
	 P01   P23   P45   P67

Now assume that T3 gets lost:

	T0 T1 T2 T4 T5 T6 T7
	 \ /   \ /      \ /
	 P01   P24      P67

According to the length information, cAudioRepacker bundles T2 and T4
into PES packet P24. As the tail of this audio packet is incorrect,
playing it might result in some noise. cAudioRepacker will go out of
sync when reading packet T5 as it doesn't find an audio frame sync
marker and starts skipping bytes until it reaches T6. There it finds the
audio frame sync marker and reports that syncing skipped some bytes.
After that it continues it's work normally.

BTW: just in case you have a look at the code, please keep in mind that
this explanation has been simplified for clarity -- the real work is
done differently.

What's the result of the above process?

In the error case, you will only get 3 audio packets and moreover, if
you simply concat them, you will replay packet P67 at the point in time
where the original packet P45 would have been replayed. If one uses the
same technique for video packets and frames it is obvious, that audio
and video are nolonger in sync with each other.

The issue is solved by adding a so called presentation time stamp (PTS)
to some PES packets. Let's assume that packet P01 has a PTS of 0 and
packet P67 has a PTS of 6. When replaying, the PTS of packet P01 is
stored internally and according to the length information of the audio
packet, the internal counter is advanced by 2. The next packet P24
doesn't have a PTS value so it is assumed it would have the PTS value of
the internal counter which is 2 at that time. This packet is simply
output and the internal counter is advanced by 2, resulting in 4. Now,
when packet P67 is arriving, a PTS of 6 is read, but the internal
counter just shows 4, which means that there is a gap of duration 2
which needs to be filled for example with silence to stay in sync. After
that packet P67 is replayed.

Keeping audio and video in sync is done by using similar PTS values on
both video and audio PES packets. Consider two buffers for decoded audio
and video frames. The frames in each buffer were assigned either the PTS
value contained in the PES packet or the internally determined PTS value
(e. g. in the sample above, for the audio frame in packet P24 a PTS
value of 2 was determined). It's now quite easy to replay audio and
video in sync by introducing the so called system time counter STC,
which shall periodically advance by one -- in the above example starting
at 0. To replay audio and video frames in sync, those frames need to be
taken from the buffers and presented to the user which have an assigned
PTS value less than the current STC value.

BTW: this was a simplified explanation how xine manages sync of audio
and video while replaying.

Actually, I don't know how this is done in the case of a FF card and
what the firmware has to do in this regard. A guess -- which could
explain the issues you see -- would be that sync is not maintained
continuously. So after having maintained sync for some time, audio and
video frames are simply taken out of some FIFOs at a constant rate and
presented to the user -- this should keep audio and video in sync as
originally maintained. But when then for example an audio frame gets
lost due to a lost TS packet, audio and video get out of sync as the
lost packet brakes filling the FIFOs at a constant rate. When you try to
reproduce this effect by seeking back in the recording, then sync is
maintained actively and you don't see this issue again at that position
in the recording.

Please keep in mind that the last paragraph was just a guess -- I do not
want to blame anybody with this email.

Bye.
  

On Wed, Feb 21, 2007 at 11:12:43PM +0100, Reinhard Nissl wrote:
> 
> Actually, I don't know how this is done in the case of a FF card and
> what the firmware has to do in this regard. A guess -- which could
> explain the issues you see -- would be that sync is not maintained
> continuously. So after having maintained sync for some time, audio and
> video frames are simply taken out of some FIFOs at a constant rate and
> presented to the user -- this should keep audio and video in sync as
> originally maintained. But when then for example an audio frame gets
> lost due to a lost TS packet, audio and video get out of sync as the
> lost packet brakes filling the FIFOs at a constant rate. When you try to
> reproduce this effect by seeking back in the recording, then sync is
> maintained actively and you don't see this issue again at that position
> in the recording.

If the resulting Mpeg-Audio stream is broken in such a way that
the HW-Decoder runs into trouble from which it can not recover
the Audio HW_buffer gets emptied very fast which .. in fact ..
results in a silent but very fast video sequence.  For the next
firmware I've added a dectection of such an unrecoverable audio
decoder error to restart the audio decoder as fast as possible.

Btw: With xine and mplayer I hear a short noise and nothing
happen with the running picture. Maybe the mplay software
decoder its self has some checking about the Mpeg-Audio stream
and the AV synchronization does not depend on the audio PTS.


         Werner
  

Hi Werner,

Any idea when you will be able to release the new firmware for testing?

Kind Regards,

Morfsta


On 2/22/07, Dr. Werner Fink <werner@suse.de> wrote:
>
> For the next firmware I've added a dectection of such an unrecoverable
> audio
> decoder error to restart the audio decoder as fast as possible.
  

Hi,

Dr. Werner Fink wrote:

> On Wed, Feb 21, 2007 at 11:12:43PM +0100, Reinhard Nissl wrote:
>
>> Actually, I don't know how this is done in the case of a FF card and
>> what the firmware has to do in this regard. A guess -- which could
>> explain the issues you see -- would be that sync is not maintained
>> continuously. So after having maintained sync for some time, audio and
>> video frames are simply taken out of some FIFOs at a constant rate and
>> presented to the user -- this should keep audio and video in sync as
>> originally maintained. But when then for example an audio frame gets
>> lost due to a lost TS packet, audio and video get out of sync as the
>> lost packet brakes filling the FIFOs at a constant rate. When you try to
>> reproduce this effect by seeking back in the recording, then sync is
>> maintained actively and you don't see this issue again at that position
>> in the recording.
> 
> If the resulting Mpeg-Audio stream is broken in such a way that
> the HW-Decoder runs into trouble from which it can not recover
> the Audio HW_buffer gets emptied very fast which .. in fact ..
> results in a silent but very fast video sequence.  For the next
> firmware I've added a dectection of such an unrecoverable audio
> decoder error to restart the audio decoder as fast as possible.
> 
> Btw: With xine and mplayer I hear a short noise and nothing
> happen with the running picture. Maybe the mplay software
> decoder its self has some checking about the Mpeg-Audio stream
> and the AV synchronization does not depend on the audio PTS.

Well, in xine it works like that:

The demuxer thread reads the PES packets from the input stream,
determines whether the packet is audio respectively video, get's an
empty buffer from either the audio respectively video input buffer pool,
fills the buffer with the PES payload, transfers PTS from the PES packet
to the buffer when PTS is available in the PES packet and puts the
buffer into the audio respectively video input FIFO of the decoders.

Audio and video is decoded by different threads. Each one takes a buffer
from its input FIFO, stores the contained PTS value internally when one
is available to have it ready when the next access unit (= an audio
respectively video frame) is started. The buffer's data is decoded into
the current access unit's output buffer. When a new access unit starts,
the current output buffer is put into the audio respectively video
output FIFO and a new output buffer is allocated from the audio
respectively video output buffer pool. When a PTS value is available,
the PTS value is transferred into the current access unit's output
buffer. Otherwise, the PTS value of the last access unit's output buffer
is incremented by the last access unit's duration (= either audio
respectively video frame duration) and transferred instead. As a result
each decoded audio respectively video frame has now a PTS value assigned
to it (to be precise, transferring the PTS value translates it into an
internal representation which is used next).

Audio and video presentation is done by different threads. Each one
takes an output buffer (= access unit (= audio respectively video
frame)) from its associated decoder output FIFO, reads the PTS value and
compares it to the STC which is provided by the so called metronom. When
PTS is larger than STC, the video output thread simply sleeps until PTS
is less or equal to STC and when it awakes, it writes the output buffer
into the graphics card's video memory. The audio output thread behaves
similar, although it has to generate silence audio data for the sound
card in the case where sleeping for a longer time would cause the sound
card's input buffer to underrun.

So from the above explanation, I don't think that AV synchronization
doesn't depend on the audio PTS -- at least in the case of xine.

Bye.
  

On Thu, Feb 22, 2007 at 06:18:11PM +0000, Morfsta wrote:
> Hi Werner,
> 
> Any idea when you will be able to release the new firmware for testing?

I'm just testing the stuff in real live on my productions system ;)
Knock on wood that this stays stable ...

      Werner
  

Hi Werner,

Were there problems?

Morfsta

On 2/23/07, Dr. Werner Fink <werner@suse.de> wrote:
>
> On Thu, Feb 22, 2007 at 06:18:11PM +0000, Morfsta wrote:
> > Hi Werner,
> >
> > Any idea when you will be able to release the new firmware for testing?
>
> I'm just testing the stuff in real live on my productions system ;)
> Knock on wood that this stays stable ...
>
>      Werner
>
> --
> "Having a smoking section in a restaurant is like having
>          a peeing section in a swimming pool." -- Edward Burr
>
> _______________________________________________
> vdr mailing list
> vdr@linuxtv.org
> http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
>
  

On Sun, Mar 04, 2007 at 12:42:11PM +0000, Morfsta wrote:
> Hi Werner,
> 
> Were there problems?

Beside a small problem of Video underruns during change
from AC3 to Mpeg-Audio and at the start of a replay,
I see no problems.  At least the problem with broken
Mpeg-Audio frames are gone.  Hopefully I've something
ready at this evening :^)


       Werner

> On 2/23/07, Dr. Werner Fink <werner@suse.de> wrote:
> >On Thu, Feb 22, 2007 at 06:18:11PM +0000, Morfsta wrote:
> >> Hi Werner,
> >>
> >> Any idea when you will be able to release the new firmware for testing?
> >
> >I'm just testing the stuff in real live on my productions system ;)
> >Knock on wood that this stays stable ...
> >
> >     Werner
  

On 3/7/07, Dr. Werner Fink <werner@suse.de> wrote:
>
> AC3 loop through sound card http://bitstreamout.sourceforge.net/
> Howto http://www.vdr-portal.de/board/thread.php?threadid=1958
>

German language howtos are useless to all us english-speaking vdr users (and
there are a LOT of us!).  Perhaps someone can bother to translate.

Thanks.
  

VDR User wrote:
> On 3/7/07, *Dr. Werner Fink* <werner@suse.de <mailto:werner@suse.de>> wrote:
> 
>     AC3 loop through sound card http://bitstreamout.sourceforge.net/
>     Howto http://www.vdr-portal.de/board/thread.php?threadid=1958
>     <http://www.vdr-portal.de/board/thread.php?threadid=1958>
> 
> 
> German language howtos are useless to all us english-speaking vdr users
> (and there are a LOT of us!).  Perhaps someone can bother to translate.
> 
> Thanks.

how about this

http://www.linuxtv.org/vdrwiki/index.php/Bitstreamout-plugin
  

Hi Werner and List,
On Mi, Mär 07, 2007 at 04:36:12 +0100, Dr. Werner 
Fink wrote:
> Beside a small problem of Video underruns during change
> from AC3 to Mpeg-Audio and at the start of a replay,
> I see no problems.  At least the problem with broken
> Mpeg-Audio frames are gone.  Hopefully I've something
> ready at this evening :^)

Is this the mentioned firmware?  
http://www.suse.de/~werner/test_av-f22623.tar.bz2

What is the current status of the dvb firmware?
Are you planning to release a new version?
BR.
Halim
  

On Sat, Mar 15, 2008 at 03:12:49PM +0100, Halim Sahin wrote:
> Hi Werner and List,
> On Mi, Mär 07, 2007 at 04:36:12 +0100, Dr. Werner 
> Fink wrote:
> > Beside a small problem of Video underruns during change
> > from AC3 to Mpeg-Audio and at the start of a replay,
> > I see no problems.  At least the problem with broken
> > Mpeg-Audio frames are gone.  Hopefully I've something
> > ready at this evening :^)
> 
> Is this the mentioned firmware?  
> http://www.suse.de/~werner/test_av-f22623.tar.bz2
> 
> What is the current status of the dvb firmware?
> Are you planning to release a new version?

I'm currently very busy with my little daughter most
free time but I'm also working on the firmware.  It
has taken some time and will take some time.

       Werner
  

On Mon, Mar 17, 2008 at 6:19 AM, Dr. Werner Fink <werner@suse.de> wrote:
>  I'm currently very busy with my little daughter most
>  free time but I'm also working on the firmware.  It
>  has taken some time and will take some time.

This is why it's a shame more people aren't allowed to help code.  The
problem could have been solve years ago.  Well at least we should be
thankful Werner does any work at all because otherwise all hope would
be dead.