Instability with recordings on VDR-1.7.4 when recording to a NTFS partition
Commit Message
On 23.03.2009 08:35, Klaus Schmidinger wrote:
> On 23.03.2009 01:09, Niedermeier Günter wrote:
>>> There must be an other problem that's causing this, but since this doesn't
>>> happen here on my system, I'm afraid you'll need to do the debugging ;-)
>> Which changes have been made between 1.7.2 and 1.7.3 in
>> file writing mechanism?
>>
>> Not codechanges, because I dont understand them, but in words please.
>> E.g. blocksize changed from xxx to yyy changed algo. changed cache or
>> something else which can influence the performance.
>>
>> I found out, that in 172 the most time up to 20 stream data blocks are
>> transmitted via NFS between one "NFS WRITE CALL / WRITE REPLAY" and
>> "NFS COMMIT CALL / COMMIT REPLAY" combination and the next one.
>>
>> In 173/174 the number of stream data blocks decreases to an amount of
>> 5 blocks maximal. Therefor the number of "NFS WRITE CALL / WRITE REPLAY"
>> and "NFS COMMIT CALL / COMMIT REPLAY" combinations increases up to
>> 4 to 5 times higher than in 172.
>>
>> This produces an enormous overhead, and this overhead could be
>> reasonable for the two MegaByte/s networkoverload above the normal
>> load with 1 MB/s per stream.
>>
>> Perhaps Klaus, you have an idea.
>
> I believe I do. With PES recordings, data was written to the file
> in larger chunks, while with TS recordings it is written in blocks
> of 188 byte (TS_SIZE). I'll chnage cFrameDetector::Analyze() to
> handle more data at once.
> Will try to provide a patch for testing tonight.
Here's a quick shot - totally untested (no time, sorry).
Please try it and let me know if it helps.
Klaus
Comments
> Here's a quick shot - totally untested (no time, sorry).
> Please try it and let me know if it helps.
Hi,
I've tried it:
files are created, but with zero filesize.
Only info is correct.
After a few seconds vdr is restarting with an "emergency exit!".
-Günter
--
This message was scanned by ESVA and is believed to be clean.
On 23.03.2009 21:42, Niedermeier Günter wrote:
>> Here's a quick shot - totally untested (no time, sorry).
>> Please try it and let me know if it helps.
>
> Hi,
>
> I've tried it:
> files are created, but with zero filesize.
> Only info is correct.
>
> After a few seconds vdr is restarting with an "emergency exit!".
Well, then I guess I do need to spend a little more time on this ;-)
Klaus
On 23.03.2009 22:13, Klaus Schmidinger wrote:
> On 23.03.2009 21:42, Niedermeier Günter wrote:
>>> Here's a quick shot - totally untested (no time, sorry).
>>> Please try it and let me know if it helps.
>> Hi,
>>
>> I've tried it:
>> files are created, but with zero filesize.
>> Only info is correct.
>>
>> After a few seconds vdr is restarting with an "emergency exit!".
>
> Well, then I guess I do need to spend a little more time on this ;-)
Just another quick thing to test: please add
}
Data += TS_SIZE; <========== this line
Length -= TS_SIZE;
Processed += TS_SIZE;
}
return Processed;
}
to the end of cFrameDetector::Analyze().
Klaus
> Just another quick thing to test: please add
>
> }
> Data += TS_SIZE; <========== this line
> Length -= TS_SIZE;
> Processed += TS_SIZE;
> }
> return Processed;
> }
>
> to the end of cFrameDetector::Analyze().
...looks good at all!
I've tested streaming of four channels via the "slow"
NFS share without any problem until now.
The overall netload was at a maximum of 4,5 MB/s for four streams.
A single stream (ARD) needs roundabout 1,1 MB/s.
...Die ARD'ler müssen Geld und Bandbreite ohne Ende haben!!!
-->15Min/850MB TS File
Wonderful !
One question (OT): is it normal that replaying a PES record causes
in a very high CPU load on 1.7.2 and higher? -> 80-90% on a P4/2400
Replaying TS produces "no" CPU load.
-Günter
--
This message was scanned by ESVA and is believed to be clean.
> One question (OT): is it normal that replaying a PES record causes
> in a very high CPU load on 1.7.2 and higher? -> 80-90% on a P4/2400
>
> Replaying TS produces "no" CPU load.
Forget it! This seems to be fixed too since your patch.
Could that be possible?
-Günter
--
This message was scanned by ESVA and is believed to be clean.
On 03/24/09 01:09, Niedermeier Günter wrote:
>> One question (OT): is it normal that replaying a PES record causes
>> in a very high CPU load on 1.7.2 and higher? -> 80-90% on a P4/2400
>>
>> Replaying TS produces "no" CPU load.
>
> Forget it! This seems to be fixed too since your patch.
>
> Could that be possible?
I don't think so. The modified code is not involved
in replaying, only in recording.
Klaus
>>> Replaying TS produces "no" CPU load.
>> Forget it! This seems to be fixed too since your patch.
>>
>> Could that be possible?
>
> I don't think so. The modified code is not involved
> in replaying, only in recording.
However, currently 174 works without high CPU load in my
environment. I had no chance to reproduce it again yesterday.
In 173 or 172 I can do.
Is this a known problem?
-Günter
--
This message was scanned by ESVA and is believed to be clean.
>>> One question (OT): is it normal that replaying a PES record causes
>>> in a very high CPU load on 1.7.2 and higher? -> 80-90% on a P4/2400
>>>
>>> Replaying TS produces "no" CPU load.
>>
>> Forget it! This seems to be fixed too since your patch.
>>
>> Could that be possible?
>I don't think so. The modified code is not involved
>in replaying, only in recording.
>Klaus
I too can confirm that this has fixed my problem recording to an NTFS
Partition. With the patches applied I have successfully recorded two
streams simultaneously while playing a third.
Thanks.
Mark.
This is an email from Fujitsu Australia Limited, ABN 19 001 011 427. It is confidential to the ordinary user of the email address to which it was addressed and may contain copyright and/or legally privileged information. No one else may read, print, store, copy or forward all or any of it or its attachments. If you receive this email in error, please return to sender. Thank you.
If you do not wish to receive commercial email messages from Fujitsu Australia Limited, please email unsubscribe@au.fujitsu.com
Klaus Schmidinger kirjoitti:
> On 23.03.2009 22:13, Klaus Schmidinger wrote:
>
>> On 23.03.2009 21:42, Niedermeier Günter wrote:
>>
>>>> Here's a quick shot - totally untested (no time, sorry).
>>>> Please try it and let me know if it helps.
>>>>
>>> Hi,
>>>
>>> I've tried it:
>>> files are created, but with zero filesize.
>>> Only info is correct.
>>>
>>> After a few seconds vdr is restarting with an "emergency exit!".
>>>
>> Well, then I guess I do need to spend a little more time on this ;-)
>>
I hope the patch does what I thought it would: collects writes in larger
chunks. For a networked application, the key performance bottleneck is
the number of needed transactions. See e.g.
http://portal.acm.org/citation.cfm?id=1066051.1066069 (and the PDF file
in there).
yours,
Jouni
On 24.03.2009 17:05, Jouni Karvo wrote:
> Klaus Schmidinger kirjoitti:
>> On 23.03.2009 22:13, Klaus Schmidinger wrote:
>>
>>> On 23.03.2009 21:42, Niedermeier Günter wrote:
>>>
>>>>> Here's a quick shot - totally untested (no time, sorry).
>>>>> Please try it and let me know if it helps.
>>>>>
>>>> Hi,
>>>>
>>>> I've tried it:
>>>> files are created, but with zero filesize.
>>>> Only info is correct.
>>>>
>>>> After a few seconds vdr is restarting with an "emergency exit!".
>>>>
>>> Well, then I guess I do need to spend a little more time on this ;-)
>>>
>
> I hope the patch does what I thought it would: collects writes in larger
> chunks. For a networked application, the key performance bottleneck is
> the number of needed transactions. See e.g.
> http://portal.acm.org/citation.cfm?id=1066051.1066069 (and the PDF file
> in there).
That's exactly what the fix does - and apparently it works, judging from
the feedback.
Klaus
On 24.03.2009 09:22, Niedermeier Günter wrote:
>>>> Replaying TS produces "no" CPU load.
>>> Forget it! This seems to be fixed too since your patch.
>>>
>>> Could that be possible?
>> I don't think so. The modified code is not involved
>> in replaying, only in recording.
>
> However, currently 174 works without high CPU load in my
> environment. I had no chance to reproduce it again yesterday.
> In 173 or 172 I can do.
>
> Is this a known problem?
Not to my knowledge.
But since it works better in version 1.7.4, that's a step
in the right direction, I'd say ;-)
Klaus
@@ -672,8 +672,9 @@
int cFrameDetector::Analyze(const uchar *Data, int Length)
{
+ int Processed = 0;
newFrame = independentFrame = false;
- if (Length >= TS_SIZE) {
+ while (Length >= TS_SIZE) {
if (TsHasPayload(Data) && !TsIsScrambled(Data) && TsPid(Data) == pid) {
if (TsPayloadStart(Data)) {
if (!frameDuration) {
@@ -718,11 +719,12 @@
case 0x02: // MPEG 2 video
if (scanner == 0x00000100) { // Picture Start Code
if (frameDuration) {
+ if (Processed)
+ return Processed;
newFrame = true;
independentFrame = ((Data[i + 2] >> 3) & 0x07) == 1; // I-Frame
if (framesPerPayloadUnit == 1) {
scanning = false;
- return TS_SIZE;
}
}
else {
@@ -735,11 +737,12 @@
case 0x1B: // MPEG 4 video
if (scanner == 0x00000109) { // Access Unit Delimiter
if (frameDuration) {
+ if (Processed)
+ return Processed;
newFrame = true;
independentFrame = Data[i + 1] == 0x10;
if (framesPerPayloadUnit == 1) {
scanning = false;
- return TS_SIZE;
}
}
else {
@@ -752,6 +755,8 @@
case 0x04: // MPEG audio
case 0x06: // AC3 audio
if (frameDuration) {
+ if (Processed)
+ return Processed;
newFrame = true;
independentFrame = true;
scanning = false;
@@ -765,7 +770,8 @@
}
}
}
- return TS_SIZE;
+ Length -= TS_SIZE;
+ Processed += TS_SIZE;
}
- return 0;
+ return Processed;
}