From patchwork Wed May 9 23:55:51 2007 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Artur Skawina X-Patchwork-Id: 12460 Received: from mx10.go2.pl ([193.17.41.74] helo=poczta.o2.pl) by www.linuxtv.org with esmtp (Exim 4.50) id 1Hlw19-00064w-2s for vdr@linuxtv.org; Thu, 10 May 2007 01:56:23 +0200 Received: from poczta.o2.pl (mx10.go2.pl [127.0.0.1]) by poczta.o2.pl (Postfix) with ESMTP id E74C958016 for ; Thu, 10 May 2007 01:55:51 +0200 (CEST) Received: from dly69.neoplus.adsl.tpnet.pl (dly69.neoplus.adsl.tpnet.pl [83.24.54.69]) by poczta.o2.pl (Postfix) with ESMTP for ; Thu, 10 May 2007 01:55:51 +0200 (CEST) Received: (qmail 19398 invoked from network); 9 May 2007 23:55:51 -0000 Received: from unknown (HELO ?172.19.43.221?) (172.19.43.221) by 172.19.43.250 with SMTP; 9 May 2007 23:55:51 -0000 Message-ID: <46425F87.6040606@o2.pl> Date: Thu, 10 May 2007 01:55:51 +0200 From: Artur Skawina User-Agent: Thunderbird 3.0a1 (X11/20070320) MIME-Version: 1.0 To: VDR Mailing List X-Enigmail-Version: 0.95b Subject: [vdr] [PATCH] dynamically sized ringbuffers v2 X-BeenThere: vdr@linuxtv.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: VDR Mailing List List-Id: VDR Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 May 2007 23:56:23 -0000 Status: O X-Status: X-Keywords: X-UID: 12835 Auto sized ringbuffers, changes since v1: - increased maximum sizes for a few rb users. Most of the time just a small part will be used, but there will be more room for times when more is required. - a little smarter cRingBufferLinear::Read(), handles the TS buffer better. - faster buffer growth. Also makes it less likely that there won't be an opportunity to resize before an overflow. - a potentially sleeping cRecorder:Receive(). Mostly to find out how much more buffer/time would have been needed, ie debugging. - more verbose logging. It still wouldn't surprise me if this version caused a few overflows, but hopefully these will be very rare. artur diff --git a/dvbdevice.c b/dvbdevice.c index 955483e..20dcf29 100644 --- a/dvbdevice.c +++ b/dvbdevice.c @@ -1184,7 +1184,7 @@ bool cDvbDevice::OpenDvr(void) CloseDvr(); fd_dvr = DvbOpen(DEV_DVB_DVR, CardIndex(), O_RDONLY | O_NONBLOCK, true); if (fd_dvr >= 0) - tsBuffer = new cTSBuffer(fd_dvr, MEGABYTE(2), CardIndex() + 1); + tsBuffer = new cTSBuffer(fd_dvr, MEGABYTE(8), CardIndex() + 1); return fd_dvr >= 0; } diff --git a/recorder.c b/recorder.c index 8bb1621..3c0e002 100644 --- a/recorder.c +++ b/recorder.c @@ -157,8 +157,20 @@ void cRecorder::Receive(uchar *Data, int Length) { if (Running()) { int p = ringBuffer->Put(Data, Length); - if (p != Length && Running()) + if (p != Length && Running()) { + for (int ms=20; ms<1000; ms+=ms) { + cCondWait::SleepMs(ms); + if (!Running()) + return; + int r = ringBuffer->Put(Data+p, Length-p); + p += r; + if (r) + dsyslog("saved extra %d bytes in recorder ring buffer after %d ms delay", r, ms); + if (p == Length || !Running()) + return; + } ringBuffer->ReportOverflow(Length - p); + } } } diff --git a/remux.c b/remux.c index da805b7..bd29b2f 100644 --- a/remux.c +++ b/remux.c @@ -1851,7 +1851,7 @@ void cTS2PES::ts_to_pes(const uint8_t *Buf) // don't need count (=188) // --- cRemux ---------------------------------------------------------------- -#define RESULTBUFFERSIZE KILOBYTE(256) +#define RESULTBUFFERSIZE MEGABYTE(2) cRemux::cRemux(int VPid, const int *APids, const int *DPids, const int *SPids, bool ExitOnFailure) { diff --git a/ringbuffer.c b/ringbuffer.c index 0633bd3..39cc02e 100644 --- a/ringbuffer.c +++ b/ringbuffer.c @@ -151,13 +151,30 @@ void cRingBufferLinear::PrintDebugRBL(void) } #endif +// cRingBufferLinear are dynamically sized, or at least we can pretend they are ;) +// We treat 'Size' as the maximum size, but start with a small buffer, which can +// grow later as it fills up. Memory is always allocated for the full buffer, but +// the unused RAM portion remains untouched until (if at all) it is actually needed. +// Note that we can not start with a larger than requested buffer because there are +// ring buffer users that cause crashes then (eg softdevice mpegdecoder absolutely +// needs 32/64k). + +// Startup size. 64k still causes overflows before buffer starts to grow, 128k doesn't. +// The buffers grow to 200..500k anyway, so maybe increasing this a bit more would +// make sense, but let's first see if it's really needed. +// In fact 128k is on the low side, but let's try not going for 256k yet. +#define DEFRBSIZE KILOBYTE(128) + cRingBufferLinear::cRingBufferLinear(int Size, int Margin, bool Statistics, const char *Description) -:cRingBuffer(Size, Statistics) +:cRingBuffer(min(Size,DEFRBSIZE), Statistics) { description = Description ? strdup(Description) : NULL; tail = head = margin = Margin; gotten = 0; buffer = NULL; + maxsize = Size; + growbuf = 0; + dsyslog("New ring buffer \"%s\" size: %d margin: %d", description, Size, Margin ); if (Size > 1) { // 'Size - 1' must not be 0! if (Margin <= Size / 2) { buffer = MALLOC(uchar, Size); @@ -183,6 +200,8 @@ cRingBufferLinear::~cRingBufferLinear() #ifdef DEBUGRINGBUFFERS DelDebugRBL(this); #endif + dsyslog("Deleting ring buffer \"%s\" size: %d / %d (used %g%% of requested size)", description, + size, maxsize, size/(double)maxsize*100.0 ); free(buffer); free(description); } @@ -205,8 +224,21 @@ void cRingBufferLinear::Clear(void) EnablePut(); } +// Must only be called by the producer (ie Read()/Put()), not the consumer (Get()). +void cRingBufferLinear::GrowBuffer(void) +{ + size = min(size+size, maxsize); + dsyslog("Enlarging ring buffer \"%s\": %d bytes (trigger %d)", + description, size, growbuf); + growbuf = 0; +} + int cRingBufferLinear::Read(int FileHandle, int Max) { + if (Available() >= size/2) + growbuf = 1; + if (growbuf && head>=tail && size 0) ? diff - 1 : Size() - head; @@ -219,6 +251,10 @@ int cRingBufferLinear::Read(int FileHandle, int Max) Count = safe_read(FileHandle, buffer + head, free); if (Count > 0) { int Head = head + Count; + if (Available()+Count >= size/2) + growbuf = 2; + if (growbuf && head>=tail && size= Size()) Head = margin; head = Head; @@ -245,6 +281,10 @@ int cRingBufferLinear::Read(int FileHandle, int Max) int cRingBufferLinear::Put(const uchar *Data, int Count) { if (Count > 0) { + if (Available()+Count >= size/2) + growbuf = 3; + if (growbuf && head>=tail && size