[XviD-devel] HUGE ME regressions between beta1 and beta2

Fri Nov 25 17:02:39 CET 2005

Ok, well looking at the file sizes you report in your test it's rather 
likely you used another sequence than me - and likely DVD input...

Michael

Quoting Michael Militzer <michael at xvid.org>:

> Hi,
> 
> well, I'm rather confident that the current ("traditional") way finds
> better vectors in ME than before, especially at high quants. What I can
> imagine is that there's a larger influence on mode decision that I expected
> (and experienced) in non R-D optimized mode. I've tested with lots more
> sequences than just the ones I posted to the list and never had a 
> regression with the new code. However, I used standard testing sequences
> from uncompressed input. Perhaps, mode decision with the new lambdas doesn't
> work well on DVD/MPEG-2 compressed input.
> 
> That should be easy to find out: if there's no regression with VHQ>0, then
> the degradation on some sequences likely comes from mode decision. That
> should be easy to fix then by applying additional weights after vector
> search. Also, it may explain why the problem slipped through unnoticed - 
> most people use VHQ I guess.
> 
> BTW: with what sequence do you test? With the same I used and you get
> different results (that would be very concerning)? Or with another one?
> 
> Regards,
> Michael
> 
> 
> Quoting Radek Czyz <radoslaw at syskin.cjb.net>:
> 
> > Hi again,
> > 
> > Michael Militzer wrote:
> > > well, I've basically just changed our lambda tables.
> > 
> > Yes, but you also changed much more: the way lambda actually works went 
> > back to "traditional" way (which is in theory more correct). The lambda 
> > value multiplier changed from 10.5 to 0.6 (and from 40.0 to 0.6 for 
> > inter4v).
> > 
> > In general, I'd expect that this kind of change has large consequences, 
> > in particular to inter/inter4v decisions.
> > 
> > Overal, that was a reasonably large patch. The important thing is that I 
> > cannot confirm your results. This is what I get at pure defaults:
> > 
> > Before: 1:53, 25.6MB, 41.70dB
> > After:  2:03, 26.0MB, 41.78dB
> > 
> > I am still unsure what causes the slowdown. Your patch removes one 
> > multiplication in important spot so I'd expect is to be faster.
> > 
> > I've put the "old" files online, if anyone wants to test: 
> > http://syskin.is.dreaming.org/motion_old.zip
> > 
> > Radek
> > 
> > 
> >   Before, the values for
> > > lambda were borrowed from h.264 it seemed however didn't fit XviD's
> linear
> > > qscale at all. In result, R-D performance especially for higher QPs was
> > > absolutely horrible. I've changed it to a more lambda = 0.85*Q scheme
> (ok,
> > > can't recall the exact constant from memory now). I've tested the change
> > > for VHQ and non-VHQ modes and had always superior performance, sometimes
> 
> > > with really huge gains especially a low bit-rates/high quants. 
> > > 
> > > I've attached my original mail to the list at the end of this mails and
> > also
> > > reposted some of the R-D curves I did at those times. Your testing case
> is
> > > covered as well (it seems).
> > > 
> > > Sure we can postpone 1.1 final until this problem is fixed. BTW: did you
> > use
> > > a standard sequence for your test so we can reproduce the problem?
> > > 
> > > Regards,
> > > Michael
> > > 
> > > 
> > > Quoting Radek Czyz <radoslaw at syskin.cjb.net>:
> > > 
> > > 
> > >>Hi everyone,
> > >>
> > >>Apparently our testing community sucks ;))
> > >>
> > >>There was a HUGE quality and speed regression between beta1 and beta2. I
> 
> > >>narrowed it down to SAD-based ME. The major changes in that time was 
> > >>Isibaar's new lambda code, but apparently we have hardly any logs what 
> > >>happened exactly (well, we've got cvs, I'll check that).
> > >>
> > >>For fast settings (no VHQ, no trellis, no chromaME, defaults otherwise):
> > >>
> > >>Beta1's ME:  time 1:19 filesize 26.2MB psnr 41.19dB
> > >>Current ME:  time 1:24 filesize 28.8MB psnr 41.14dB
> > >>
> > >>
> > >>What the heck happened? ;_;
> > >>
> > >>I suggest we postpone 1.1final until we figure this out - looks like all
> 
> > >>speed improvements over 1.0 are gone (10% slower at these settings, 5% 
> > >>slower at defaults), and 10% more filesize for *lower* quality is...
> bad.
> > >>
> > >>Radek
> > >>_______________________________________________
> > >>XviD-devel mailing list
> > >>XviD-devel at xvid.org
> > >>http://list.xvid.org/mailman/listinfo/xvid-devel
> > >>
> > > 
> > > 
> > > 
> > > ----- Forwarded message from Michael Militzer <michael at xvid.org> -----
> > >     Date: Mon, 14 Mar 2005 19:59:37 +0100
> > >     From: Michael Militzer <michael at xvid.org>
> > > Reply-To: Michael Militzer <michael at xvid.org>
> > >  Subject: low-bitrate patch
> > >       To: xvid-devel at xvid.org
> > > 
> > > Hi all,
> > > 
> > > I've committed a patch to cvs that improves XVID's performance at low-
> > > bitrates by using better lambda values for R-D vector search. By this
> also
> > > mode decision (SAD-based) is improved at low-bitrates. At mid/higher
> > bitrates
> > > there's no change in performance, so only people who aim at really low
> > > bitrate (and quality) encodes will benefit from this patch.
> > > 
> > > I've attached some R-D graphs that compare the behavior of patched and 
> > > unpatched XVID for various sequences and different bit-rates. Note that
> > I've
> > > used a logarithmic scale, so that the differences at low bitrates can be
> > > better perceived. As can be clearly noticed, the compression performance
> > is
> > > enhanced significantly at very low bitrates (up to 2-3 dB PSNR).
> > > 
> > > Well, it's another question how useful this patch actually is because
> such
> > > extremely low bit-rates are barely used. But for completeness, XVID
> should
> > > perform equally great at all bitrates...
> > > 
> > > bye,
> > > Michael
> > > 
> > > ----- End forwarded message -----
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ------------------------------------------------------------------------
> > > 
> > > _______________________________________________
> > > XviD-devel mailing list
> > > XviD-devel at xvid.org
> > > http://list.xvid.org/mailman/listinfo/xvid-devel
> > _______________________________________________
> > XviD-devel mailing list
> > XviD-devel at xvid.org
> > http://list.xvid.org/mailman/listinfo/xvid-devel
> > 
> 
> 
> 
> 
> _______________________________________________
> XviD-devel mailing list
> XviD-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
> 