[XviD-devel] HUGE ME regressions between beta1 and beta2

[Michael Militzer]

Hi,

well, I'm rather confident that the current ("traditional") way finds
better vectors in ME than before, especially at high quants. What I can
imagine is that there's a larger influence on mode decision that I expected
(and experienced) in non R-D optimized mode. I've tested with lots more
sequences than just the ones I posted to the list and never had a 
regression with the new code. However, I used standard testing sequences
from uncompressed input. Perhaps, mode decision with the new lambdas doesn't
work well on DVD/MPEG-2 compressed input.

That should be easy to find out: if there's no regression with VHQ>0, then
the degradation on some sequences likely comes from mode decision. That
should be easy to fix then by applying additional weights after vector
search. Also, it may explain why the problem slipped through unnoticed - 
most people use VHQ I guess.

BTW: with what sequence do you test? With the same I used and you get
different results (that would be very concerning)? Or with another one?

Regards,
Michael


Quoting Radek Czyz <radoslaw at syskin.cjb.net>:

> Hi again,
> 
> Michael Militzer wrote:
> > well, I've basically just changed our lambda tables.
> 
> Yes, but you also changed much more: the way lambda actually works went 
> back to "traditional" way (which is in theory more correct). The lambda 
> value multiplier changed from 10.5 to 0.6 (and from 40.0 to 0.6 for 
> inter4v).
> 
> In general, I'd expect that this kind of change has large consequences, 
> in particular to inter/inter4v decisions.
> 
> Overal, that was a reasonably large patch. The important thing is that I 
> cannot confirm your results. This is what I get at pure defaults:
> 
> Before: 1:53, 25.6MB, 41.70dB
> After:  2:03, 26.0MB, 41.78dB
> 
> I am still unsure what causes the slowdown. Your patch removes one 
> multiplication in important spot so I'd expect is to be faster.
> 
> I've put the "old" files online, if anyone wants to test: 
> http://syskin.is.dreaming.org/motion_old.zip
> 
> Radek
> 
> 
>   Before, the values for
> > lambda were borrowed from h.264 it seemed however didn't fit XviD's linear
> > qscale at all. In result, R-D performance especially for higher QPs was
> > absolutely horrible. I've changed it to a more lambda = 0.85*Q scheme (ok,
> > can't recall the exact constant from memory now). I've tested the change
> > for VHQ and non-VHQ modes and had always superior performance, sometimes 
> > with really huge gains especially a low bit-rates/high quants. 
> > 
> > I've attached my original mail to the list at the end of this mails and
> also
> > reposted some of the R-D curves I did at those times. Your testing case is
> > covered as well (it seems).
> > 
> > Sure we can postpone 1.1 final until this problem is fixed. BTW: did you
> use
> > a standard sequence for your test so we can reproduce the problem?
> > 
> > Regards,
> > Michael
> > 
> > 
> > Quoting Radek Czyz <radoslaw at syskin.cjb.net>:
> > 
> > 
> >>Hi everyone,
> >>
> >>Apparently our testing community sucks ;))
> >>
> >>There was a HUGE quality and speed regression between beta1 and beta2. I 
> >>narrowed it down to SAD-based ME. The major changes in that time was 
> >>Isibaar's new lambda code, but apparently we have hardly any logs what 
> >>happened exactly (well, we've got cvs, I'll check that).
> >>
> >>For fast settings (no VHQ, no trellis, no chromaME, defaults otherwise):
> >>
> >>Beta1's ME:  time 1:19 filesize 26.2MB psnr 41.19dB
> >>Current ME:  time 1:24 filesize 28.8MB psnr 41.14dB
> >>
> >>
> >>What the heck happened? ;_;
> >>
> >>I suggest we postpone 1.1final until we figure this out - looks like all 
> >>speed improvements over 1.0 are gone (10% slower at these settings, 5% 
> >>slower at defaults), and 10% more filesize for *lower* quality is... bad.
> >>
> >>Radek
> >>_______________________________________________
> >>XviD-devel mailing list
> >>XviD-devel at xvid.org
> >>http://list.xvid.org/mailman/listinfo/xvid-devel
> >>
> > 
> > 
> > 
> > ----- Forwarded message from Michael Militzer <michael at xvid.org> -----
> >     Date: Mon, 14 Mar 2005 19:59:37 +0100
> >     From: Michael Militzer <michael at xvid.org>
> > Reply-To: Michael Militzer <michael at xvid.org>
> >  Subject: low-bitrate patch
> >       To: xvid-devel at xvid.org
> > 
> > Hi all,
> > 
> > I've committed a patch to cvs that improves XVID's performance at low-
> > bitrates by using better lambda values for R-D vector search. By this also
> > mode decision (SAD-based) is improved at low-bitrates. At mid/higher
> bitrates
> > there's no change in performance, so only people who aim at really low
> > bitrate (and quality) encodes will benefit from this patch.
> > 
> > I've attached some R-D graphs that compare the behavior of patched and 
> > unpatched XVID for various sequences and different bit-rates. Note that
> I've
> > used a logarithmic scale, so that the differences at low bitrates can be
> > better perceived. As can be clearly noticed, the compression performance
> is
> > enhanced significantly at very low bitrates (up to 2-3 dB PSNR).
> > 
> > Well, it's another question how useful this patch actually is because such
> > extremely low bit-rates are barely used. But for completeness, XVID should
> > perform equally great at all bitrates...
> > 
> > bye,
> > Michael
> > 
> > ----- End forwarded message -----
> > 
> > 
> > 
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> > _______________________________________________
> > XviD-devel mailing list
> > XviD-devel at xvid.org
> > http://list.xvid.org/mailman/listinfo/xvid-devel
> _______________________________________________
> XviD-devel mailing list
> XviD-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
>