[XviD-devel] HUGE ME regressions between beta1 and beta2

[Michael Militzer]

Hi Radek,

Quoting Radek Czyz <radoslaw at syskin.cjb.net>:

> Michael, I have a question: the lambda tables for 16x16 search are 
> different than for 8x8 search. For 8x8, they are linear:
> 1, 2, 3, 4, 5, 6, 7, 8, 9...
> 
> but for 16x16, there's a kink at low quants:
> 
> 0.5, 1, 1.5, 2, 2.5, 5, 7, 8, 9 ......
> 
> Why's that?
> I'm currently getting much better results for linear tables everywhere.

Well, the idea was: imho, the lambda*MV_bits cost function bears the danger
that lowers perceived quality even though it increases PSNR. That's because
PSNR is a metric that does not consider temporal artifacts. For example,
the higher the lambda value you use the higher the danger for smearing 
artifacts. Try some high lambda and watch moving objects leaving trails
even though it doesn't hurt R-D performance (because the PSNR reduction is
compensated by bit gain). Again imho, temporal artifacts are minimized (and
subjective quality with this regard maximized) when your ME finds vectors
that correlate with the true motion in the sequence. In this case, MBs will
temporally predict from those past areas the human eye expect it.

So in theory, lambda=0 should give best results with this regard. However,
of course the R-D optimized vector search helps a lot, so we cannot go with
lambda=0. But certainly, a lower lambda should result in less smearing or
trails, so if we can afford it we should go for it. And for low quants we
typically have large textual residuals, which take up most of the output
file size. So reducing the MV bits is less a consideration - simply because
MV bits will take up just a minor fraction of the total size. So a smaller
lambda should hurt R-D performance just very mildly while providing the
better visual quality. That was the idea ;)
 
> In other news,
> 
> The slowdown is caused by all motion searches checking many more vectors 
> than before. For some frames, twice as many. Apparenty diamonds like to 
> wander far away now.

That basically also shouldn't be the case at low quants. At higher quants,
yes sure, as the old lambda tables were introducing such a huge cost penalty
that each MB was pinned at MV (0,0). Certainly faster but resulted in
horrible quality at higher quants.

Perhaps, one of the early stop criterias don't work properly anymore. E.g.
the early stop to prevent extensive 8x8 search. I've just seen that these
early stops use fixed thresholds. Imho, they should be adaptive to be more
robust...

> The filesize difference at fixed quant is caused almost entirely by 
> b-frames being much larger than before. P-frames were same size, and are 
> smaller now with linear tables.

Actually, there shouldn't be large file size differences. If you look at
the graphs I plotted, the sample points of before and after the lambda
change are rather close at low quants. With the new linear lambda tables
just performing a bit better.

Regards,
Michael