[XviD-devel] Quality optimization

[Michael Militzer]

Hi,

> I had a look at quite a few FFmpeg tests lately, and from what I saw
> there, there is quite a lot of space left for optimizing quality within
> MPEG-4 bounds.
>
> Please comment on these:

ok ;)

> a) Mode decision. The FFMPEG "vhq" mode decision (checking by brute force
> which coding type needs the fewest bits) often improves the PSNR by
> 0.2-0.5, and it (almost) never does any harm (except for lower speed, of
> course).
>
> Proposal #1: Implement brute force mode decision into XVID !
> (The routines are all there, maybe they have to be slightly modified to be
> "reentrant")

yes, vhq mode especially seems to help with 4MV mode. INTER4V mode does not
give much benefit currently, maybe we can improve this

> b) Using SAD of Hadamard-transformed blocks instead of ordinary
> SAD for _subpixel_ resolution refinement (not for ordinary search!)
> can also increase PSNR significantly (though not for anime).
>
> Proposal #2: Implement SAD_hadamard which does the transform and SAD.
> (The routines are almost all there (or used to be), the result just
> might have to be scaled to fit into current SAD-checks)

why hadamard and not DCT? (because it's faster?) - shouldn't DCT give better
results (at least in theory)? And why SAD and not SSE? SSE should more
evenly distribute the error over a whole block which should make it easier
to code.

> c) Rate-Distorsion optimized Quantization
>
> Sometimes it's good not to just quantize the values and save the result,
> but modify results a little to safe bits while at the same time no do much
> damage to the image. ffmpeg has "Trellis Quantization" for this, which is
> really slow, but often gives a boost to PSNR. (Note that from the
> computational point this is by far less work than "optimal adaptive
> quantization")
>
> Proposal #3: Implement "intelligent" quantization, e.g. Trellis.
> (or port it from Ffmpeg ;-)

yes, but it's quite slow. Also some additional optimal adaptive quantization
would be nice...

> d) Set B-frames only when helpful
>
> A clip can benefit very much from Bframes, or the results can be horrible.
> In one of the VQEG test clips, I had a PSNR drop from 46.05 to 41.40
> (-4.95db) just because of activating 1 bframe! Average(!) quantizers
> jumped from 2.26 to 5.22. But there is also a clip where PSNR was
> increased from 34.3 to 35.5 (+1.2dB). XVID already has a dynamic test for
> this, but it's a very simple check...
>
> Proposal #4: Implement a better criterion for dynamic bframes.
> (first we have to find one, but if necessary, just use brute force)

I'm not sure if the benefits of b-frames can be measured by simple PSNR
comparisons: Of course the PSNR of bframes is generally lower than if we had
coded a pframe instead (simply because bframe quant is higher), however
bframes' lower quality don't influence the remaining picture sequence
(because bframes are not used as reference). If we now assume normal viewing
conditions (a video played back at ~25 fps), the quality decrease when a
bframe is presented for only a short time (1/25s) might not be noticed at
all. So for a fixed quant encoding with maybe 5 - 10 bframes/sec, the
perceived video quality might stay the same while we have a noticable
decrease in file size at the same time.

Of course there might be video clips, where b-frames are really doing harm
(and not only PSNR-wise). So for example if a PBP sequence (viewing order)
is not significantly smaller than a corresponding PPP sequence, bframes
should of course not be used (well, I guess I don't tell you anything new
here ;-))

bye,
Michael