[XviD-devel] Quality optimization

[Christoph Lampert]

Hi,

I had a look at quite a few FFmpeg tests lately, and from what I saw
there, there is quite a lot of space left for optimizing quality within
MPEG-4 bounds. 

Please comment on these: 

a) Mode decision. The FFMPEG "vhq" mode decision (checking by brute force
which coding type needs the fewest bits) often improves the PSNR by
0.2-0.5, and it (almost) never does any harm (except for lower speed, of
course). 

Proposal #1: Implement brute force mode decision into XVID ! 
(The routines are all there, maybe they have to be slightly modified to be
"reentrant")

............................

b) Using SAD of Hadamard-transformed blocks instead of ordinary 
SAD for _subpixel_ resolution refinement (not for ordinary search!)
can also increase PSNR significantly (though not for anime). 

Proposal #2: Implement SAD_hadamard which does the transform and SAD. 
(The routines are almost all there (or used to be), the result just
might have to be scaled to fit into current SAD-checks) 

............................

c) Rate-Distorsion optimized Quantization

Sometimes it's good not to just quantize the values and save the result,
but modify results a little to safe bits while at the same time no do much
damage to the image. ffmpeg has "Trellis Quantization" for this, which is
really slow, but often gives a boost to PSNR. (Note that from the
computational point this is by far less work than "optimal adaptive
quantization")

Proposal #3: Implement "intelligent" quantization, e.g. Trellis.
(or port it from Ffmpeg ;-)

...............................

d) Set B-frames only when helpful

A clip can benefit very much from Bframes, or the results can be horrible.
In one of the VQEG test clips, I had a PSNR drop from 46.05 to 41.40
(-4.95db) just because of activating 1 bframe! Average(!) quantizers
jumped from 2.26 to 5.22. But there is also a clip where PSNR was
increased from 34.3 to 35.5 (+1.2dB). XVID already has a dynamic test for
this, but it's a very simple check... 

Proposal #4: Implement a better criterion for dynamic bframes. 
(first we have to find one, but if necessary, just use brute force) 


----------------------------------
extra, because untested: 

* I'm sure that GMC can do better than our current version, in particular
GME and mode decision.

* For B-frames, we buffer a number of frames internally before encoding
them. They could also be used for encoding decisions (single pass
ratecontrol, skipMB-check, frame-skip-check), no only for B-framing them.
Since the encoding software supports because of bframes anyway, we could
also prebuffer frames when _not_ using bframes. 

---------------------------------


gruel