[XviD-devel] GME and Block ME

[Marco Al]

From: "Radek Czyz" <radoslaw at airnet.com.au>

> Yay, trellis quant :-) I was about to ask you what it's all about,
> googling for it gave me nothing :(.

You need to improve your googling skill :) Anyway, trellis quantization is a
bit of a misnomer ... looks to much like trellis coded quantization which is
something entirely different. I like "trellis searched quantization".

Heres my googling results :

http://www.icsl.ucla.edu/~ipl/Publications/Journal/trellis-based_r-d_optimal
_quantization_in_h263%2B.pdf
http://www.informatik.uni-mannheim.de/informatik/pi4/publications/library/Fa
rin2002a.pdf

On a slightly different note the following is interesting too :
http://vision.ece.ucsb.edu/publications/97ICIPVideo.pdf

It does adaptive quantization, but taking into account the quasi RLE coding
of the MB quantizers (through DQUANT). It does this by a suboptimal search
of the trellis, they call it a L-best search but it is usually called a
M-best search I think ... this is an alternative to the optimal search used
in the above papers, which is sometimes called viterbi algorithm/search.

> Anyway, could you please tell me,
> briefly, how it works? I understand that it's looking for
> optimal bits/distortion, but the question is: what does it do to
> change bits/distortion? At first I thought it's changing the quantizer
> but no - ffmpeg's file with trellis quant had constant quantizer. So I
> thought that it's decreasing, maybe discarding coefficients - that
> would decrease number of bits and increase distortion, ok. But again,
> no - ffmpeg's file was _larger_ and with much better PSNR...
> I have just no idea how is it possible to increase PSNR, or even
> increase bits (or both..) during quantization....

With R-D optimal quantization at a fixed quantizer it should be impossible
for a single frame. There is a possibility that interaction with MC causes
the larger file in the bigger picture ... but are you sure the quantizer is
fixed?

Marco