[XviD-devel] mpeg intra quantization

[skal]

   Hey Radek,

On Sat, 2006-07-08 at 18:39 +0930, Radek Czyz wrote:
[...]
> For example, if mpeg matrix entry is 32 at Q=1, a level of 1 is 
> quantized to 1:
> 
> level = ((level << 4) + (intra_matrix[i] >> 1)) / intra_matrix[i];
> level = ((level + quantd) * mult) >> SCALEBITS;
[...]
> 
> In this simple example, changing quantd=0 gives:
> 18:30:11 : <Alex_W> xvid 1.1: 17772 KB 41.8 dB
> 18:30:11 : <Alex_W> xvid quantd = 0: 10813 KB 46.42 dB

   ouch. Just to be sure: this doesn't show up with
   the ASM code, right?

> Can someone (Skal?:) ) explain what was *meant* to be happening here?

   There's a problem indeed: the intended bias (3q/4) isn't spread
   evenly between the two 'divides'. I suggest we replace the above 
   two lines by:

level = ((level<<3) + ((3*quant*intra_matrix[i])>>3)) / (quant*intra_matrix[i]);
 
    Should be about the same, speed-wise. What do you think?

    Also:

> In fact, can someone explain to me just why we don't keep 
> quant*matrx_entry in memory and use it to both quantize and dequantize? 
> Seems to me it should be both more precise (rounding) and faster.

    Yep, that's doable. Just you need to store a set a matrices for
    each 31 possible quant values (oh, and the biases, too).
    Oh, and if you want to store the (1<<FIX)/(Q.M[i]) reciprocal,
    you'll probably need more than 17 bits for the max possible
    denominator (=31*255). Iirc, it's 20bits at least.

   bye!

Skal