[XviD-devel] Adaptive Quantization

Marco Al xvid-devel@xvid.org
Mon, 30 Sep 2002 01:32:17 +0200


Christoph Lampert wrote:

> I changed this MB's quant, because with the new quant, the error
> (MSE) between original DCT coefficients and dequantized coeffifients
> is lower. This doesn't happen often, but it does sometimes.
> (e.g. a value of 20 is better quantized with 4 or 5 than with 3.)

I dont feel like doing the math, but intuitively Id say that the probability of
a bias towards one side of the quantization interval will occur for all the
coefficients when this occurs and you make use of it. Normally the distribution
along the intervals will be noisy, maybe the bias causes an amplification of the
error in the pixel domain?

> So the frame's PSNR should end up very slightly higher. But it
> doesn't, it gets lower. The same happens if I don't use MSE of DCT
> coefficients but of real (motion compensated) YUV data.
> Still the error is lower, but PSNR doesn't get better.

Maybe in these cases the decrease in MSE is so small the precision of the log
calculation isnt high enough to catch it? The only other alternative is a bug in
your code obviously (since PSNR monotonically increases with decreasing MSE).

Marco