[XviD-devel] R-D "optimal"

[Radek Czyz]

> E.g. one rule of thumb is to set   lambda = 0.85*quant*quant ;
> and calculate cost function as

> J(parameters) = sum-of-squared-error + lambda * bits-needed,

Hi, I just have a small question about this trellis quant.
When you do that, you compute 'error' as a difference between
quantized value and new value. However, this 'error' is already
quantized... sometimes the value was 0.51 rounded to 1, but
sometimes it was 1.49. Changing both to zero will not mean the same
error.
Is there something which stops you from de-quantizing the coefficient
and comparing it to original?

Radek