[XviD-devel] R-D "optimal"

[Christoph Lampert]

On Wed, 7 May 2003, Radek Czyz wrote:

> 
> > E.g. one rule of thumb is to set   lambda = 0.85*quant*quant ;
> > and calculate cost function as
> 
> > J(parameters) = sum-of-squared-error + lambda * bits-needed,
> 
> Hi, I just have a small question about this trellis quant.
> When you do that, you compute 'error' as a difference between
> quantized value and new value. However, this 'error' is already
> quantized... sometimes the value was 0.51 rounded to 1, but
> sometimes it was 1.49. Changing both to zero will not mean the same
> error.
> Is there something which stops you from de-quantizing the coefficient
> and comparing it to original?

You mean perform inverse DCT and calculate error in the image domain?
I guess only performance penalties stop us from that. 

gruel