[XviD-devel] big trellis bug

[Michael Militzer]

Hi,

I suggest to solve the problem as Skal suggested: Simply introduce a
threshold and don't optimize big levels at all. This a) solves the over-
flow problem and b) prevents trellis from potentially introducing visible
distortion (by 'overoptimizing' the DC component). Very large values are
coded in fixed length codewords anyway, so it's pretty much impossible to
save many bits without introducing huge distortion. I assume it could
only be beneficial to check if a fixed length coded value could also be
VL coded by just little modification (means little distortion but huge
bit savings).

bye,
Michael

Quoting Edouard Gomez <ed.gomez at free.fr>:

> Edouard Gomez (ed.gomez at free.fr) wrote:
> > I'm currently  testing the changes.  I also  introduced TL_SHIFT=10 to
> > avoid overflows.  It doesn't fix the real problem but avoids its nasty
> > effects.
> 
> Ok tests done:
>  - h263 works hopefully as before (output matches exactly if i set
>    TL_SHIFT 16)
>  - MPEG seems to benefit from trellis as well ;-)
> 
> Nota bene, i suffered the chroma bug in trellis for the first time while
> running a mpeg+trellis  test (a red block appeared in  a white zone).  I
> guess we should really leave DC unoptimized for high values.
> 
> May a simple thresholding be enough ?
> 
> -- 
> Edouard Gomez
> _______________________________________________
> XviD-devel mailing list
> XviD-devel at xvid.org
> http://list.xvid.org/mailman/listinfo/xvid-devel
>